system

Abstract

We provide the system. [Solution] Means of collecting player data, A means for analyzing the collected player data to identify the player's skill level and behavioral patterns, A means for generating the optimal virtual opponent based on the analysis results, Means for adjusting the behavioral patterns of the virtual opponent in accordance with the real-time performance of the player, A system that includes means for collecting player feedback and incorporating it into future generation processes.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a modern online game environment, it is difficult to provide a game experience suitable for each player. In particular, players may not be able to find opponents suitable for their skill levels and play styles, and may be forced to have an inappropriate difficulty level or an unsatisfactory playing experience. Such a situation has an adverse effect on players' satisfaction and continuity in the game, so there is a need for a system that always provides an optimal game experience for individual players.

Means for Solving the Problems

[0005] This invention provides a system that collects player gameplay data, analyzes that data to identify the player's skill level and behavior patterns, and generates an optimal virtual opponent based on the analysis results. This system has a function to adjust the virtual opponent's behavior patterns according to the player's real-time performance, and further collects feedback from the player and reflects it in subsequent generation processes, thereby always providing the player with an optimal gaming experience.

[0006] "Player data" refers to information collected during gameplay, including player actions, skill levels, and selected characters and weapons.

[0007] "Analysis" is the process of using collected player data to identify the skill level and behavioral patterns of individual players.

[0008] A "virtual opponent" refers to an AI character or agent generated based on analysis results, intended for competition or cooperation with the player.

[0009] "Behavioral patterns" refer to the typical actions and choices of a player or virtual opponent, indicating their tendencies in how they behave in specific situations.

[0010] "Feedback" refers to the opinions and evaluations that players provide after playing a game, including their thoughts on the difficulty and enjoyment of their opponents, and is used to improve future gameplay experiences. [Brief explanation of the drawing]

[0011] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3]This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0012] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0013] First, let's explain the terminology used in the following explanation.

[0014] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0015] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0016] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0017] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0019] [First Embodiment]

[0020] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0021] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0024] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0027] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0031] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0032] This invention is a system designed to enable players to enjoy the optimal gaming experience within online games. This system is implemented through the following operations.

[0033] Data collection and analysis

[0034] The server acquires data in real time during the user's game session. The terminal sends the user's input actions, choices made during gameplay, and various metrics to the server. This data includes scores, stage completion times, and statistics on the characters and skills used. The server analyzes this data to identify the user's skill level and gameplay characteristics.

[0035] Generating a virtual opponent

[0036] Based on the analyzed data, the server generates a virtual opponent suited to the user. This virtual opponent has a difficulty level that matches the user's skill level and possesses tactics that can counter the user's play style. For example, a user who is heavily focused on offense will be matched with an opponent who excels in defense.

[0037] Real-time adaptation and adjustment

[0038] During gameplay, the server monitors the user's real-time performance. When a user employs more advanced strategies, the server instantly adjusts the opponent's tactics and difficulty level. This adaptation ensures that users are always challenged at an appropriate level.

[0039] Feedback and Learning

[0040] Users provide feedback on their game experience through an interface presented after the game ends. This feedback includes information about the opponent's strength and how enjoyable the game was. The server collects this feedback and uses it to generate virtual opponents for future matches.

[0041] Explanation of specific examples

[0042] For example, suppose a user is playing a shooting game. This user has an offensive play style and is at an intermediate level. The server generates a virtual opponent that prioritizes defense for this user and adjusts the opponent in real time in response to changes in the user's strategy. This allows the user to improve their skills while always enjoying a fresh and optimized gaming experience.

[0043] The following describes the processing flow.

[0044] Step 1:

[0045] The server retrieves the user's gamer profile information at the start of the game. This information includes the user's past gaming history, current rank, and registered preference settings.

[0046] Step 2:

[0047] The device records user actions in real time during gameplay and sends them to the server. This includes actions such as movement, attacking, and defending, as well as their frequency and effect.

[0048] Step 3:

[0049] The server analyzes the user's skill level and behavioral patterns based on the real-time data it receives. It uses machine learning algorithms to identify the user's strengths and weaknesses in tactics.

[0050] Step 4:

[0051] The server generates a virtual opponent based on the analyzed results. This opponent's skills and tactics are adjusted to the user's level and adapted to the user's expected play style.

[0052] Step 5:

[0053] During gameplay, the server continuously monitors user performance and adjusts the virtual opponent's actions in real time as needed. For example, if a user introduces a new tactic, the opponent's strategy will be changed.

[0054] Step 6:

[0055] Users evaluate their experience through a feedback form after the game ends. The server collects this feedback and incorporates it into the algorithm for generating opponents for the next match.

[0056] Step 7:

[0057] The server collects feedback and game data, updating the system's overall learning database. This data is used to improve the experience for other users and develop new features.

[0058] (Example 1)

[0059] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0060] In online games, there is a need to provide players with an optimal gaming experience tailored to their individual skill levels and strategies, while maintaining consistent engagement and enjoyment. Furthermore, a challenge exists where mismatched opponents or game environments can lead to unsuitable difficulty levels and decreased player satisfaction.

[0061] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0062] In this invention, the server includes means for acquiring user information, means for analyzing the acquired user information to identify the user's skill level and behavioral patterns, and means for the generated virtual battle unit to determine its skill parameters and strategy using a generated AI model. This makes it possible to generate virtual battle units optimized for each user's skills and play style, and to enable real-time adjustments, thereby providing users with a consistently challenging and enjoyable gaming environment.

[0063] "Means of acquiring user information" refers to technologies that collect user behavior data and choices made during gameplay in real time.

[0064] "Means of analysis to identify users' skill levels and behavioral patterns" refers to technologies that identify play skills and unique gameplay styles based on collected user data.

[0065] "Means for generating appropriate virtual battle units" refers to technology that creates opponents with optimal difficulty levels and tactics for the user based on analysis results.

[0066] "Means for adjusting the behavior patterns of virtual battle units according to real-time results" refers to technology that continuously modifies and optimizes the tactics and actions of generated virtual opponents based on the user's current game situation.

[0067] "Means of collecting opinions and reflecting them in the next generation process" refers to technology that collects feedback based on users' game experiences and utilizes it when creating virtual opponents in the future.

[0068] "Means of determining ability parameters and strategies" refers to the technology of using generative AI models to determine the actions and skills that a virtual opponent will perform and use.

[0069] This invention provides a system that highly personalizes the user's gaming experience on an online game platform. By generating and adjusting optimal virtual battle units in real time according to the user's skill level, it ensures a consistently fresh and challenging gaming experience.

[0070] The server uses data received from the device to analyze user behavior in detail while the user plays the game. The device continuously collects data on the actions, choices, and achievements the user makes throughout the game and sends this data to the server. This data includes scores, completion times, and the frequency of use of selected characters and skills.

[0071] The server uses a generative AI model to analyze the received data. This model uses machine learning algorithms to identify the user's skill level and behavioral patterns. Based on this information, the generative AI model generates a virtual battle unit (VRB) suitable for the user. The VRB has ability parameters and strategies that correspond to the user's skill level and is optimized for the user's play style.

[0072] During gameplay, the server monitors user performance in real time. When a user employs a new strategy, the server instantly adjusts the actions and difficulty of the virtual battle unit. This ensures that users always receive a challenge at an appropriate level.

[0073] Furthermore, after a game ends, users can provide feedback through a feedback function. This feedback is then incorporated into the generation of the next virtual match unit to improve the user experience. Through this feedback process, the system is constantly being improved.

[0074] As a concrete example, let's assume a user is playing a shooting game. If this user is an intermediate-level player with an offensive style, the server will generate a virtual battle unit with enhanced defensive capabilities. If the user adopts a new offensive strategy during gameplay, the server will immediately strengthen the defensive tactics of the virtual battle unit.

[0075] Examples of prompts to input into a generating AI model include: "Describe a system that analyzes user gameplay data and adjusts virtual opponents in real time to provide players with the best possible gaming experience in an online game. As a specific example, please explain how to generate an opponent suitable for an attack-oriented intermediate player in a shooting game."

[0076] In this way, we can provide a system that continuously optimizes the gaming experience to meet diverse user needs.

[0077] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0078] Step 1:

[0079] The server receives user gameplay data transmitted in real time from the terminal. Input from the terminal includes user commands, selected options, and gameplay metrics. The server stores this data in temporary storage, preparing it for use in later analysis steps.

[0080] Step 2:

[0081] The server analyzes the saved gameplay data. It processes the received data through an analysis algorithm to generate output that identifies the user's skill level and behavioral patterns. In this process, a machine learning model is used to categorize the user's play style. Specifically, it calculates the frequency of skill usage and the average completion time.

[0082] Step 3:

[0083] The server generates a virtual battle unit corresponding to the user based on the analysis results. The output data from the analysis (user's skill level, behavior patterns) is used as input. The server can use the generated AI model to determine the abilities and strategies of the virtual opponent and create the details of the battle unit to be presented to the user as output.

[0084] Step 4:

[0085] During gameplay, the server continuously monitors the user's real-time performance. By analyzing real-time data as input, the server instantly adjusts the actions of the virtual battle unit when the user adopts new tactics. This ensures that the user is always provided with a battle environment based on the latest tactics.

[0086] Step 5:

[0087] After a game ends, users provide feedback through the interface. The server collects this feedback data as input and incorporates it as adjustment data as output in subsequent generation processes to further improve the user experience. This continuously improves the overall system performance and user experience.

[0088] (Application Example 1)

[0089] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0090] In today's information-saturated world, there is a need to quickly and accurately provide each user with the most relevant information and content. However, conventional technologies have struggled to accurately grasp users' real-time preferences and reactions and dynamically adjust content accordingly. As a result, users may be provided with inappropriate or uninteresting information, leading to a decline in the quality of their experience.

[0091] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0092] In this invention, the server includes means for collecting user information, means for analyzing the collected user information to identify the user's preferences and tendencies, and means for generating optimal virtual recommendation targets based on the analysis results. This makes it possible to provide users with information and content that is appropriate and tailored in real time.

[0093] "User information" refers to data collected by the system from individual users regarding their usage history, preferences, and interactions.

[0094] "Analysis" is the process of identifying patterns and trends based on collected user information to understand user preferences.

[0095] "Preferences" refer to data that indicates the user's preferences and interests regarding specific content or genres.

[0096] "Trends" refer to behavioral patterns in how users consume information and content.

[0097] "Virtual recommendation targets" refer to information, content, or combinations thereof selected by the system based on the user's preferences and tendencies.

[0098] "Real-time adjustment" is a process that instantly updates the information and content provided to be appropriate based on the user's current reactions and circumstances.

[0099] This system is designed to provide users with the most relevant information in content delivery services. The server collects and analyzes user information to identify preferences and trends. Specifically, it records videos watched, selected content, ratings, and other data in real time. It utilizes AWS® (Amazon Web Services), employing Lambda services and DynamoDB for efficient data management.

[0100] Next, using the collected data, a machine learning algorithm in Amazon SageMaker learns the user's preferences and generates optimal virtual recommendations. These recommendations are then adjusted based on the user's current situation and past behavior. AWS SNS service is used to quickly notify the user's device of recommended content, presenting topics of interest.

[0101] For example, users interested in sports will be recommended the latest match highlights and related news. Feedback from users after they have finished viewing this information is collected and incorporated into the recommendation algorithm for future recommendations, resulting in more accurate recommendations. An example of a prompt for the generative AI model is the input format, "Recommend the next best content based on the user's current viewing history."

[0102] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0103] Step 1:

[0104] The server collects user information from the user's device in real time. It receives data such as viewing history, content click information, and rating information as input. Based on this information, the server stores the data in DynamoDB.

[0105] Step 2:

[0106] The server analyzes accumulated user information. The input is user data obtained from DynamoDB. Amazon SageMaker is used within the server to run a machine learning model to identify preferences. The output is information indicating user preferences and trends.

[0107] Step 3:

[0108] The server generates optimal virtual recommendations based on the analyzed results. It uses user preference and trend data as input. It leverages Amazon SageMaker's generation AI model to create a list of recommended content as output.

[0109] Step 4:

[0110] The server uses AWS SNS to notify the user's device of a list of recommended content. The input is encoded recommendation information, and the output is a notification message sent to the user's device. The user receives this message on their device and can view the recommended content.

[0111] Step 5:

[0112] After a user finishes viewing content, the device sends post-viewing feedback to the server. The input consists of ratings and opinions provided by the user, and the output is stored on the server as feedback information.

[0113] Step 6:

[0114] The server analyzes the collected feedback information to use in the next recommendation process. The input is feedback data, and based on this data, it retrains a machine learning model to establish an improved recommendation system as output.

[0115] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0116] This invention relates to a system for recognizing players' emotions in online games and adapting the game experience based on that information. This system comprises the following components.

[0117] Extension of data collection and analysis

[0118] The server collects data to analyze the user's emotional state, in addition to conventional player data. The terminal acquires emotion-related data through user input, facial recognition sensors, and voice tone analysis, and transmits it to the server. This information is used to identify the user's mental state in real time.

[0119] Introducing an emotional engine

[0120] The server uses an emotion engine to analyze the user's emotional state. The emotion engine uses machine learning to infer the user's emotions from the input data and identify how that state affects gameplay. This allows the server to determine whether the user is stressed, enjoying themselves, or frustrated.

[0121] Adaptation of virtual opponents

[0122] Emotional information is reflected in the actions of virtual opponents. The server adjusts the opponent's aggression and changes the game difficulty according to the user's emotional state, allowing the user to play at an optimal stress level. For example, if the server determines that the user is feeling frustrated, it will take measures to lower the difficulty level.

[0123] Customized gaming experience

[0124] The server collects user feedback based on their emotions and customizes scenarios and environments accordingly. If a user frequently exhibits a particular emotional state, the game scenario is adjusted to match, thereby increasing user satisfaction.

[0125] Explanation of specific examples

[0126] For example, if the emotion engine detects that a user is stressed while playing an action game, the server will suggest a more relaxed game environment. This may include reducing the number of enemies or changing the music. Furthermore, once the emotional state improves, the system will revert to the original difficulty level to maintain game balance. This allows users to enjoy a more comfortable gaming experience.

[0127] The following describes the processing flow.

[0128] Step 1:

[0129] The server retrieves basic information, including the user's profile and past play history, at the start of a user's game session. This information is used to understand the user's general play style and preferences.

[0130] Step 2:

[0131] The device collects user action data in real time during gameplay and transmits it to the server. This data includes actions taken in the game, the buttons pressed on the controller, and selections made within the game.

[0132] Step 3:

[0133] The device also uses built-in sensors and a webcam to collect data to measure the user's emotional state. This includes facial expression analysis, voice tone, and heart rate.

[0134] Step 4:

[0135] The server inputs the collected emotional data into the emotion engine, which analyzes the user's emotional state. The emotion engine estimates the user's current state based on multiple emotional categories and identifies levels of stress and joy.

[0136] Step 5:

[0137] The server adjusts the behavior of virtual opponents and the game environment based on the analyzed emotional state. For example, if the user is in a high-tension state, the server may reduce the aggression of virtual opponents or lower the game difficulty.

[0138] Step 6:

[0139] As users continue playing the game, they can intuitively sense the changes displayed on their device. For example, the game music might become more relaxing.

[0140] Step 7:

[0141] The server collects user ratings and feedback after the game ends. This feedback is used to improve the accuracy of the emotion engine and tune the entire system, contributing to improvements in the next game session.

[0142] (Example 2)

[0143] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0144] In modern computer games, a uniform game experience is provided that disregards the player's emotional state, making it difficult to offer an optimal game experience tailored to each individual player's emotions. Furthermore, there is a lack of features to adjust game difficulty and environment based on emotional state, resulting in insufficient player satisfaction.

[0145] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0146] In this invention, the server includes means for collecting user information, means for analyzing the collected user information and identifying the user's emotional state, and means for generating an optimal virtual opponent based on the analysis results. This enables the real-time adaptation of the game experience according to the player's emotional state.

[0147] "User information" refers to data related to players participating in the game, including information such as operation data, facial expressions, voice, and actions during gameplay.

[0148] "Emotional state" refers to the psychological and emotional state a user exhibits during gameplay, and is used to identify feelings such as joy, anger, tension, and excitement.

[0149] A "virtual opponent" is an AI-based character that the player faces in the game, designed to react appropriately to the player.

[0150] "Behavioral characteristics" refer to the behavioral traits and attributes exhibited by a virtual opponent, including aggression, tactics, and reaction speed.

[0151] "Feedback" refers to information such as comments and reactions from users regarding their game experience, which will be used to adjust the game in the future.

[0152] A "scenario" refers to the story and setting provided to the user within a game, and it changes according to their emotional state.

[0153] "Environment" refers to the overall atmosphere and situation that players experience, including elements such as sound, visual effects, and difficulty within the game.

[0154] "Cooperative players" are AI-based characters that appear in the game to support the player and are designed to enhance the player experience.

[0155] This invention relates to a system that recognizes the emotional state of a user through online games and optimizes the game experience based on that information. This system mainly consists of three main parts: collection of user information, analysis of emotional state, and adjustment of the game experience.

[0156] The device includes hardware for collecting user operation data, facial expression data, and voice data. In particular, it uses devices such as cameras and microphones to capture emotional elements while the user is playing the game. Computer vision libraries (e.g., OpenCV) are used for facial expression recognition, and acoustic processing libraries (e.g., LibROSA) are used for voice tone analysis.

[0157] Emotion-related data collected on the device is sent to the server using a secure communication protocol (e.g., TLS / SSL).

[0158] The server uses machine learning models to analyze the data it receives. This analysis employs deep learning frameworks such as TENSORFLOW® and PyTorch. This allows the server to infer the emotional state the user exhibits during gameplay and identify stress, enjoyment, frustration, and other related emotions.

[0159] The server automatically adjusts game settings based on the user's emotional state. For example, if the user is feeling stressed, this might include reducing the number of enemies in the game or changing the music to something more relaxing. These adjustments are made in real time to improve the user experience.

[0160] As a concrete example, suppose a user feels tense while playing, and the device captures this tension in their facial expression. The server analyzes the tension and adjusts the game's difficulty level to alleviate it. As a result, user satisfaction improves.

[0161] Examples of prompts include, "Adjust the game difficulty based on user sentiment data," or "If the player is dissatisfied, suggest how to change the game scenario."

[0162] In this way, the present invention makes it possible to provide a flexible gaming environment that responds to the user's emotions.

[0163] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0164] Step 1:

[0165] The device monitors the user's interactions and collects operation data, facial expression data, and voice data using various sensors. This input data includes keyboard and mouse operation speed, subtle facial expression changes, and voice tone and pressure. This data is used as basic information to infer emotional states.

[0166] Step 2:

[0167] The device transmits collected data to the server in real time. Encrypted communication protocols are used for data transmission to securely protect user data. Input data is passed to the server as a series of numerical signals, where feature extraction is performed. This results in the output of numerical information representing important sentiment indicators.

[0168] Step 3:

[0169] The server uses the transmitted features to analyze the emotional state based on a generative AI model. The machine learning algorithm performs data calculations on the received features to infer emotional categories such as "joy," "anger," and "surprise." The analysis results output a numerical model of the user's emotional state.

[0170] Step 4:

[0171] The server automatically adjusts the game environment based on the analysis results. Specifically, if it determines that the user is in a high-stress state, it changes the in-game settings. This may include reducing the number of enemy characters, changing the music, or adjusting the visual effects. The input is a numerical model of the emotional state, and the output is a profile of the adjusted game settings.

[0172] Step 5:

[0173] The user continues playing in a game environment adjusted by the server. Interactions within the adjusted game trigger further feedback data collection, generating prompts. For example, a prompt might include a task such as "Adjust game difficulty based on user sentiment data." This feedback is used to customize the environment for subsequent play sessions.

[0174] (Application Example 2)

[0175] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0176] In modern homes, adapting the living environment to the emotional state of each individual is a significant challenge. Conventional household appliances and life support robots often fail to adequately understand the user's emotions and provide services accordingly, resulting in a lack of relief from user stress and dissatisfaction. This invention aims to solve these problems and realize a comfortable living environment within the home.

[0177] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0178] In this invention, the server includes means for collecting emotion-related information, means for analyzing the collected emotion-related information and identifying the user's mental state, and means for generating an optimal virtual dialogue partner based on the mental state. This makes it possible to understand the user's emotions in real time and to provide optimal responses and adjust the environment accordingly.

[0179] "Emotion-related information" refers to data that indicates a user's psychological state, obtained from their voice, facial expressions, behavior, etc.

[0180] "Means of collection" refers to systems that use sensors, cameras, microphones, etc., to acquire information related to the user's emotions.

[0181] "Means of analysis to identify the user's mental state" refers to the process of identifying and classifying emotions and psychological states from collected data using machine learning algorithms and analytical software.

[0182] "Means for generating the optimal virtual dialogue target" refers to a system that, based on analysis results, designs and provides responses and dialogue content adapted to the user's emotions.

[0183] "Means of adjusting behavior based on mental state" refers to technologies that dynamically change the operation of a system or the settings of the environment in accordance with the user's emotions.

[0184] "Means for collecting feedback and incorporating it into future generation processes" refers to a system that accumulates user responses and opinions and uses them to improve the quality of future services and responses.

[0185] The system for carrying out the present invention is executed through a consumer robot installed in a home. It comprises the elements detailed below.

[0186] The server collects user emotion-related information using devices such as facial recognition cameras and microphones. This data is used to analyze the user's voice tone and facial expressions. Specifically, it uses facial recognition cameras such as the RealSense Depth Camera and facial analysis algorithms such as OpenCV and the Emotion Detection API to accurately capture facial data.

[0187] The collected data is analyzed on the server using machine learning algorithms, such as emotion recognition models based on TensorFlow or PyTorch. This allows the server to identify the user's mental state in real time.

[0188] Based on the analyzed mental state, the server provides the user with the most suitable virtual interaction. For example, if the user is feeling stressed, the server will provide voice responses through a smart speaker, suggesting relaxing music or advice. A smart speaker like Amazon Echo is used for this purpose.

[0189] Furthermore, the system collects user feedback and uses it as learning data to make future responses more personalized. This allows for continuous improvement of the user experience.

[0190] As a concrete example, when the server detects a user's facial expression indicating fatigue upon returning from work, it prompts the user with "Do you want to relax?" and plays appropriate music based on their response. As an example of a specific prompt sentence, the AI ​​generating model is input with the following sentence: "Design a method to recognize the emotional state of family members at home in real time and suggest appropriate chores or music. Explain the functions of the smart home system based on this." The system's response is then checked.

[0191] In this way, it becomes possible to provide a flexible assistant system that can respond to diverse living situations within the home, thereby supporting a comfortable living environment.

[0192] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0193] Step 1:

[0194] The device collects user emotion-related information using a facial recognition camera and microphone. The input data consists of the user's facial expressions and voice, which are captured by the sensors. This data is then transmitted to a server in real time.

[0195] Step 2:

[0196] The server analyzes the received facial expression and audio data. This analysis is performed using OpenCV and the Emotion Detection API, and a machine learning model (e.g., TensorFlow) is used to infer the user's emotional state. The input data consists of raw facial expressions and audio, and the output is a identified mental state (e.g., stress, happiness).

[0197] Step 3:

[0198] Based on the user's emotional state, the server generates an appropriate response. A generative AI model is used to select the most appropriate music or suggestions. The input to this process is the identified emotional state, and the output is the suggested action (e.g., music playback, voice guidance).

[0199] Step 4:

[0200] The server sends the selected response to the terminal, which then displays it to the user. This can be done through a smart speaker, for example, by playing music or sending a voice message. The input here is the action data generated in the previous step, and the output is the actual response action to the user.

[0201] Step 5:

[0202] Users can provide feedback on the responses they receive. The server collects this feedback and stores it as training data for the system. The input is the feedback content, and the output is the improved response model. This feedback-based improvement process will eventually lead to more personalized services.

[0203] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0204] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search)<url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0205] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0206] [Second Embodiment]

[0207] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0208] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0209] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0210] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0211] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0212] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0213] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0214] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0215] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0216] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0217] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0218] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0219] This invention is a system designed to enable players to enjoy the optimal gaming experience within online games. This system is implemented through the following operations.

[0220] Data collection and analysis

[0221] The server acquires data in real time during the user's game session. The terminal sends the user's input actions, choices made during gameplay, and various metrics to the server. This data includes scores, stage completion times, and statistics on the characters and skills used. The server analyzes this data to identify the user's skill level and gameplay characteristics.

[0222] Generating a virtual opponent

[0223] Based on the analyzed data, the server generates a virtual opponent suited to the user. This virtual opponent has a difficulty level that matches the user's skill level and possesses tactics that can counter the user's play style. For example, a user who is heavily focused on offense will be matched with an opponent who excels in defense.

[0224] Real-time adaptation and adjustment

[0225] During gameplay, the server monitors the user's real-time performance. When a user employs more advanced strategies, the server instantly adjusts the opponent's tactics and difficulty level. This adaptation ensures that users are always challenged at an appropriate level.

[0226] Feedback and Learning

[0227] Users provide feedback on their game experience through an interface presented after the game ends. This feedback includes information about the opponent's strength and how enjoyable the game was. The server collects this feedback and uses it to generate virtual opponents for future matches.

[0228] Explanation of specific examples

[0229] For example, suppose a user is playing a shooting game. This user has an offensive play style and is at an intermediate level. The server generates a virtual opponent that prioritizes defense for this user and adjusts the opponent in real time in response to changes in the user's strategy. This allows the user to improve their skills while always enjoying a fresh and optimized gaming experience.

[0230] The following describes the processing flow.

[0231] Step 1:

[0232] The server retrieves the user's gamer profile information at the start of the game. This information includes the user's past gaming history, current rank, and registered preference settings.

[0233] Step 2:

[0234] The device records user actions in real time during gameplay and sends them to the server. This includes actions such as movement, attacking, and defending, as well as their frequency and effect.

[0235] Step 3:

[0236] The server analyzes the user's skill level and behavioral patterns based on the real-time data it receives. It uses machine learning algorithms to identify the user's strengths and weaknesses in tactics.

[0237] Step 4:

[0238] The server generates a virtual opponent based on the analyzed results. This opponent's skills and tactics are adjusted to the user's level and adapted to the user's expected play style.

[0239] Step 5:

[0240] During gameplay, the server continuously monitors user performance and adjusts the virtual opponent's actions in real time as needed. For example, if a user introduces a new tactic, the opponent's strategy will be changed.

[0241] Step 6:

[0242] Users evaluate their experience through a feedback form after the game ends. The server collects this feedback and incorporates it into the algorithm for generating opponents for the next match.

[0243] Step 7:

[0244] The server collects feedback and game data, updating the system's overall learning database. This data is used to improve the experience for other users and develop new features.

[0245] (Example 1)

[0246] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0247] In online games, there is a need to provide players with an optimal gaming experience tailored to their individual skill levels and strategies, while maintaining consistent engagement and enjoyment. Furthermore, a challenge exists where mismatched opponents or game environments can lead to unsuitable difficulty levels and decreased player satisfaction.

[0248] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0249] In this invention, the server includes means for acquiring user information, means for analyzing the acquired user information to identify the user's skill level and behavioral patterns, and means for the generated virtual battle unit to determine its skill parameters and strategy using a generated AI model. This makes it possible to generate virtual battle units optimized for each user's skills and play style, and to enable real-time adjustments, thereby providing users with a consistently challenging and enjoyable gaming environment.

[0250] "Means of acquiring user information" refers to technologies that collect user behavior data and choices made during gameplay in real time.

[0251] "Means of analysis to identify users' skill levels and behavioral patterns" refers to technologies that identify play skills and unique gameplay styles based on collected user data.

[0252] "Means for generating appropriate virtual battle units" refers to technology that creates opponents with optimal difficulty levels and tactics for the user based on analysis results.

[0253] "Means for adjusting the behavior patterns of virtual battle units according to real-time results" refers to technology that continuously modifies and optimizes the tactics and actions of generated virtual opponents based on the user's current game situation.

[0254] "Means of collecting opinions and reflecting them in the next generation process" refers to technology that collects feedback based on users' game experiences and utilizes it when creating virtual opponents in the future.

[0255] "Means of determining ability parameters and strategies" refers to the technology of using generative AI models to determine the actions and skills that a virtual opponent will perform and use.

[0256] This invention provides a system that highly personalizes the user's gaming experience on an online game platform. By generating and adjusting optimal virtual battle units in real time according to the user's skill level, it ensures a consistently fresh and challenging gaming experience.

[0257] The server uses data received from the device to analyze user behavior in detail while the user plays the game. The device continuously collects data on the actions, choices, and achievements the user makes throughout the game and sends this data to the server. This data includes scores, completion times, and the frequency of use of selected characters and skills.

[0258] The server uses a generative AI model to analyze the received data. This model uses machine learning algorithms to identify the user's skill level and behavioral patterns. Based on this information, the generative AI model generates a virtual battle unit (VRB) suitable for the user. The VRB has ability parameters and strategies that correspond to the user's skill level and is optimized for the user's play style.

[0259] During gameplay, the server monitors user performance in real time. When a user employs a new strategy, the server instantly adjusts the actions and difficulty of the virtual battle unit. This ensures that users always receive a challenge at an appropriate level.

[0260] Furthermore, after a game ends, users can provide feedback through a feedback function. This feedback is then incorporated into the generation of the next virtual match unit to improve the user experience. Through this feedback process, the system is constantly being improved.

[0261] As a concrete example, let's assume a user is playing a shooting game. If this user is an intermediate-level player with an offensive style, the server will generate a virtual battle unit with enhanced defensive capabilities. If the user adopts a new offensive strategy during gameplay, the server will immediately strengthen the defensive tactics of the virtual battle unit.

[0262] Examples of prompts to input into a generating AI model include: "Describe a system that analyzes user gameplay data and adjusts virtual opponents in real time to provide players with the best possible gaming experience in an online game. As a specific example, please explain how to generate an opponent suitable for an attack-oriented intermediate player in a shooting game."

[0263] In this way, we can provide a system that continuously optimizes the gaming experience to meet diverse user needs.

[0264] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0265] Step 1:

[0266] The server receives user gameplay data transmitted in real time from the terminal. Input from the terminal includes user commands, selected options, and gameplay metrics. The server stores this data in temporary storage, preparing it for use in later analysis steps.

[0267] Step 2:

[0268] The server analyzes the saved gameplay data. It processes the received data through an analysis algorithm to generate output that identifies the user's skill level and behavioral patterns. In this process, a machine learning model is used to categorize the user's play style. Specifically, it calculates the frequency of skill usage and the average completion time.

[0269] Step 3:

[0270] The server generates a virtual battle unit corresponding to the user based on the analysis results. The output data from the analysis (user's skill level, behavior patterns) is used as input. The server can use the generated AI model to determine the abilities and strategies of the virtual opponent and create the details of the battle unit to be presented to the user as output.

[0271] Step 4:

[0272] During gameplay, the server continuously monitors the user's real-time performance. By analyzing real-time data as input, the server instantly adjusts the actions of the virtual battle unit when the user adopts new tactics. This ensures that the user is always provided with a battle environment based on the latest tactics.

[0273] Step 5:

[0274] After a game ends, users provide feedback through the interface. The server collects this feedback data as input and incorporates it as adjustment data as output in subsequent generation processes to further improve the user experience. This continuously improves the overall system performance and user experience.

[0275] (Application Example 1)

[0276] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0277] In modern times, in the midst of a vast flood of information, it is required to quickly and accurately provide the most suitable information and content for individual users. However, with conventional technologies, it has been difficult to accurately grasp the real-time preferences and reactions of users and dynamically adjust appropriate content. For this reason, there is a problem that information inappropriate or of little interest to the user may be provided, resulting in a decline in the quality of the experience.

[0278] The specific processing by the specific processing unit 290 of the data processing apparatus 12 in Application Example 1 is realized by the following respective means.

[0279] In this invention, the server includes means for collecting user information, means for analyzing the collected user information to identify the preferences and tendencies of the user, and means for generating an optimal virtual recommendation target based on the analysis result. Thereby, it becomes possible to provide information and content that is appropriate and adjusted in real time for the user.

[0280] "User information" is data on usage history, preferences, and interactions collected by the system from individual users.

[0281] "Analysis" is an act of identifying patterns and tendencies based on the collected user information and understanding the user's preferences.

[0282] "Preference" is data indicating the direction of the user's preferences and interests towards specific content or genres.

[0283] "Tendency" is the behavior pattern of how the user consumes information and content.

[0284] "Virtual recommendation target" is information and content selected by the system based on the user's preferences and tendencies, or a combination thereof.

[0285] "Real-time adjustment" is a process of immediately updating the provided information and content to appropriate ones according to the current reactions and situations of the user.

[0286] This system is designed to provide the most suitable information to users in the content delivery service. The server collects information about users, analyzes it to identify preferences and trends. Specifically, it records in real-time videos viewed by users, selected content, evaluations, etc. It uses AWS (Amazon Web Services) and utilizes Lambda service and DynamoDB to efficiently manage data.

[0287] Next, based on the collected data, it uses Amazon SageMaker for the machine learning algorithm to learn the user's preferences and generate optimal virtual recommendation targets. These recommendation targets are adjusted based on the user's current situation and past behaviors. It uses the AWS SNS service to quickly notify the user's device of the recommended content and present content of interest.

[0288] As a specific example, for users interested in sports, it recommends the latest game highlights and related news. By collecting feedback after the users have viewed this information and reflecting it in the subsequent recommendation algorithm, more accurate recommendations are made. Also, as an example of the prompt text for the generated AI model, an input format of "Please recommend the next optimal content based on the user's current viewing history" is used.

[0289] The flow of the specific process in Application Example 1 will be described using Figure 12.

[0290] Step 1:

[0291] The server collects user information in real-time from the user's device. As input, it receives data such as viewing history, content click information, evaluation information, etc. Based on this information, the server accumulates the data in DynamoDB.

[0292] Step 2:

[0293] The server analyzes accumulated user information. The input is user data obtained from DynamoDB. Amazon SageMaker is used within the server to run a machine learning model to identify preferences. The output is information indicating user preferences and trends.

[0294] Step 3:

[0295] The server generates optimal virtual recommendations based on the analyzed results. It uses user preference and trend data as input. It leverages Amazon SageMaker's generation AI model to create a list of recommended content as output.

[0296] Step 4:

[0297] The server uses AWS SNS to notify the user's device of a list of recommended content. The input is encoded recommendation information, and the output is a notification message sent to the user's device. The user receives this message on their device and can view the recommended content.

[0298] Step 5:

[0299] After a user finishes viewing content, the device sends post-viewing feedback to the server. The input consists of ratings and opinions provided by the user, and the output is stored on the server as feedback information.

[0300] Step 6:

[0301] The server analyzes the collected feedback information to use in the next recommendation process. The input is feedback data, and based on this data, it retrains a machine learning model to establish an improved recommendation system as output.

[0302] Furthermore, an emotion engine for estimating the user's emotions may be combined. That is, the specific processing unit 290 may estimate the user's emotions using the emotion recognition model 59 and perform specific processing using the user's emotions.

[0303] The present invention is a system for recognizing a player's emotions in an online game and adapting the game experience based on that information. This system has the following configuration.

[0304] Expansion of Data Collection and Analysis

[0305] In addition to conventional player data, the server collects data for analyzing the user's emotional state. The terminal obtains emotion-related data through the user's input operations, an expression recognition sensor, voice tone analysis, etc., and transmits it to the server. This information is used to identify the user's mental state in real time.

[0306] Introduction of an Emotion Engine

[0307] The server analyzes the user's emotional state using an emotion engine. The emotion engine uses machine learning to infer the user's emotions from the input data and identify how that state affects the game play. This determines whether the user is feeling stressed, enjoying themselves, or dissatisfied.

[0308] Adaptation of Virtual Opponents

[0309] The emotion information is reflected in the actions of the virtual opponent. The server adjusts the aggressiveness of the opponent or changes the game difficulty according to the user's emotional state so that the user can play at an optimal stress level. For example, if the user is determined to be feeling frustrated, measures are taken to lower the difficulty.

[0310] Customized Game Experience

[0311] The server collects user feedback based on their emotions and customizes scenarios and environments accordingly. If a user frequently exhibits a particular emotional state, the game scenario is adjusted to match, thereby increasing user satisfaction.

[0312] Explanation of specific examples

[0313] For example, if the emotion engine detects that a user is stressed while playing an action game, the server will suggest a more relaxed game environment. This may include reducing the number of enemies or changing the music. Furthermore, once the emotional state improves, the system will revert to the original difficulty level to maintain game balance. This allows users to enjoy a more comfortable gaming experience.

[0314] The following describes the processing flow.

[0315] Step 1:

[0316] The server retrieves basic information, including the user's profile and past play history, at the start of a user's game session. This information is used to understand the user's general play style and preferences.

[0317] Step 2:

[0318] The device collects user action data in real time during gameplay and transmits it to the server. This data includes actions taken in the game, the buttons pressed on the controller, and selections made within the game.

[0319] Step 3:

[0320] The device also uses built-in sensors and a webcam to collect data to measure the user's emotional state. This includes facial expression analysis, voice tone, and heart rate.

[0321] Step 4:

[0322] The server inputs the collected emotional data into the emotion engine, which analyzes the user's emotional state. The emotion engine estimates the user's current state based on multiple emotional categories and identifies levels of stress and joy.

[0323] Step 5:

[0324] The server adjusts the behavior of virtual opponents and the game environment based on the analyzed emotional state. For example, if the user is in a high-tension state, the server may reduce the aggression of virtual opponents or lower the game difficulty.

[0325] Step 6:

[0326] As users continue playing the game, they can intuitively sense the changes displayed on their device. For example, the game music might become more relaxing.

[0327] Step 7:

[0328] The server collects user ratings and feedback after the game ends. This feedback is used to improve the accuracy of the emotion engine and tune the entire system, contributing to improvements in the next game session.

[0329] (Example 2)

[0330] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0331] In modern computer games, a uniform game experience is provided that disregards the player's emotional state, making it difficult to offer an optimal game experience tailored to each individual player's emotions. Furthermore, there is a lack of features to adjust game difficulty and environment based on emotional state, resulting in insufficient player satisfaction.

[0332] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0333] In this invention, the server includes means for collecting user information, means for analyzing the collected user information and identifying the user's emotional state, and means for generating an optimal virtual opponent based on the analysis results. This enables the real-time adaptation of the game experience according to the player's emotional state.

[0334] "User information" refers to data related to players participating in the game, including information such as operation data, facial expressions, voice, and actions during gameplay.

[0335] "Emotional state" refers to the psychological and emotional state a user exhibits during gameplay, and is used to identify feelings such as joy, anger, tension, and excitement.

[0336] A "virtual opponent" is an AI-based character that the player faces in the game, designed to react appropriately to the player.

[0337] "Behavioral characteristics" refer to the behavioral traits and attributes exhibited by a virtual opponent, including aggression, tactics, and reaction speed.

[0338] "Feedback" refers to information such as comments and reactions from users regarding their game experience, which will be used to adjust the game in the future.

[0339] A "scenario" refers to the story and setting provided to the user within a game, and it changes according to their emotional state.

[0340] "Environment" refers to the overall atmosphere and situation that players experience, including elements such as sound, visual effects, and difficulty within the game.

[0341] "Cooperative players" are AI-based characters that appear in the game to support the player and are designed to enhance the player experience.

[0342] This invention relates to a system that recognizes the emotional state of a user through online games and optimizes the game experience based on that information. This system mainly consists of three main parts: collection of user information, analysis of emotional state, and adjustment of the game experience.

[0343] The device includes hardware for collecting user operation data, facial expression data, and voice data. In particular, it uses devices such as cameras and microphones to capture emotional elements while the user is playing the game. Computer vision libraries (e.g., OpenCV) are used for facial expression recognition, and acoustic processing libraries (e.g., LibROSA) are used for voice tone analysis.

[0344] Emotion-related data collected on the device is sent to the server using a secure communication protocol (e.g., TLS / SSL).

[0345] The server uses machine learning models to analyze the data it receives. This analysis employs deep learning frameworks such as TensorFlow and PyTorch. This allows the server to infer the emotional state the user exhibits during gameplay and identify stress, enjoyment, frustration, and other related emotions.

[0346] The server automatically adjusts game settings based on the user's emotional state. For example, if the user is feeling stressed, this might include reducing the number of enemies in the game or changing the music to something more relaxing. These adjustments are made in real time to improve the user experience.

[0347] As a concrete example, suppose a user feels tense while playing, and the device captures this tension in their facial expression. The server analyzes the tension and adjusts the game's difficulty level to alleviate it. As a result, user satisfaction improves.

[0348] Examples of prompts include, "Adjust the game difficulty based on user sentiment data," or "If the player is dissatisfied, suggest how to change the game scenario."

[0349] In this way, the present invention makes it possible to provide a flexible gaming environment that responds to the user's emotions.

[0350] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0351] Step 1:

[0352] The device monitors the user's interactions and collects operation data, facial expression data, and voice data using various sensors. This input data includes keyboard and mouse operation speed, subtle facial expression changes, and voice tone and pressure. This data is used as basic information to infer emotional states.

[0353] Step 2:

[0354] The device transmits collected data to the server in real time. Encrypted communication protocols are used for data transmission to securely protect user data. Input data is passed to the server as a series of numerical signals, where feature extraction is performed. This results in the output of numerical information representing important sentiment indicators.

[0355] Step 3:

[0356] The server uses the transmitted features to analyze the emotional state based on a generative AI model. The machine learning algorithm performs data calculations on the received features to infer emotional categories such as "joy," "anger," and "surprise." The analysis results output a numerical model of the user's emotional state.

[0357] Step 4:

[0358] The server automatically adjusts the game environment based on the analysis results. Specifically, if it determines that the user is in a high-stress state, it changes the in-game settings. This may include reducing the number of enemy characters, changing the music, or adjusting the visual effects. The input is a numerical model of the emotional state, and the output is a profile of the adjusted game settings.

[0359] Step 5:

[0360] The user continues playing in a game environment adjusted by the server. Interactions within the adjusted game trigger further feedback data collection, generating prompts. For example, a prompt might include a task such as "Adjust game difficulty based on user sentiment data." This feedback is used to customize the environment for subsequent play sessions.

[0361] (Application Example 2)

[0362] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0363] In modern homes, adapting the living environment to the emotional state of each individual is a significant challenge. Conventional household appliances and life support robots often fail to adequately understand the user's emotions and provide services accordingly, resulting in a lack of relief from user stress and dissatisfaction. This invention aims to solve these problems and realize a comfortable living environment within the home.

[0364] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0365] In this invention, the server includes means for collecting emotion-related information, means for analyzing the collected emotion-related information and identifying the user's mental state, and means for generating an optimal virtual dialogue partner based on the mental state. This makes it possible to understand the user's emotions in real time and to provide optimal responses and adjust the environment accordingly.

[0366] "Emotion-related information" refers to data that indicates a user's psychological state, obtained from their voice, facial expressions, behavior, etc.

[0367] "Means of collection" refers to systems that use sensors, cameras, microphones, etc., to acquire information related to the user's emotions.

[0368] "Means of analysis to identify the user's mental state" refers to the process of identifying and classifying emotions and psychological states from collected data using machine learning algorithms and analytical software.

[0369] "Means for generating the optimal virtual dialogue target" refers to a system that, based on analysis results, designs and provides responses and dialogue content adapted to the user's emotions.

[0370] "Means of adjusting behavior based on mental state" refers to technologies that dynamically change the operation of a system or the settings of the environment in accordance with the user's emotions.

[0371] "Means for collecting feedback and incorporating it into future generation processes" refers to a system that accumulates user responses and opinions and uses them to improve the quality of future services and responses.

[0372] The system for carrying out the present invention is executed through a consumer robot installed in a home. It comprises the elements detailed below.

[0373] The server collects user emotion-related information using devices such as facial recognition cameras and microphones. This data is used to analyze the user's voice tone and facial expressions. Specifically, it uses facial recognition cameras such as the RealSense Depth Camera and facial analysis algorithms such as OpenCV and the Emotion Detection API to accurately capture facial data.

[0374] The collected data is analyzed on the server using machine learning algorithms, such as emotion recognition models based on TensorFlow or PyTorch. This allows the server to identify the user's mental state in real time.

[0375] Based on the analyzed mental state, the server provides the user with the most suitable virtual interaction. For example, if the user is feeling stressed, the server will provide voice responses through a smart speaker, suggesting relaxing music or advice. A smart speaker like Amazon Echo is used for this purpose.

[0376] Furthermore, the system collects user feedback and uses it as learning data to make future responses more personalized. This allows for continuous improvement of the user experience.

[0377] As a concrete example, when the server detects a user's facial expression indicating fatigue upon returning from work, it prompts the user with "Do you want to relax?" and plays appropriate music based on their response. As an example of a specific prompt sentence, the AI ​​generating model is input with the following sentence: "Design a method to recognize the emotional state of family members at home in real time and suggest appropriate chores or music. Explain the functions of the smart home system based on this." The system's response is then checked.

[0378] In this way, it becomes possible to provide a flexible assistant system that can respond to diverse living situations within the home, thereby supporting a comfortable living environment.

[0379] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0380] Step 1:

[0381] The device collects user emotion-related information using a facial recognition camera and microphone. The input data consists of the user's facial expressions and voice, which are captured by the sensors. This data is then transmitted to a server in real time.

[0382] Step 2:

[0383] The server analyzes the received facial expression and audio data. This analysis is performed using OpenCV and the Emotion Detection API, and a machine learning model (e.g., TensorFlow) is used to infer the user's emotional state. The input data consists of raw facial expressions and audio, and the output is a identified mental state (e.g., stress, happiness).

[0384] Step 3:

[0385] Based on the user's emotional state, the server generates an appropriate response. A generative AI model is used to select the most appropriate music or suggestions. The input to this process is the identified emotional state, and the output is the suggested action (e.g., music playback, voice guidance).

[0386] Step 4:

[0387] The server sends the selected response to the terminal, which then displays it to the user. This can be done through a smart speaker, for example, by playing music or sending a voice message. The input here is the action data generated in the previous step, and the output is the actual response action to the user.

[0388] Step 5:

[0389] Users can provide feedback on the responses they receive. The server collects this feedback and stores it as training data for the system. The input is the feedback content, and the output is the improved response model. This feedback-based improvement process will eventually lead to more personalized services.

[0390] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0391] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0392] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0393] [Third Embodiment]

[0394] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0395] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0396] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0397] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0398] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0399] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0400] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0401] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0402] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0403] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0404] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0405] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0406] This invention is a system designed to enable players to enjoy the optimal gaming experience within online games. This system is implemented through the following operations.

[0407] Data collection and analysis

[0408] The server acquires data in real time during the user's game session. The terminal sends the user's input actions, choices made during gameplay, and various metrics to the server. This data includes scores, stage completion times, and statistics on the characters and skills used. The server analyzes this data to identify the user's skill level and gameplay characteristics.

[0409] Generating a virtual opponent

[0410] Based on the analyzed data, the server generates a virtual opponent suited to the user. This virtual opponent has a difficulty level that matches the user's skill level and possesses tactics that can counter the user's play style. For example, a user who is heavily focused on offense will be matched with an opponent who excels in defense.

[0411] Real-time adaptation and adjustment

[0412] During gameplay, the server monitors the user's real-time performance. When a user employs more advanced strategies, the server instantly adjusts the opponent's tactics and difficulty level. This adaptation ensures that users are always challenged at an appropriate level.

[0413] Feedback and Learning

[0414] Users provide feedback on their game experience through an interface presented after the game ends. This feedback includes information about the opponent's strength and how enjoyable the game was. The server collects this feedback and uses it to generate virtual opponents for future matches.

[0415] Explanation of specific examples

[0416] For example, suppose a user is playing a shooting game. This user has an offensive play style and is at an intermediate level. The server generates a virtual opponent that prioritizes defense for this user and adjusts the opponent in real time in response to changes in the user's strategy. This allows the user to improve their skills while always enjoying a fresh and optimized gaming experience.

[0417] The following describes the processing flow.

[0418] Step 1:

[0419] The server retrieves the user's gamer profile information at the start of the game. This information includes the user's past gaming history, current rank, and registered preference settings.

[0420] Step 2:

[0421] The device records user actions in real time during gameplay and sends them to the server. This includes actions such as movement, attacking, and defending, as well as their frequency and effect.

[0422] Step 3:

[0423] The server analyzes the user's skill level and behavioral patterns based on the real-time data it receives. It uses machine learning algorithms to identify the user's strengths and weaknesses in tactics.

[0424] Step 4:

[0425] The server generates a virtual opponent based on the analyzed results. This opponent's skills and tactics are adjusted to the user's level and adapted to the user's expected play style.

[0426] Step 5:

[0427] During gameplay, the server continuously monitors user performance and adjusts the virtual opponent's actions in real time as needed. For example, if a user introduces a new tactic, the opponent's strategy will be changed.

[0428] Step 6:

[0429] Users evaluate their experience through a feedback form after the game ends. The server collects this feedback and incorporates it into the algorithm for generating opponents for the next match.

[0430] Step 7:

[0431] The server collects feedback and game data, updating the system's overall learning database. This data is used to improve the experience for other users and develop new features.

[0432] (Example 1)

[0433] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0434] In online games, there is a need to provide players with an optimal gaming experience tailored to their individual skill levels and strategies, while maintaining consistent engagement and enjoyment. Furthermore, a challenge exists where mismatched opponents or game environments can lead to unsuitable difficulty levels and decreased player satisfaction.

[0435] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0436] In this invention, the server includes means for acquiring user information, means for analyzing the acquired user information to identify the user's skill level and behavioral patterns, and means for the generated virtual battle unit to determine its skill parameters and strategy using a generated AI model. This makes it possible to generate virtual battle units optimized for each user's skills and play style, and to enable real-time adjustments, thereby providing users with a consistently challenging and enjoyable gaming environment.

[0437] "Means of acquiring user information" refers to technologies that collect user behavior data and choices made during gameplay in real time.

[0438] "Means of analysis to identify users' skill levels and behavioral patterns" refers to technologies that identify play skills and unique gameplay styles based on collected user data.

[0439] "Means for generating appropriate virtual battle units" refers to technology that creates opponents with optimal difficulty levels and tactics for the user based on analysis results.

[0440] "Means for adjusting the behavior patterns of virtual battle units according to real-time results" refers to technology that continuously modifies and optimizes the tactics and actions of generated virtual opponents based on the user's current game situation.

[0441] "Means of collecting opinions and reflecting them in the next generation process" refers to technology that collects feedback based on users' game experiences and utilizes it when creating virtual opponents in the future.

[0442] "Means of determining ability parameters and strategies" refers to the technology of using generative AI models to determine the actions and skills that a virtual opponent will perform and use.

[0443] This invention provides a system that highly personalizes the user's gaming experience on an online game platform. By generating and adjusting optimal virtual battle units in real time according to the user's skill level, it ensures a consistently fresh and challenging gaming experience.

[0444] The server uses data received from the device to analyze user behavior in detail while the user plays the game. The device continuously collects data on the actions, choices, and achievements the user makes throughout the game and sends this data to the server. This data includes scores, completion times, and the frequency of use of selected characters and skills.

[0445] The server uses a generative AI model to analyze the received data. This model uses machine learning algorithms to identify the user's skill level and behavioral patterns. Based on this information, the generative AI model generates a virtual battle unit (VRB) suitable for the user. The VRB has ability parameters and strategies that correspond to the user's skill level and is optimized for the user's play style.

[0446] During gameplay, the server monitors user performance in real time. When a user employs a new strategy, the server instantly adjusts the actions and difficulty of the virtual battle unit. This ensures that users always receive a challenge at an appropriate level.

[0447] Furthermore, after a game ends, users can provide feedback through a feedback function. This feedback is then incorporated into the generation of the next virtual match unit to improve the user experience. Through this feedback process, the system is constantly being improved.

[0448] As a concrete example, let's assume a user is playing a shooting game. If this user is an intermediate-level player with an offensive style, the server will generate a virtual battle unit with enhanced defensive capabilities. If the user adopts a new offensive strategy during gameplay, the server will immediately strengthen the defensive tactics of the virtual battle unit.

[0449] Examples of prompts to input into a generating AI model include: "Describe a system that analyzes user gameplay data and adjusts virtual opponents in real time to provide players with the best possible gaming experience in an online game. As a specific example, please explain how to generate an opponent suitable for an attack-oriented intermediate player in a shooting game."

[0450] In this way, we can provide a system that continuously optimizes the gaming experience to meet diverse user needs.

[0451] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0452] Step 1:

[0453] The server receives user gameplay data transmitted in real time from the terminal. Input from the terminal includes user commands, selected options, and gameplay metrics. The server stores this data in temporary storage, preparing it for use in later analysis steps.

[0454] Step 2:

[0455] The server analyzes the saved gameplay data. It processes the received data through an analysis algorithm to generate output that identifies the user's skill level and behavioral patterns. In this process, a machine learning model is used to categorize the user's play style. Specifically, it calculates the frequency of skill usage and the average completion time.

[0456] Step 3:

[0457] The server generates a virtual battle unit corresponding to the user based on the analysis results. The output data from the analysis (user's skill level, behavior patterns) is used as input. The server can use the generated AI model to determine the abilities and strategies of the virtual opponent and create the details of the battle unit to be presented to the user as output.

[0458] Step 4:

[0459] During gameplay, the server continuously monitors the user's real-time performance. By analyzing real-time data as input, the server instantly adjusts the actions of the virtual battle unit when the user adopts new tactics. This ensures that the user is always provided with a battle environment based on the latest tactics.

[0460] Step 5:

[0461] After a game ends, users provide feedback through the interface. The server collects this feedback data as input and incorporates it as adjustment data as output in subsequent generation processes to further improve the user experience. This continuously improves the overall system performance and user experience.

[0462] (Application Example 1)

[0463] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0464] In today's information-saturated world, there is a need to quickly and accurately provide each user with the most relevant information and content. However, conventional technologies have struggled to accurately grasp users' real-time preferences and reactions and dynamically adjust content accordingly. As a result, users may be provided with inappropriate or uninteresting information, leading to a decline in the quality of their experience.

[0465] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0466] In this invention, the server includes means for collecting user information, means for analyzing the collected user information to identify the user's preferences and tendencies, and means for generating optimal virtual recommendation targets based on the analysis results. This makes it possible to provide users with information and content that is appropriate and tailored in real time.

[0467] "User information" refers to data collected by the system from individual users regarding their usage history, preferences, and interactions.

[0468] "Analysis" is the process of identifying patterns and trends based on collected user information to understand user preferences.

[0469] "Preferences" refer to data that indicates the user's preferences and interests regarding specific content or genres.

[0470] "Trends" refer to behavioral patterns in how users consume information and content.

[0471] "Virtual recommendation targets" refer to information, content, or combinations thereof selected by the system based on the user's preferences and tendencies.

[0472] "Real-time adjustment" is a process that instantly updates the information and content provided to be appropriate based on the user's current reactions and circumstances.

[0473] This system is designed to provide users with the most relevant information in a content delivery service. The server collects and analyzes user information to identify preferences and trends. Specifically, it records videos watched, selected content, ratings, and other data in real time. It utilizes AWS (Amazon Web Services), employing Lambda services and DynamoDB for efficient data management.

[0474] Next, using the collected data, a machine learning algorithm in Amazon SageMaker learns the user's preferences and generates optimal virtual recommendations. These recommendations are then adjusted based on the user's current situation and past behavior. AWS SNS service is used to quickly notify the user's device of recommended content, presenting topics of interest.

[0475] For example, users interested in sports will be recommended the latest match highlights and related news. Feedback from users after they have finished viewing this information is collected and incorporated into the recommendation algorithm for future recommendations, resulting in more accurate recommendations. An example of a prompt for the generative AI model is the input format, "Recommend the next best content based on the user's current viewing history."

[0476] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0477] Step 1:

[0478] The server collects user information from the user's device in real time. It receives data such as viewing history, content click information, and rating information as input. Based on this information, the server stores the data in DynamoDB.

[0479] Step 2:

[0480] The server analyzes accumulated user information. The input is user data obtained from DynamoDB. Amazon SageMaker is used within the server to run a machine learning model to identify preferences. The output is information indicating user preferences and trends.

[0481] Step 3:

[0482] The server generates optimal virtual recommendations based on the analyzed results. It uses user preference and trend data as input. It leverages Amazon SageMaker's generation AI model to create a list of recommended content as output.

[0483] Step 4:

[0484] The server uses AWS SNS to notify the user's device of a list of recommended content. The input is encoded recommendation information, and the output is a notification message sent to the user's device. The user receives this message on their device and can view the recommended content.

[0485] Step 5:

[0486] After a user finishes viewing content, the device sends post-viewing feedback to the server. The input consists of ratings and opinions provided by the user, and the output is stored on the server as feedback information.

[0487] Step 6:

[0488] The server analyzes the collected feedback information to use in the next recommendation process. The input is feedback data, and based on this data, it retrains a machine learning model to establish an improved recommendation system as output.

[0489] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0490] This invention relates to a system for recognizing players' emotions in online games and adapting the game experience based on that information. This system comprises the following components.

[0491] Extension of data collection and analysis

[0492] The server collects data to analyze the user's emotional state, in addition to conventional player data. The terminal acquires emotion-related data through user input, facial recognition sensors, and voice tone analysis, and transmits it to the server. This information is used to identify the user's mental state in real time.

[0493] Introducing an emotional engine

[0494] The server uses an emotion engine to analyze the user's emotional state. The emotion engine uses machine learning to infer the user's emotions from the input data and identify how that state affects gameplay. This allows the server to determine whether the user is stressed, enjoying themselves, or frustrated.

[0495] Adaptation of virtual opponents

[0496] Emotional information is reflected in the actions of virtual opponents. The server adjusts the opponent's aggression and changes the game difficulty according to the user's emotional state, allowing the user to play at an optimal stress level. For example, if the server determines that the user is feeling frustrated, it will take measures to lower the difficulty level.

[0497] Customized gaming experience

[0498] The server collects user feedback based on their emotions and customizes scenarios and environments accordingly. If a user frequently exhibits a particular emotional state, the game scenario is adjusted to match, thereby increasing user satisfaction.

[0499] Explanation of specific examples

[0500] For example, if the emotion engine detects that a user is stressed while playing an action game, the server will suggest a more relaxed game environment. This may include reducing the number of enemies or changing the music. Furthermore, once the emotional state improves, the system will revert to the original difficulty level to maintain game balance. This allows users to enjoy a more comfortable gaming experience.

[0501] The following describes the processing flow.

[0502] Step 1:

[0503] The server retrieves basic information, including the user's profile and past play history, at the start of a user's game session. This information is used to understand the user's general play style and preferences.

[0504] Step 2:

[0505] The device collects user action data in real time during gameplay and transmits it to the server. This data includes actions taken in the game, the buttons pressed on the controller, and selections made within the game.

[0506] Step 3:

[0507] The device also uses built-in sensors and a webcam to collect data to measure the user's emotional state. This includes facial expression analysis, voice tone, and heart rate.

[0508] Step 4:

[0509] The server inputs the collected emotional data into the emotion engine, which analyzes the user's emotional state. The emotion engine estimates the user's current state based on multiple emotional categories and identifies levels of stress and joy.

[0510] Step 5:

[0511] The server adjusts the behavior of virtual opponents and the game environment based on the analyzed emotional state. For example, if the user is in a high-tension state, the server may reduce the aggression of virtual opponents or lower the game difficulty.

[0512] Step 6:

[0513] As users continue playing the game, they can intuitively sense the changes displayed on their device. For example, the game music might become more relaxing.

[0514] Step 7:

[0515] The server collects user ratings and feedback after the game ends. This feedback is used to improve the accuracy of the emotion engine and tune the entire system, contributing to improvements in the next game session.

[0516] (Example 2)

[0517] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0518] In modern computer games, a uniform game experience is provided that disregards the player's emotional state, making it difficult to offer an optimal game experience tailored to each individual player's emotions. Furthermore, there is a lack of features to adjust game difficulty and environment based on emotional state, resulting in insufficient player satisfaction.

[0519] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0520] In this invention, the server includes means for collecting user information, means for analyzing the collected user information and identifying the user's emotional state, and means for generating an optimal virtual opponent based on the analysis results. This enables the real-time adaptation of the game experience according to the player's emotional state.

[0521] "User information" refers to data related to players participating in the game, including information such as operation data, facial expressions, voice, and actions during gameplay.

[0522] "Emotional state" refers to the psychological and emotional state a user exhibits during gameplay, and is used to identify feelings such as joy, anger, tension, and excitement.

[0523] A "virtual opponent" is an AI-based character that the player faces in the game, designed to react appropriately to the player.

[0524] "Behavioral characteristics" refer to the behavioral traits and attributes exhibited by a virtual opponent, including aggression, tactics, and reaction speed.

[0525] "Feedback" refers to information such as comments and reactions from users regarding their game experience, which will be used to adjust the game in the future.

[0526] A "scenario" refers to the story and setting provided to the user within a game, and it changes according to their emotional state.

[0527] "Environment" refers to the overall atmosphere and situation that players experience, including elements such as sound, visual effects, and difficulty within the game.

[0528] "Cooperative players" are AI-based characters that appear in the game to support the player and are designed to enhance the player experience.

[0529] This invention relates to a system that recognizes the emotional state of a user through online games and optimizes the game experience based on that information. This system mainly consists of three main parts: collection of user information, analysis of emotional state, and adjustment of the game experience.

[0530] The device includes hardware for collecting user operation data, facial expression data, and voice data. In particular, it uses devices such as cameras and microphones to capture emotional elements while the user is playing the game. Computer vision libraries (e.g., OpenCV) are used for facial expression recognition, and acoustic processing libraries (e.g., LibROSA) are used for voice tone analysis.

[0531] Emotion-related data collected on the device is sent to the server using a secure communication protocol (e.g., TLS / SSL).

[0532] The server uses machine learning models to analyze the data it receives. This analysis employs deep learning frameworks such as TensorFlow and PyTorch. This allows the server to infer the emotional state the user exhibits during gameplay and identify stress, enjoyment, frustration, and other related emotions.

[0533] The server automatically adjusts game settings based on the user's emotional state. For example, if the user is feeling stressed, this might include reducing the number of enemies in the game or changing the music to something more relaxing. These adjustments are made in real time to improve the user experience.

[0534] As a concrete example, suppose a user feels tense while playing, and the device captures this tension in their facial expression. The server analyzes the tension and adjusts the game's difficulty level to alleviate it. As a result, user satisfaction improves.

[0535] Examples of prompts include, "Adjust the game difficulty based on user sentiment data," or "If the player is dissatisfied, suggest how to change the game scenario."

[0536] In this way, the present invention makes it possible to provide a flexible gaming environment that responds to the user's emotions.

[0537] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0538] Step 1:

[0539] The device monitors the user's interactions and collects operation data, facial expression data, and voice data using various sensors. This input data includes keyboard and mouse operation speed, subtle facial expression changes, and voice tone and pressure. This data is used as basic information to infer emotional states.

[0540] Step 2:

[0541] The device transmits collected data to the server in real time. Encrypted communication protocols are used for data transmission to securely protect user data. Input data is passed to the server as a series of numerical signals, where feature extraction is performed. This results in the output of numerical information representing important sentiment indicators.

[0542] Step 3:

[0543] The server uses the transmitted features to analyze the emotional state based on a generative AI model. The machine learning algorithm performs data calculations on the received features to infer emotional categories such as "joy," "anger," and "surprise." The analysis results output a numerical model of the user's emotional state.

[0544] Step 4:

[0545] The server automatically adjusts the game environment based on the analysis results. Specifically, if it determines that the user is in a high-stress state, it changes the in-game settings. This may include reducing the number of enemy characters, changing the music, or adjusting the visual effects. The input is a numerical model of the emotional state, and the output is a profile of the adjusted game settings.

[0546] Step 5:

[0547] The user continues playing in a game environment adjusted by the server. Interactions within the adjusted game trigger further feedback data collection, generating prompts. For example, a prompt might include a task such as "Adjust game difficulty based on user sentiment data." This feedback is used to customize the environment for subsequent play sessions.

[0548] (Application Example 2)

[0549] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0550] In modern homes, adapting the living environment to the emotional state of each individual is a significant challenge. Conventional household appliances and life support robots often fail to adequately understand the user's emotions and provide services accordingly, resulting in a lack of relief from user stress and dissatisfaction. This invention aims to solve these problems and realize a comfortable living environment within the home.

[0551] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0552] In this invention, the server includes means for collecting emotion-related information, means for analyzing the collected emotion-related information and identifying the user's mental state, and means for generating an optimal virtual dialogue partner based on the mental state. This makes it possible to understand the user's emotions in real time and to provide optimal responses and adjust the environment accordingly.

[0553] "Emotion-related information" refers to data that indicates a user's psychological state, obtained from their voice, facial expressions, behavior, etc.

[0554] "Means of collection" refers to systems that use sensors, cameras, microphones, etc., to acquire information related to the user's emotions.

[0555] "Means of analysis to identify the user's mental state" refers to the process of identifying and classifying emotions and psychological states from collected data using machine learning algorithms and analytical software.

[0556] "Means for generating the optimal virtual dialogue target" refers to a system that, based on analysis results, designs and provides responses and dialogue content adapted to the user's emotions.

[0557] "Means of adjusting behavior based on mental state" refers to technologies that dynamically change the operation of a system or the settings of the environment in accordance with the user's emotions.

[0558] "Means for collecting feedback and incorporating it into future generation processes" refers to a system that accumulates user responses and opinions and uses them to improve the quality of future services and responses.

[0559] The system for carrying out the present invention is executed through a consumer robot installed in a home. It comprises the elements detailed below.

[0560] The server collects user emotion-related information using devices such as facial recognition cameras and microphones. This data is used to analyze the user's voice tone and facial expressions. Specifically, it uses facial recognition cameras such as the RealSense Depth Camera and facial analysis algorithms such as OpenCV and the Emotion Detection API to accurately capture facial data.

[0561] The collected data is analyzed on the server using machine learning algorithms, such as emotion recognition models based on TensorFlow or PyTorch. This allows the server to identify the user's mental state in real time.

[0562] Based on the analyzed mental state, the server provides the user with the most suitable virtual interaction. For example, if the user is feeling stressed, the server will provide voice responses through a smart speaker, suggesting relaxing music or advice. A smart speaker like Amazon Echo is used for this purpose.

[0563] Furthermore, the system collects user feedback and uses it as learning data to make future responses more personalized. This allows for continuous improvement of the user experience.

[0564] As a concrete example, when the server detects a user's facial expression indicating fatigue upon returning from work, it prompts the user with "Do you want to relax?" and plays appropriate music based on their response. As an example of a specific prompt sentence, the AI ​​generating model is input with the following sentence: "Design a method to recognize the emotional state of family members at home in real time and suggest appropriate chores or music. Explain the functions of the smart home system based on this." The system's response is then checked.

[0565] In this way, it becomes possible to provide a flexible assistant system that can respond to diverse living situations within the home, thereby supporting a comfortable living environment.

[0566] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0567] Step 1:

[0568] The device collects user emotion-related information using a facial recognition camera and microphone. The input data consists of the user's facial expressions and voice, which are captured by the sensors. This data is then transmitted to a server in real time.

[0569] Step 2:

[0570] The server analyzes the received facial expression and audio data. This analysis is performed using OpenCV and the Emotion Detection API, and a machine learning model (e.g., TensorFlow) is used to infer the user's emotional state. The input data consists of raw facial expressions and audio, and the output is a identified mental state (e.g., stress, happiness).

[0571] Step 3:

[0572] Based on the user's emotional state, the server generates an appropriate response. A generative AI model is used to select the most appropriate music or suggestions. The input to this process is the identified emotional state, and the output is the suggested action (e.g., music playback, voice guidance).

[0573] Step 4:

[0574] The server sends the selected response to the terminal, which then displays it to the user. This can be done through a smart speaker, for example, by playing music or sending a voice message. The input here is the action data generated in the previous step, and the output is the actual response action to the user.

[0575] Step 5:

[0576] Users can provide feedback on the responses they receive. The server collects this feedback and stores it as training data for the system. The input is the feedback content, and the output is the improved response model. This feedback-based improvement process will eventually lead to more personalized services.

[0577] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0578] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0579] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0580] [Fourth Embodiment]

[0581] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0582] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0583] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0584] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0585] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0586] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0587] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0588] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0589] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0590] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0591] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0592] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0593] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0594] This invention is a system designed to enable players to enjoy the optimal gaming experience within online games. This system is implemented through the following operations.

[0595] Data collection and analysis

[0596] The server acquires data in real time during the user's game session. The terminal sends the user's input actions, choices made during gameplay, and various metrics to the server. This data includes scores, stage completion times, and statistics on the characters and skills used. The server analyzes this data to identify the user's skill level and gameplay characteristics.

[0597] Generating a virtual opponent

[0598] Based on the analyzed data, the server generates a virtual opponent suited to the user. This virtual opponent has a difficulty level that matches the user's skill level and possesses tactics that can counter the user's play style. For example, a user who is heavily focused on offense will be matched with an opponent who excels in defense.

[0599] Real-time adaptation and adjustment

[0600] During gameplay, the server monitors the user's real-time performance. When a user employs more advanced strategies, the server instantly adjusts the opponent's tactics and difficulty level. This adaptation ensures that users are always challenged at an appropriate level.

[0601] Feedback and Learning

[0602] Users provide feedback on their game experience through an interface presented after the game ends. This feedback includes information about the opponent's strength and how enjoyable the game was. The server collects this feedback and uses it to generate virtual opponents for future matches.

[0603] Explanation of specific examples

[0604] For example, suppose a user is playing a shooting game. This user has an offensive play style and is at an intermediate level. The server generates a virtual opponent that prioritizes defense for this user and adjusts the opponent in real time in response to changes in the user's strategy. This allows the user to improve their skills while always enjoying a fresh and optimized gaming experience.

[0605] The following describes the processing flow.

[0606] Step 1:

[0607] The server retrieves the user's gamer profile information at the start of the game. This information includes the user's past gaming history, current rank, and registered preference settings.

[0608] Step 2:

[0609] The device records user actions in real time during gameplay and sends them to the server. This includes actions such as movement, attacking, and defending, as well as their frequency and effect.

[0610] Step 3:

[0611] The server analyzes the user's skill level and behavioral patterns based on the real-time data it receives. It uses machine learning algorithms to identify the user's strengths and weaknesses in tactics.

[0612] Step 4:

[0613] The server generates a virtual opponent based on the analyzed results. This opponent's skills and tactics are adjusted to the user's level and adapted to the user's expected play style.

[0614] Step 5:

[0615] During gameplay, the server continuously monitors user performance and adjusts the virtual opponent's actions in real time as needed. For example, if a user introduces a new tactic, the opponent's strategy will be changed.

[0616] Step 6:

[0617] Users evaluate their experience through a feedback form after the game ends. The server collects this feedback and incorporates it into the algorithm for generating opponents for the next match.

[0618] Step 7:

[0619] The server collects feedback and game data, updating the system's overall learning database. This data is used to improve the experience for other users and develop new features.

[0620] (Example 1)

[0621] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0622] In online games, there is a need to provide players with an optimal gaming experience tailored to their individual skill levels and strategies, while maintaining consistent engagement and enjoyment. Furthermore, a challenge exists where mismatched opponents or game environments can lead to unsuitable difficulty levels and decreased player satisfaction.

[0623] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0624] In this invention, the server includes means for acquiring user information, means for analyzing the acquired user information to identify the user's skill level and behavioral patterns, and means for the generated virtual battle unit to determine its skill parameters and strategy using a generated AI model. This makes it possible to generate virtual battle units optimized for each user's skills and play style, and to enable real-time adjustments, thereby providing users with a consistently challenging and enjoyable gaming environment.

[0625] "Means of acquiring user information" refers to technologies that collect user behavior data and choices made during gameplay in real time.

[0626] "Means of analysis to identify users' skill levels and behavioral patterns" refers to technologies that identify play skills and unique gameplay styles based on collected user data.

[0627] "Means for generating appropriate virtual battle units" refers to technology that creates opponents with optimal difficulty levels and tactics for the user based on analysis results.

[0628] "Means for adjusting the behavior patterns of virtual battle units according to real-time results" refers to technology that continuously modifies and optimizes the tactics and actions of generated virtual opponents based on the user's current game situation.

[0629] "Means of collecting opinions and reflecting them in the next generation process" refers to technology that collects feedback based on users' game experiences and utilizes it when creating virtual opponents in the future.

[0630] "Means of determining ability parameters and strategies" refers to the technology of using generative AI models to determine the actions and skills that a virtual opponent will perform and use.

[0631] This invention provides a system that highly personalizes the user's gaming experience on an online game platform. By generating and adjusting optimal virtual battle units in real time according to the user's skill level, it ensures a consistently fresh and challenging gaming experience.

[0632] The server uses data received from the device to analyze user behavior in detail while the user plays the game. The device continuously collects data on the actions, choices, and achievements the user makes throughout the game and sends this data to the server. This data includes scores, completion times, and the frequency of use of selected characters and skills.

[0633] The server uses a generative AI model to analyze the received data. This model uses machine learning algorithms to identify the user's skill level and behavioral patterns. Based on this information, the generative AI model generates a virtual battle unit (VRB) suitable for the user. The VRB has ability parameters and strategies that correspond to the user's skill level and is optimized for the user's play style.

[0634] During gameplay, the server monitors user performance in real time. When a user employs a new strategy, the server instantly adjusts the actions and difficulty of the virtual battle unit. This ensures that users always receive a challenge at an appropriate level.

[0635] Furthermore, after a game ends, users can provide feedback through a feedback function. This feedback is then incorporated into the generation of the next virtual match unit to improve the user experience. Through this feedback process, the system is constantly being improved.

[0636] As a concrete example, let's assume a user is playing a shooting game. If this user is an intermediate-level player with an offensive style, the server will generate a virtual battle unit with enhanced defensive capabilities. If the user adopts a new offensive strategy during gameplay, the server will immediately strengthen the defensive tactics of the virtual battle unit.

[0637] Examples of prompts to input into a generating AI model include: "Describe a system that analyzes user gameplay data and adjusts virtual opponents in real time to provide players with the best possible gaming experience in an online game. As a specific example, please explain how to generate an opponent suitable for an attack-oriented intermediate player in a shooting game."

[0638] In this way, we can provide a system that continuously optimizes the gaming experience to meet diverse user needs.

[0639] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0640] Step 1:

[0641] The server receives user gameplay data transmitted in real time from the terminal. Input from the terminal includes user commands, selected options, and gameplay metrics. The server stores this data in temporary storage, preparing it for use in later analysis steps.

[0642] Step 2:

[0643] The server analyzes the saved gameplay data. It processes the received data through an analysis algorithm to generate output that identifies the user's skill level and behavioral patterns. In this process, a machine learning model is used to categorize the user's play style. Specifically, it calculates the frequency of skill usage and the average completion time.

[0644] Step 3:

[0645] The server generates a virtual battle unit corresponding to the user based on the analysis results. The output data from the analysis (user's skill level, behavior patterns) is used as input. The server can use the generated AI model to determine the abilities and strategies of the virtual opponent and create the details of the battle unit to be presented to the user as output.

[0646] Step 4:

[0647] During gameplay, the server continuously monitors the user's real-time performance. By analyzing real-time data as input, the server instantly adjusts the actions of the virtual battle unit when the user adopts new tactics. This ensures that the user is always provided with a battle environment based on the latest tactics.

[0648] Step 5:

[0649] After a game ends, users provide feedback through the interface. The server collects this feedback data as input and incorporates it as adjustment data as output in subsequent generation processes to further improve the user experience. This continuously improves the overall system performance and user experience.

[0650] (Application Example 1)

[0651] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0652] In today's information-saturated world, there is a need to quickly and accurately provide each user with the most relevant information and content. However, conventional technologies have struggled to accurately grasp users' real-time preferences and reactions and dynamically adjust content accordingly. As a result, users may be provided with inappropriate or uninteresting information, leading to a decline in the quality of their experience.

[0653] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0654] In this invention, the server includes means for collecting user information, means for analyzing the collected user information to identify the user's preferences and tendencies, and means for generating optimal virtual recommendation targets based on the analysis results. This makes it possible to provide users with information and content that is appropriate and tailored in real time.

[0655] "User information" refers to data collected by the system from individual users regarding their usage history, preferences, and interactions.

[0656] "Analysis" is the process of identifying patterns and trends based on collected user information to understand user preferences.

[0657] "Preferences" refer to data that indicates the user's preferences and interests regarding specific content or genres.

[0658] "Trends" refer to behavioral patterns in how users consume information and content.

[0659] "Virtual recommendation targets" refer to information, content, or combinations thereof selected by the system based on the user's preferences and tendencies.

[0660] "Real-time adjustment" is a process that instantly updates the information and content provided to be appropriate based on the user's current reactions and circumstances.

[0661] This system is designed to provide users with the most relevant information in a content delivery service. The server collects and analyzes user information to identify preferences and trends. Specifically, it records videos watched, selected content, ratings, and other data in real time. It utilizes AWS (Amazon Web Services), employing Lambda services and DynamoDB for efficient data management.

[0662] Next, using the collected data, a machine learning algorithm in Amazon SageMaker learns the user's preferences and generates optimal virtual recommendations. These recommendations are then adjusted based on the user's current situation and past behavior. AWS SNS service is used to quickly notify the user's device of recommended content, presenting topics of interest.

[0663] For example, users interested in sports will be recommended the latest match highlights and related news. Feedback from users after they have finished viewing this information is collected and incorporated into the recommendation algorithm for future recommendations, resulting in more accurate recommendations. An example of a prompt for the generative AI model is the input format, "Recommend the next best content based on the user's current viewing history."

[0664] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0665] Step 1:

[0666] The server collects user information from the user's device in real time. It receives data such as viewing history, content click information, and rating information as input. Based on this information, the server stores the data in DynamoDB.

[0667] Step 2:

[0668] The server analyzes accumulated user information. The input is user data obtained from DynamoDB. Amazon SageMaker is used within the server to run a machine learning model to identify preferences. The output is information indicating user preferences and trends.

[0669] Step 3:

[0670] The server generates optimal virtual recommendations based on the analyzed results. It uses user preference and trend data as input. It leverages Amazon SageMaker's generation AI model to create a list of recommended content as output.

[0671] Step 4:

[0672] The server uses AWS SNS to notify the user's device of a list of recommended content. The input is encoded recommendation information, and the output is a notification message sent to the user's device. The user receives this message on their device and can view the recommended content.

[0673] Step 5:

[0674] After a user finishes viewing content, the device sends post-viewing feedback to the server. The input consists of ratings and opinions provided by the user, and the output is stored on the server as feedback information.

[0675] Step 6:

[0676] The server analyzes the collected feedback information to use in the next recommendation process. The input is feedback data, and based on this data, it retrains a machine learning model to establish an improved recommendation system as output.

[0677] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0678] This invention relates to a system for recognizing players' emotions in online games and adapting the game experience based on that information. This system comprises the following components.

[0679] Extension of data collection and analysis

[0680] The server collects data to analyze the user's emotional state, in addition to conventional player data. The terminal acquires emotion-related data through user input, facial recognition sensors, and voice tone analysis, and transmits it to the server. This information is used to identify the user's mental state in real time.

[0681] Introducing an emotional engine

[0682] The server uses an emotion engine to analyze the user's emotional state. The emotion engine uses machine learning to infer the user's emotions from the input data and identify how that state affects gameplay. This allows the server to determine whether the user is stressed, enjoying themselves, or frustrated.

[0683] Adaptation of virtual opponents

[0684] Emotional information is reflected in the actions of virtual opponents. The server adjusts the opponent's aggression and changes the game difficulty according to the user's emotional state, allowing the user to play at an optimal stress level. For example, if the server determines that the user is feeling frustrated, it will take measures to lower the difficulty level.

[0685] Customized gaming experience

[0686] The server collects user feedback based on their emotions and customizes scenarios and environments accordingly. If a user frequently exhibits a particular emotional state, the game scenario is adjusted to match, thereby increasing user satisfaction.

[0687] Explanation of specific examples

[0688] For example, if the emotion engine detects that a user is stressed while playing an action game, the server will suggest a more relaxed game environment. This may include reducing the number of enemies or changing the music. Furthermore, once the emotional state improves, the system will revert to the original difficulty level to maintain game balance. This allows users to enjoy a more comfortable gaming experience.

[0689] The following describes the processing flow.

[0690] Step 1:

[0691] The server retrieves basic information, including the user's profile and past play history, at the start of a user's game session. This information is used to understand the user's general play style and preferences.

[0692] Step 2:

[0693] The device collects user action data in real time during gameplay and transmits it to the server. This data includes actions taken in the game, the buttons pressed on the controller, and selections made within the game.

[0694] Step 3:

[0695] The device also uses built-in sensors and a webcam to collect data to measure the user's emotional state. This includes facial expression analysis, voice tone, and heart rate.

[0696] Step 4:

[0697] The server inputs the collected emotional data into the emotion engine, which analyzes the user's emotional state. The emotion engine estimates the user's current state based on multiple emotional categories and identifies levels of stress and joy.

[0698] Step 5:

[0699] The server adjusts the behavior of virtual opponents and the game environment based on the analyzed emotional state. For example, if the user is in a high-tension state, the server may reduce the aggression of virtual opponents or lower the game difficulty.

[0700] Step 6:

[0701] As users continue playing the game, they can intuitively sense the changes displayed on their device. For example, the game music might become more relaxing.

[0702] Step 7:

[0703] The server collects user ratings and feedback after the game ends. This feedback is used to improve the accuracy of the emotion engine and tune the entire system, contributing to improvements in the next game session.

[0704] (Example 2)

[0705] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0706] In modern computer games, a uniform game experience is provided that disregards the player's emotional state, making it difficult to offer an optimal game experience tailored to each individual player's emotions. Furthermore, there is a lack of features to adjust game difficulty and environment based on emotional state, resulting in insufficient player satisfaction.

[0707] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0708] In this invention, the server includes means for collecting user information, means for analyzing the collected user information and identifying the user's emotional state, and means for generating an optimal virtual opponent based on the analysis results. This enables the real-time adaptation of the game experience according to the player's emotional state.

[0709] "User information" refers to data related to players participating in the game, including information such as operation data, facial expressions, voice, and actions during gameplay.

[0710] "Emotional state" refers to the psychological and emotional state a user exhibits during gameplay, and is used to identify feelings such as joy, anger, tension, and excitement.

[0711] A "virtual opponent" is an AI-based character that the player faces in the game, designed to react appropriately to the player.

[0712] "Behavioral characteristics" refer to the behavioral traits and attributes exhibited by a virtual opponent, including aggression, tactics, and reaction speed.

[0713] "Feedback" refers to information such as comments and reactions from users regarding their game experience, which will be used to adjust the game in the future.

[0714] A "scenario" refers to the story and setting provided to the user within a game, and it changes according to their emotional state.

[0715] "Environment" refers to the overall atmosphere and situation that players experience, including elements such as sound, visual effects, and difficulty within the game.

[0716] "Cooperative players" are AI-based characters that appear in the game to support the player and are designed to enhance the player experience.

[0717] This invention relates to a system that recognizes the emotional state of a user through online games and optimizes the game experience based on that information. This system mainly consists of three main parts: collection of user information, analysis of emotional state, and adjustment of the game experience.

[0718] The device includes hardware for collecting user operation data, facial expression data, and voice data. In particular, it uses devices such as cameras and microphones to capture emotional elements while the user is playing the game. Computer vision libraries (e.g., OpenCV) are used for facial expression recognition, and acoustic processing libraries (e.g., LibROSA) are used for voice tone analysis.

[0719] Emotion-related data collected on the device is sent to the server using a secure communication protocol (e.g., TLS / SSL).

[0720] The server uses machine learning models to analyze the data it receives. This analysis employs deep learning frameworks such as TensorFlow and PyTorch. This allows the server to infer the emotional state the user exhibits during gameplay and identify stress, enjoyment, frustration, and other related emotions.

[0721] The server automatically adjusts game settings based on the user's emotional state. For example, if the user is feeling stressed, this might include reducing the number of enemies in the game or changing the music to something more relaxing. These adjustments are made in real time to improve the user experience.

[0722] As a concrete example, suppose a user feels tense while playing, and the device captures this tension in their facial expression. The server analyzes the tension and adjusts the game's difficulty level to alleviate it. As a result, user satisfaction improves.

[0723] Examples of prompts include, "Adjust the game difficulty based on user sentiment data," or "If the player is dissatisfied, suggest how to change the game scenario."

[0724] In this way, the present invention makes it possible to provide a flexible gaming environment that responds to the user's emotions.

[0725] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0726] Step 1:

[0727] The device monitors the user's interactions and collects operation data, facial expression data, and voice data using various sensors. This input data includes keyboard and mouse operation speed, subtle facial expression changes, and voice tone and pressure. This data is used as basic information to infer emotional states.

[0728] Step 2:

[0729] The device transmits collected data to the server in real time. Encrypted communication protocols are used for data transmission to securely protect user data. Input data is passed to the server as a series of numerical signals, where feature extraction is performed. This results in the output of numerical information representing important sentiment indicators.

[0730] Step 3:

[0731] The server uses the transmitted features to analyze the emotional state based on a generative AI model. The machine learning algorithm performs data calculations on the received features to infer emotional categories such as "joy," "anger," and "surprise." The analysis results output a numerical model of the user's emotional state.

[0732] Step 4:

[0733] The server automatically adjusts the game environment based on the analysis results. Specifically, if it determines that the user is in a high-stress state, it changes the in-game settings. This may include reducing the number of enemy characters, changing the music, or adjusting the visual effects. The input is a numerical model of the emotional state, and the output is a profile of the adjusted game settings.

[0734] Step 5:

[0735] The user continues playing in a game environment adjusted by the server. Interactions within the adjusted game trigger further feedback data collection, generating prompts. For example, a prompt might include a task such as "Adjust game difficulty based on user sentiment data." This feedback is used to customize the environment for subsequent play sessions.

[0736] (Application Example 2)

[0737] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0738] In modern homes, adapting the living environment to the emotional state of each individual is a significant challenge. Conventional household appliances and life support robots often fail to adequately understand the user's emotions and provide services accordingly, resulting in a lack of relief from user stress and dissatisfaction. This invention aims to solve these problems and realize a comfortable living environment within the home.

[0739] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0740] In this invention, the server includes means for collecting emotion-related information, means for analyzing the collected emotion-related information and identifying the user's mental state, and means for generating an optimal virtual dialogue partner based on the mental state. This makes it possible to understand the user's emotions in real time and to provide optimal responses and adjust the environment accordingly.

[0741] "Emotion-related information" refers to data that indicates a user's psychological state, obtained from their voice, facial expressions, behavior, etc.

[0742] "Means of collection" refers to systems that use sensors, cameras, microphones, etc., to acquire information related to the user's emotions.

[0743] "Means of analysis to identify the user's mental state" refers to the process of identifying and classifying emotions and psychological states from collected data using machine learning algorithms and analytical software.

[0744] "Means for generating the optimal virtual dialogue target" refers to a system that, based on analysis results, designs and provides responses and dialogue content adapted to the user's emotions.

[0745] "Means of adjusting behavior based on mental state" refers to technologies that dynamically change the operation of a system or the settings of the environment in accordance with the user's emotions.

[0746] "Means for collecting feedback and incorporating it into future generation processes" refers to a system that accumulates user responses and opinions and uses them to improve the quality of future services and responses.

[0747] The system for carrying out the present invention is executed through a consumer robot installed in a home. It comprises the elements detailed below.

[0748] The server collects user emotion-related information using devices such as facial recognition cameras and microphones. This data is used to analyze the user's voice tone and facial expressions. Specifically, it uses facial recognition cameras such as the RealSense Depth Camera and facial analysis algorithms such as OpenCV and the Emotion Detection API to accurately capture facial data.

[0749] The collected data is analyzed on the server using machine learning algorithms, such as emotion recognition models based on TensorFlow or PyTorch. This allows the server to identify the user's mental state in real time.

[0750] Based on the analyzed mental state, the server provides the user with the most suitable virtual interaction. For example, if the user is feeling stressed, the server will provide voice responses through a smart speaker, suggesting relaxing music or advice. A smart speaker like Amazon Echo is used for this purpose.

[0751] Furthermore, the system collects user feedback and uses it as learning data to make future responses more personalized. This allows for continuous improvement of the user experience.

[0752] As a concrete example, when the server detects a user's facial expression indicating fatigue upon returning from work, it prompts the user with "Do you want to relax?" and plays appropriate music based on their response. As an example of a specific prompt sentence, the AI ​​generating model is input with the following sentence: "Design a method to recognize the emotional state of family members at home in real time and suggest appropriate chores or music. Explain the functions of the smart home system based on this." The system's response is then checked.

[0753] In this way, it becomes possible to provide a flexible assistant system that can respond to diverse living situations within the home, thereby supporting a comfortable living environment.

[0754] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0755] Step 1:

[0756] The device collects user emotion-related information using a facial recognition camera and microphone. The input data consists of the user's facial expressions and voice, which are captured by the sensors. This data is then transmitted to a server in real time.

[0757] Step 2:

[0758] The server analyzes the received facial expression and audio data. This analysis is performed using OpenCV and the Emotion Detection API, and a machine learning model (e.g., TensorFlow) is used to infer the user's emotional state. The input data consists of raw facial expressions and audio, and the output is a identified mental state (e.g., stress, happiness).

[0759] Step 3:

[0760] Based on the user's emotional state, the server generates an appropriate response. A generative AI model is used to select the most appropriate music or suggestions. The input to this process is the identified emotional state, and the output is the suggested action (e.g., music playback, voice guidance).

[0761] Step 4:

[0762] The server sends the selected response to the terminal, which then displays it to the user. This can be done through a smart speaker, for example, by playing music or sending a voice message. The input here is the action data generated in the previous step, and the output is the actual response action to the user.

[0763] Step 5:

[0764] Users can provide feedback on the responses they receive. The server collects this feedback and stores it as training data for the system. The input is the feedback content, and the output is the improved response model. This feedback-based improvement process will eventually lead to more personalized services.

[0765] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0766] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0767] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0768] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0769] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0770] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0771] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0772] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0773] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0774] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0775] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0776] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0777] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0778] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0779] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0780] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0781] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0782] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0783] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0784] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0785] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0786] The following is further disclosed regarding the embodiments described above.

[0787] (Claim 1)

[0788] Means of collecting player data,

[0789] A means for analyzing the collected player data to identify the player's skill level and behavioral patterns,

[0790] A means for generating the optimal virtual opponent based on the analysis results,

[0791] Means for adjusting the behavioral patterns of the virtual opponent in accordance with the real-time performance of the player,

[0792] A system that includes means for collecting player feedback and incorporating it into future generation processes.

[0793] (Claim 2)

[0794] The system according to claim 1, further comprising means for customizing a scenario or environment based on the player's preferences.

[0795] (Claim 3)

[0796] The system according to claim 1, further comprising means for generating cooperative play partners and improving the player's gameplay experience.

[0797] "Example 1"

[0798] (Claim 1)

[0799] Means of obtaining user information,

[0800] A means for analyzing the acquired user information to identify the user's ability level and behavioral patterns,

[0801] A means for generating appropriate virtual battle units based on analysis results,

[0802] Means for adjusting the behavior of the virtual battle unit according to the user's real-time results,

[0803] A means of collecting user feedback and incorporating it into future production processes,

[0804] The generated virtual battle unit is a means of determining ability parameters and strategies using a generated AI model,

[0805] A system that includes this.

[0806] (Claim 2)

[0807] The system according to claim 1, further comprising means for adapting a situation or environment based on the user's preferences.

[0808] (Claim 3)

[0809] The system according to claim 1, further comprising means for generating cooperative partners and improving the user experience.

[0810] "Application Example 1"

[0811] (Claim 1)

[0812] Means for collecting user information,

[0813] A means for analyzing the collected user information and identifying the user's preferences and tendencies,

[0814] A means for generating the optimal virtual recommendation target based on the analysis results,

[0815] Means for adjusting the virtual recommendation target in response to the user's real-time response,

[0816] A system that includes means for collecting user feedback and incorporating it into future generation processes.

[0817] (Claim 2)

[0818] The system according to claim 1, further comprising means for customizing content or settings based on user preferences.

[0819] (Claim 3)

[0820] The system according to claim 1, further comprising means for generating collaborative recommendation partners to improve the user experience.

[0821] "Example 2 of combining an emotion engine"

[0822] (Claim 1)

[0823] Means for collecting user information,

[0824] A means for analyzing the collected user information and identifying the user's emotional state,

[0825] A means for generating the optimal virtual opponent based on the analysis results,

[0826] Means for adjusting the behavioral characteristics of the virtual opponent according to the emotional state of the user,

[0827] A system that includes means for collecting user feedback and incorporating it into the scenario generation process for future scenarios.

[0828] (Claim 2)

[0829] The system according to claim 1, further comprising means for customizing a scenario or environment based on the user's emotional state.

[0830] (Claim 3)

[0831] The system according to claim 1, further comprising means for generating cooperative players and improving the user experience.

[0832] "Application example 2 when combining with an emotional engine"

[0833] (Claim 1)

[0834] Means of collecting emotion-related information,

[0835] A means for analyzing the collected emotion-related information and identifying the user's mental state,

[0836] Means for generating an optimal virtual dialogue target based on the aforementioned mental state,

[0837] Means for adjusting the behavior of the virtual dialogue target according to its emotional state,

[0838] A system that includes means for collecting user feedback and incorporating it into future generation processes.

[0839] (Claim 2)

[0840] The system according to claim 1, further comprising means for customizing a scenario or environment based on the user's emotional state.

[0841] (Claim 3)

[0842] The system according to claim 1, further comprising means for generating a collaborative interaction partner and improving user comfort. [Explanation of symbols]

[0843] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means of collecting player data, A means for analyzing the collected player data to identify the player's skill level and behavioral patterns, A means for generating the optimal virtual opponent based on the analysis results, Means for adjusting the behavioral patterns of the virtual opponent in accordance with the real-time performance of the player, A system that includes means for collecting player feedback and incorporating it into future generation processes.

2. The system according to claim 1, further comprising means for customizing a scenario or environment based on the player's preferences.

3. The system according to claim 1, further comprising means for generating cooperative play partners and improving the player's gameplay experience.

Images