system

Abstract

Providing an engaging gaming experience for players. [Solution] A system including: a means for generating a character image using an image generation algorithm based on attribute information; a means for generating movement using an algorithm that generates movement data based on the generated character image; a means for synthesizing voice using dialogue data based on a scenario or story; a means for generating conversations or stories using an NLP model; a means for detecting bugs in code using a bug detection AI and generating debug instructions; and a means for optimizing game difficulty and reward settings based on game progress and player data.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The game market has expanded rapidly in recent years, creating a demand for a continuous supply of new and engaging content for players. However, traditional game development requires significant resources and time, and specialized skills are essential for creative tasks such as character generation, motion production, voice synthesis, and story generation. In addition, bug detection and debugging require significant manpower, and balancing game difficulty and rewards is also time-consuming. This makes it difficult to develop games quickly and provide high-quality content. To solve these challenges, a system that utilizes generative AI to enable efficient and creative game development is needed. [Means for solving the problem]

[0005] This invention relates to a system that utilizes generative AI to improve game development efficiency and creativity, providing endlessly enjoyable games. The system includes a means for generating character images, a means for generating movements based on the generated character images, a means for synthesizing voice based on dialogue, a means for generating dialogue and stories, a means for detecting bugs and generating debug instructions, and a means for optimizing game difficulty and reward settings. Specifically, the system generates character images using an image generation algorithm based on attribute information, and generates movements using an algorithm for generating motion data based on the character images. It also synthesizes voice using dialogue data based on a scenario or story, and generates dialogue and stories using an NLP model. Furthermore, it uses bug detection AI to detect bugs in the code, generates debug instructions, and releases fix patches. By optimizing difficulty and reward settings based on game progress and player data, the system provides players with an engaging gaming experience.

[0006] The "means for generating a character image" is a method for automatically creating a visual representation of a character using an image generation algorithm based on attribute information.

[0007] The "means for generating movement based on the generated character image" is an algorithm that uses the generated character image to generate movement data for the character to move within the game.

[0008] The "means for synthesizing voice based on dialogue" is a voice synthesis algorithm that analyzes dialogue data and automatically generates the voice of a character.

[0009] "Means for generating conversations and stories" refers to a method for automatically generating conversations between characters and story progression within a game using artificial intelligence.

[0010] The "means for detecting bugs and generating debug instructions" is an algorithm for automatically detecting bugs in game code and providing instructions on how to fix them.

[0011] The "means for optimizing game difficulty and reward settings" is an algorithm that automatically adjusts the balance between game difficulty and rewards to an optimal state based on player data.

[0012] "Attribute information" is data that indicates a character's appearance and characteristics.

[0013] An "image generation algorithm" is a computational method that automatically generates an image based on input data.

[0014] "Movement data" is digital data that indicates how a character moves in the game.

[0015] "Dialogue data" is text data that indicates what a character says.

[0016] A "speech synthesis algorithm" is a computational method for generating speech based on text data.

[0017] An "NLP model" is an artificial intelligence framework for natural language processing.

[0018] "Bug detection AI" is an artificial intelligence system that automatically detects problems in program code.

[0019] "Debugging instructions" are specific work instructions for fixing the detected bug.

[0020] "Player data" refers to data relating to the game's progress and the player's actions. [Brief explanation of the drawings]

[0021] [Figure 1]1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] This invention relates to a game system that utilizes generative AI to provide endless fun. The system includes multiple algorithms and modules for generating character images, generating movements, synthesizing voices, generating dialogue and stories, detecting and debugging bugs, and optimizing game difficulty and reward settings.

[0043] First, the server receives a request from the user and generates a character image. This uses an image generation algorithm based on attribute information. The user selects the character's appearance and characteristics on the game's start screen, and the device sends this selection to the server. The server creates a character image using an image generation algorithm such as StableDiffusion based on the received attribute information, and sends the generated image back to the device. The device displays this image to the user.

[0044] The server then generates character movement based on the generated character image. This is done using algorithms such as MotionDiffuse. The device uploads the character image to the server, which analyzes the image and generates movement data. The generated movement data is sent to the device, which applies it to the character and displays it.

[0045] Furthermore, for voice synthesis, the server generates voice based on the dialogue data. Dialogue data is generated from the scenario or story, and this is input into a voice synthesis algorithm using LLM. The generated voice data is sent to the device, which then plays it in the game.

[0046] Conversations and stories are generated by the server using an NLP model. When a user interacts with an NPC character in the game, the device sends that context data to the server. The server uses this context data to generate conversations between the user and the NPC and the ongoing story, and sends them to the device. The device then displays the generated data to the user.

[0047] Bug detection and debug instructions are carried out by a bug detection AI on the server. When a user discovers a bug during the game, they report the problem to the server via their device. The server analyzes the report and uses AI to detect bugs in the code. Debug instructions for the detected bug are generated and sent to the development team. The problem is resolved by releasing a patch to fix the problem.

[0048] Finally, the server also optimizes the game's difficulty and reward settings. The device periodically sends the player's progress and gameplay data to the server. The server analyzes this data and sets the optimal balance between difficulty and reward. The new setting data is sent back to the device, which applies it to the game and provides it to the player.

[0049] Through the above process, this invention enables efficient and creative game development, reducing development efforts while providing players with a consistently new and engaging gaming experience.

[0050] The processing flow will be explained below.

[0051] Character generation process

[0052] Step 1:

[0053] The user selects the character's attributes (e.g., hair color, clothing, gender) on the game's start screen.

[0054] Step 2:

[0055] The terminal transmits the selected attribute data to the server.

[0056] Step 3:

[0057] Based on the received attribute data, the server invokes an image generation algorithm such as StableDiffusion to generate a character image.

[0058] Step 4:

[0059] The server transmits the generated character image to the terminal.

[0060] Step 5:

[0061] The terminal displays the received character image to the user.

[0062] Motion generation processing

[0063] Step 1:

[0064] The terminal uploads the generated character image to the server.

[0065] Step 2:

[0066] Based on the received character image, the server invokes algorithms such as MotionDiffuse to generate character movement data.

[0067] Step 3:

[0068] The server transmits the generated motion data to the terminal.

[0069] Step 4:

[0070] The device stores the received movement data locally and applies it to the character for display.

[0071] Speech synthesis processing

[0072] Step 1:

[0073] The server generates dialogue data based on a scenario or story.

[0074] Step 2:

[0075] The server passes the generated dialogue data to a speech synthesis algorithm to generate voice data.

[0076] Step 3:

[0077] The server transmits the generated voice data to the terminal.

[0078] Step 4:

[0079] The device stores the received audio data locally and plays it in the game.

[0080] Conversation / story generation processing

[0081] Step 1:

[0082] The user talks to an NPC character in the game.

[0083] Step 2:

[0084] The device transmits user input and the current game state to the server.

[0085] Step 3:

[0086] Based on the received context data, the server invokes an NLP model such as ChatGPT (registered trademark) to generate conversations and stories.

[0087] Step 4:

[0088] The server sends the generated conversations and stories to the device.

[0089] Step 5:

[0090] The terminal stores the received conversation data locally and displays it to the user.

[0091] Bug detection and debug instruction handling

[0092] Step 1:

[0093] A user discovers a bug in the game and reports the problem through the "Report a Bug" form.

[0094] Step 2:

[0095] The terminal sends the report to the server.

[0096] Step 3:

[0097] The server analyzes the received report and calls a bug detection AI to detect bugs in the code.

[0098] Step 4:

[0099] The server generates debugging instructions based on the detected bug information and sends them to the development team.

[0100] Step 5:

[0101] The server applies the corrected code or patch to the game and notifies the device of the update.

[0102] Difficulty and reward setting optimization process

[0103] Step 1:

[0104] The device transmits the player's progress and gameplay data to the server.

[0105] Step 2:

[0106] Based on the received player data, the server invokes an algorithm to optimize difficulty and reward settings and analyzes the data.

[0107] Step 3:

[0108] Based on the analysis results, the server generates new difficulty and reward settings and sends them to the device.

[0109] Step 4:

[0110] The device applies the received new setting data to the game and reflects it to the player.

[0111] The above is a specific program processing flow for the embodiment of the invention. This system enables efficient and creative game development, and constantly provides new content to players.

[0112] Example 1

[0113] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0114] In modern games, it is extremely important to be able to highly customize character appearance, movement, voice, dialogue, and story, as well as efficiently detect and fix bugs and adjust difficulty and rewards. However, there is still a lack of a system that can integrate these diverse elements and provide them to users in real time. Furthermore, there is also a lack of a means to automatically and effectively generate and optimize these elements. Therefore, improving the efficiency of game development while providing players with a fresh and engaging experience is a challenge.

[0115] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0116] In this invention, the server includes: [means for generating a character image using digital image generation means based on property information upon receiving a request from a user; [means for generating movement data from the digital image based on the generated character image;] [means for synthesizing voice using a natural language processing model based on lines generated from a scenario;] [means for generating dialogue and a story using a natural language processing model based on user input;] [means including artificial intelligence for detecting system malfunctions and generating correction instructions based thereon; and [means for analyzing user progress and gameplay data to optimize game difficulty and reward settings.] This makes it possible [to generate and optimize a variety of game elements in real time and provide players with an always fresh and engaging game experience].

[0117] "User" refers to the person who operates the system and plays the game.

[0118] "Attribute information" refers to attribute data necessary for generating images and movements, such as a character's appearance and characteristics.

[0119] "Digital image generation means" refers to algorithms or software for generating a digital image based on property information.

[0120] "Character image" refers to digital image data that represents the appearance of a generated character.

[0121] "Movement data" refers to digital data generated to represent the movements of a character.

[0122] "Scenario" refers to text data that includes instructions such as the story and dialogue used in the game.

[0123] "Natural language processing model" refers to a machine learning model used to understand and generate human language.

[0124] "Speech synthesis" refers to the technology of generating voice data from text data.

[0125] "Dialogue and Narrative" refers to the streams of text and dialogue that contain interactions between the user and in-game characters and NPCs.

[0126] "System malfunction" refers to an abnormality or error that occurs in software or hardware.

[0127] "Artificial intelligence" refers to technology that analyzes data, recognizes patterns, and solves problems through learning and inference.

[0128] "Progression" refers to a player's achievements or progress within a game.

[0129] "Gameplay data" refers to information such as player behavior, scores, and play history.

[0130] "Difficulty and reward settings" refers to the balance between the level of challenge in the game and the rewards obtained in return.

[0131] This invention relates to a game system that utilizes generative AI to provide endless fun. The system includes multiple algorithms and modules for generating character images, generating movements, synthesizing voices, generating dialogue and stories, detecting and debugging bugs, and optimizing game difficulty and reward settings.

[0132] Character image generation

[0133] First, the user selects the character's appearance and characteristics on the game's start screen. The specific steps are explained below.

[0134] The user selects the character's appearance and characteristics.

[0135] Example: User selects "Character with blue hair."

[0136] The terminal transmits the user's selection information to the server.

[0137] Example prompt: "Submit blue hair info for character generation."

[0138] The server generates a character image based on the received property information using a digital image generation means such as StableDiffusion.

[0139] The generated character image is sent back to the terminal, which displays this image to the user.

[0140] Example: A blue-haired character appears on the device screen.

[0141] Generating character movements

[0142] Next, a procedure for generating character movements based on the generated character image will be described.

[0143] The terminal uploads the generated character image to the server.

[0144] Example prompt: "Send an image of a blue-haired character to the server."

[0145] The server generates the motion data using an algorithm such as MotionDiffuse.

[0146] Example: The server generates walking motion data for a blue-haired character.

[0147] The generated movement data is sent to the terminal, which applies it to the character and displays it.

[0148] An animation of a blue-haired character walking appears on the device screen.

[0149] Speech synthesis

[0150] The procedure for synthesizing voice based on character lines is explained below.

[0151] The server generates dialogue data from a scenario or story.

[0152] Example: Generate dialogue data for "Hello"

[0153] The server generates the voice using a speech synthesis algorithm that uses LLM.

[0154] Example prompt: "Generate the greeting 'hello' aloud."

[0155] The generated audio data is sent to the terminal, which then plays it back in the game.

[0156] Example: A character in a game says "Hello."

[0157] Conversation and story generation

[0158] The steps for generating conversations between users and NPCs and game stories are explained below.

[0159] The user provides input to an NPC in the game.

[0160] Example: A user asks, "What's the next mission?"

[0161] The terminal transmits the context data to the server.

[0162] Example prompt: Send the question "What's your next mission?"

[0163] The server uses NLP models to generate dialogue and stories.

[0164] Example: Generate a description for the next mission

[0165] The generated data is sent to the terminal, which displays it to the user.

[0166] Example: An NPC in a game explains the details of the next mission.

[0167] Bug detection and debugging instructions

[0168] The procedure for detecting system malfunctions and generating debug instructions is described below.

[0169] When a user discovers a bug during the game, they report the problem to the server via their device.

[0170] Example prompt: "The character walks through the wall."

[0171] The server analyzes the reports and uses AI to detect bugs.

[0172] Example: Server detects bug that allows walking through walls

[0173] The server generates debug instructions and sends them to the development team.

[0174] Example: Generate specific debug instructions for fixing a bug

[0175] The development team makes corrections based on the debugging instructions and releases the corrected version.

[0176] Example: The modified game is provided to the user

[0177] Optimizing game difficulty and reward settings

[0178] The procedure for optimizing the game difficulty and reward settings is described below.

[0179] The device periodically transmits the player's progress and gameplay data to the server.

[0180] Example prompt: "Send data such as game completion time and success rate."

[0181] The server analyzes the received data and optimizes the balance between difficulty and reward.

[0182] Example: Adjusting difficulty based on player performance

[0183] The server sends the new difficulty and reward setting data back to the device.

[0184] Example: Sending optimized configuration data to the device

[0185] The device will apply the new configuration data to the game and reflect it to the player.

[0186] Example: The game restarts with a new difficulty level

[0187] By following the above procedure, the system of the present invention can generate and optimize a variety of game elements in real time, and provide users with a constantly fresh and engaging game experience.

[0188] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0189] Step 1:

[0190] The user selects the character's appearance and characteristics.

[0191] Input: User-selected character appearance and characteristics (e.g., a character with blue hair).

[0192] Specific operation: The user selects the character's attributes on the game's start screen.

[0193] Output: The selected character attribute information is generated on the terminal.

[0194] Step 2:

[0195] The terminal transmits the user's selection information to the server.

[0196] Input: Character selection information (attribute data for a blue-haired character).

[0197] Specific operation: The terminal transmits the selection information to the server via the network.

[0198] Output: The selection information sent to the server.

[0199] Step 3:

[0200] The server generates a character image based on the received property information using a digital image generation algorithm.

[0201] Input: Character attribute data (blue hair information).

[0202] Data processing or data calculation: Using a digital image generation algorithm such as StableDiffusion, a character image is generated based on the specified attribute information.

[0203] Specific operation: The server runs StableDiffusion and creates a character image.

[0204] Output: The generated character image.

[0205] Step 4:

[0206] The server transmits the generated character image to the terminal.

[0207] Input: Generated character image.

[0208] Specific operation: The server sends the generated image back to the terminal via the network.

[0209] Output: Character image sent to the device.

[0210] Step 5:

[0211] The terminal displays the received character image to the user.

[0212] Input: Character image sent from the server.

[0213] Specific operation: Display a character image on the device screen.

[0214] Output: The character image displayed to the user.

[0215] Step 6:

[0216] The terminal uploads the generated character image to the server.

[0217] Input: Character image stored on the device.

[0218] Specific operation: The device uploads the character image to the server via the network.

[0219] Output: Character image uploaded to the server.

[0220] Step 7:

[0221] The server generates movement data based on the character image.

[0222] Input: Uploaded character image.

[0223] Data processing or data calculation: Using algorithms such as MotionDiffuse to analyze character images and generate motion data.

[0224] Specific operation: The server executes MotionDiffuse and creates character movement data.

[0225] Output: Generated character motion data.

[0226] Step 8:

[0227] The server transmits the generated exercise data to the terminal.

[0228] Input: Generated movement data.

[0229] Specific operation: The server sends the exercise data to the terminal via the network.

[0230] Output: Exercise data sent to the device.

[0231] Step 9:

[0232] The device applies the received movement data to the character and displays it.

[0233] Input: Exercise data sent from the server.

[0234] Specific operation: The device applies the movement data to the character and displays it on the screen.

[0235] Output: A moving character displayed on the terminal.

[0236] Step 10:

[0237] The server synthesizes voice based on the dialogue data generated from the scenario.

[0238] Input: Scenario and dialogue data.

[0239] Data processing or data computation: Generate speech data from dialogue data using a natural language processing model.

[0240] Specific operation: The server executes a speech synthesis algorithm to convert the dialogue data into voice data.

[0241] Output: The generated audio data.

[0242] Step 11:

[0243] The server transmits the generated voice data to the terminal.

[0244] Input: The generated audio data.

[0245] Specific operation: The server sends the audio data to the terminal via the network.

[0246] Output: The audio data sent to the device.

[0247] Step 12:

[0248] The device then plays the received audio data as the character's lines.

[0249] Input: Audio data sent from the server.

[0250] Specific operation: The device applies the voice data to the character and plays it in the game.

[0251] Output: Character dialogue played in-game.

[0252] Step 13:

[0253] The context data entered by the user is transmitted from the terminal to the server.

[0254] Input: A question or request that the user types in the game.

[0255] Specific operation: The terminal sends user input to the server.

[0256] Output: The context data sent to the server.

[0257] Step 14:

[0258] The server generates conversations and stories based on the contextual data.

[0259] Input: Context data.

[0260] Data manipulation or data computation: Using natural language processing models to generate conversations or stories.

[0261] What it does: The server runs NLP models and creates conversations and stories.

[0262] Output: Generated conversation or story data.

[0263] Step 15:

[0264] The server sends the generated conversation and story data to the terminal.

[0265] Input: Generated conversation or story data.

[0266] Specific operation: The server sends data to the terminal via the network.

[0267] Output: Conversation or story data sent to your device.

[0268] Step 16:

[0269] The terminal displays the received conversation and story data to the user.

[0270] Input: Conversation or story data sent from the server.

[0271] Specific behavior: The device displays the data on the screen.

[0272] Output: The conversation or story that is displayed to the user.

[0273] Step 17:

[0274] If a user discovers a bug during the game, they report the problem to the server via their device.

[0275] Input: The description of the bug found by the user.

[0276] Specific operation: A user reports a bug to the server through a terminal.

[0277] Output: The bug report sent to the server.

[0278] Step 18:

[0279] The server analyzes the reported bug and uses AI to detect it.

[0280] Input: Bug report data.

[0281] Data manipulation or data manipulation: Using bug detection AI to analyze reports and detect bugs.

[0282] Specific operation: The server runs AI to identify the location and cause of the bug.

[0283] Output: Detected bug data.

[0284] Step 19:

[0285] The server generates debug instructions for the detected bugs and sends them to the development team.

[0286] Input: Detected bug data.

[0287] What happens: The server generates debug instructions and sends them to the development team.

[0288] Output: Debugging instructions sent to the development team.

[0289] Step 20:

[0290] The device periodically transmits the player's progress and gameplay data to the server.

[0291] Input: Player progress data, gameplay data.

[0292] Specific operation: The device sends progress and play data to the server.

[0293] Output: Progress data and play data sent to the server.

[0294] Step 21:

[0295] The server analyzes the received data and optimizes the balance between difficulty and reward.

[0296] Input: Progress data, play data.

[0297] Data processing or data arithmetic: Using data analysis algorithms to calculate the optimal balance between difficulty and reward.

[0298] Specific operation: The server runs the algorithm and adjusts the difficulty and reward.

[0299] Output: New difficulty and reward settings data.

[0300] Step 22:

[0301] The server sends the new difficulty and reward setting data back to the device.

[0302] Input: New difficulty and reward settings data.

[0303] Specific operation: The server sends the setting data to the terminal.

[0304] Output: The new configuration data sent to the terminal.

[0305] Step 23:

[0306] The device will apply the new configuration data to the game and reflect it to the player.

[0307] Input: New difficulty and reward settings data.

[0308] Specific operation: The device applies and reflects the new configuration data in the game.

[0309] Output: The game environment with the new settings.

[0310] (Application example 1)

[0311] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0312] It is necessary to make the virtual shopping experience more engaging and interactive, provide an entertainment-rich shopping experience that users can enjoy endlessly, and increase purchasing motivation by providing product recommendations and navigation in real time based on users' preferences.

[0313] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0314] In this invention, the server includes: [means for generating character images;] [means for generating movements based on the generated character images;] [means for synthesizing voice based on lines;] [means for generating conversations and stories;] [means for detecting bugs and generating debugging instructions;] [means for optimizing game difficulty and reward settings;] [means for recommending products based on user attribute information and displaying them as navigation characters;] [voice synthesis means for explaining products using the voice lines of the generated characters; and [means for generating conversations and dialogues based on interactions with the user. This makes it possible to provide users with an endlessly enjoyable virtual shopping experience and increase their desire to purchase.

[0315] A "character image" is a visual representation of a virtual person or object generated based on the user's attribute information.

[0316] "Movement data" is data on the actions and movements that a character appears to perform based on the generated character image.

[0317] "Speech synthesis" is the process of creating artificially generated speech based on text data.

[0318] "Conversations and stories" refer to dialogues and scenarios created based on user interaction.

[0319] "Bug detection" is the process of discovering errors or defects in software or systems.

[0320] "Debugging instructions" are specific steps or instructions for fixing a detected bug.

[0321] "Optimizing difficulty and reward settings" is the process of adjusting the appropriate challenge and reward levels depending on the progress of a game or experience.

[0322] "User attribute information" is data that includes user preferences, behavioral history, personal settings, etc.

[0323] "Product recommendation" is the process of suggesting highly relevant products based on a user's attribute information.

[0324] A "navigation character" is a virtual character that acts as a guide for users within a virtual store.

[0325] An "interaction" is an interaction between a user and a system.

[0326] This invention relates to a system that provides users with an endlessly enjoyable virtual shopping experience. The system uses a generative AI model and multiple algorithms and modules to generate character images, generate movements, synthesize voices, and generate conversations and stories. It also includes a function that recommends products based on the user's attribute information and displays them as navigation characters.

[0327] The server generates a character image upon receiving a request from the user. This uses StableDiffusion, an image generation algorithm based on the user's attribute information. The user selects the character's appearance and characteristics on their smartphone and sends their selection to the server. The server creates a character image based on the received attribute information and sends the generated image back to the device. The device displays this image to the user.

[0328] The server then uses algorithms such as MotionDiffuse to generate character movement based on the generated character image. The device uploads the character image to the server, which analyzes the image and generates motion data. The generated motion data is then sent to the device, which applies it to the character and displays it.

[0329] The server then generates voice based on the dialogue data. Dialogue data is generated from the scenario or story, and this is input into a speech synthesis algorithm using a large-scale language model (LLM). The generated voice data is sent to the device, which then plays it back to the user.

[0330] The server generates conversations and stories using NLP models. When the user interacts with the navigation character, the device sends the context data to the server. The server generates conversations between the user and the character and the ongoing story based on this context data and sends it to the device. The device then displays the generated data to the user.

[0331] Bug detection and debug instructions are performed by a bug detection AI on the server. When a user finds a bug in the system, they report the problem to the server via their terminal. The server analyzes the report and uses AI to detect bugs in the code. Debug instructions for the detected bug are generated and sent to the development team. The problem is resolved by releasing a patch to fix it.

[0332] Finally, the system periodically transmits user progress and interaction data to the server, which analyzes this data and configures the system to optimize the balance between difficulty and reward. The new configuration data is sent back to the device, which then applies it to the system and reflects it to the user.

[0333] For example, the following conversation might be generated by a generative AI model:

[0334] User: "What are your recent recommendations?"

[0335] AI: "My latest recommendation is a newly released smartwatch. It features a long battery life and comprehensive health tracking."

[0336] In this way, the system can provide users with an endlessly enjoyable virtual shopping experience and increase their desire to purchase.

[0337] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0338] Step 1:

[0339] The user selects the character's appearance and characteristics using a smartphone device.

[0340] Input: User selection information (attribute information)

[0341] Output: Sends user selection information to the server

[0342] Specific operation: The user selects the character's appearance and characteristics on the application interface and sends that information to the server.

[0343] Step 2:

[0344] The server generates a character image using StableDiffusion based on the attribute information received from the user.

[0345] Input: User attribute information

[0346] Output: Generated character image

[0347] Specific operation: The server uses the StableDiffusion algorithm to generate a character image from the user's attribute information.

[0348] Step 3:

[0349] The server returns the generated character image to the terminal.

[0350] Input: Generated character image

[0351] Output: Character image returned to the device

[0352] Specific operation: The server sends the generated character image to the terminal, and the terminal displays this image.

[0353] Step 4:

[0354] The device uploads the character image to the server, and the server generates the motion data using MotionDiffuse.

[0355] Input: Character image

[0356] Output: Generated motion data

[0357] Specific operation: The device uploads the character image to the server, and the server generates motion data using the MotionDiffuse algorithm.

[0358] Step 5:

[0359] The server transmits the generated motion data to the terminal.

[0360] Input: Generated motion data

[0361] Output: Movement data sent to the device

[0362] Specific operation: The server sends the generated movement data to the device, which then applies it to the character and displays it.

[0363] Step 6:

[0364] The server synthesizes voice based on the scenario or story.

[0365] Input: Scenario and story dialogue data

[0366] Output: Generated audio data

[0367] Specific operation: The server generates speech from dialogue data using a speech synthesis algorithm based on a large-scale language model (LLM) and sends it to the terminal.

[0368] Step 7:

[0369] The terminal reproduces the generated voice data and lets the user hear it.

[0370] Input: Generated audio data

[0371] Output: The audio played to the user

[0372] Specific operation: The device plays the audio data received from the server.

[0373] Step 8:

[0374] The user interacts with the navigation character and the terminal transmits the context data to the server.

[0375] Input: User interaction

[0376] Output: Context data sent to the server

[0377] Specific operation: The user interacts with the navigation character, and the content of the interaction is sent from the terminal to the server as context data.

[0378] Step 9:

[0379] The server uses NLP models to generate conversations and stories and send them to the device.

[0380] Input: Context data

[0381] Output: Generated conversations and story data

[0382] Specific operation: The server analyzes the context data using an NLP model, generates a conversation or story, and sends it to the device.

[0383] Step 10:

[0384] The terminal displays the generated conversations and stories to the user.

[0385] Input: Generated conversation or story data

[0386] Output: The conversation or story that is displayed to the user

[0387] Specific operation: The device displays the conversation and story data received from the server and shows it to the user.

[0388] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0389] This invention relates to a system that utilizes generative AI and an emotion engine to provide users with an endlessly enjoyable gaming experience. This system includes multiple algorithms and modules that generate character images and create movements, voices, dialogue, and stories. It also incorporates an emotion engine that recognizes the user's emotions and dynamically changes the game content.

[0390] First, the system generates a character image based on the attributes of the character selected by the user. The user selects the character's appearance and characteristics on the game's opening screen, and the device sends that selection to the server. The server uses an image generation algorithm (e.g., StableDiffusion) to create a character image based on the received attribute information, and sends the generated image back to the device. The device then displays this image to the user.

[0391] Next, the server generates character movement based on the generated character image. The device uploads the character image to the server, and the server analyzes the image and invokes an algorithm (such as MotionDiffuse) to generate motion data. The generated motion data is sent to the device, which applies it to the character and displays it.

[0392] For voice synthesis, the server generates voice based on dialogue data. Dialogue data is generated from the scenario or story, and this is input into the voice synthesis algorithm. The generated voice data is sent to the device, which plays it in the game.

[0393] Conversations and stories are generated by an NLP model (e.g., ChatGPT) on the server. When a user talks to an NPC character in the game, the device sends context data to the server. The server generates conversations and stories based on this context data and sends them to the device. The device then displays the generated data to the user.

[0394] The emotion engine has the ability to recognize emotions by analyzing the user's facial expression and voice data. The device collects data provided by the user through the camera and microphone and sends it to the server. The server inputs the received data into the emotion engine and recognizes the user's emotional state. The game content is dynamically changed based on the emotion recognition results. For example, if the user is surprised, an enemy character in the game can suddenly appear.

[0395] Bug detection and debug instructions are carried out by a bug detection AI on the server. When a user discovers a bug during the game, they report the problem to the server via their device. The server analyzes the report and uses AI to detect bugs in the code. Debug instructions for the detected bug are generated and sent to the development team. The problem is resolved by releasing a patch to fix the problem.

[0396] Finally, the server also optimizes the game's difficulty and reward settings. The device periodically sends the player's progress and gameplay data to the server. The server analyzes this data and sets the optimal balance between difficulty and reward. The new setting data is sent back to the device, which applies it to the game and provides it to the player.

[0397] For example, if a user selects red hair and armor as their costume during character creation, the server generates an image of the character wearing red hair and armor based on those attributes. The server then generates movements and sounds for the character, and the emotion engine recognizes the user's emotions and dynamically changes in-game events. For example, if the user looks tired, an event will occur in which a friendly NPC in the game offers a support item.

[0398] Through the above process, this invention enables efficient and creative game development, reducing development effort while providing players with a consistently new and engaging gaming experience. Furthermore, incorporating the user's emotional state can provide a more personalized gaming experience.

[0399] The processing flow will be explained below.

[0400] Character generation process

[0401] Step 1:

[0402] The user selects the character's attributes (e.g., hair color, clothing, gender) on the game's start screen.

[0403] Step 2:

[0404] The terminal transmits the selected attribute data to the server.

[0405] Step 3:

[0406] Based on the received attribute data, the server invokes an image generation algorithm (e.g., StableDiffusion) to generate a character image.

[0407] Step 4:

[0408] The server transmits the generated character image to the terminal.

[0409] Step 5:

[0410] The terminal displays the received character image to the user.

[0411] Motion generation processing

[0412] Step 1:

[0413] The terminal uploads the generated character image to the server.

[0414] Step 2:

[0415] The server calls an algorithm (e.g., MotionDiffuse) that generates motion data based on the received character image.

[0416] Step 3:

[0417] The server transmits the generated motion data to the terminal.

[0418] Step 4:

[0419] The terminal applies the received motion data to the character and displays the motion to the user.

[0420] Speech synthesis processing

[0421] Step 1:

[0422] The server generates dialogue data based on a scenario or story.

[0423] Step 2:

[0424] The server passes the dialogue data to a speech synthesis algorithm to generate speech data.

[0425] Step 3:

[0426] The server transmits the generated voice data to the terminal.

[0427] Step 4:

[0428] The device plays the received audio data in the game.

[0429] Conversation / story generation processing

[0430] Step 1:

[0431] The user talks to an NPC character in the game.

[0432] Step 2:

[0433] The device transmits user input and the current game state to the server.

[0434] Step 3:

[0435] The server invokes an NLP model (e.g., ChatGPT) based on the received context data to generate a conversation or story.

[0436] Step 4:

[0437] The server sends the generated conversations and stories to the device.

[0438] Step 5:

[0439] The terminal displays the received conversation data to the user.

[0440] Emotion Recognition Processing

[0441] Step 1:

[0442] The user provides facial expression data and voice data through a camera and microphone while playing the game.

[0443] Step 2:

[0444] The terminal transmits the collected facial expression data and voice data to the server.

[0445] Step 3:

[0446] The server inputs the received data into an emotion engine to recognize the user's emotion.

[0447] Step 4:

[0448] The server dynamically adjusts in-game events and conversations based on the emotion recognition results.

[0449] Step 5:

[0450] The device displays the adjusted events and conversations in the game and reflects them to the user.

[0451] Bug detection and debug instruction handling

[0452] Step 1:

[0453] A user discovers a bug in the game and reports the problem through a bug report form.

[0454] Step 2:

[0455] The terminal sends the report to the server.

[0456] Step 3:

[0457] The server analyzes the received report and calls a bug detection AI to detect bugs in the code.

[0458] Step 4:

[0459] The server generates debug instructions for the detected bugs and sends them to the development team.

[0460] Step 5:

[0461] The server applies the corrected code or patch to the game and notifies the device of the update.

[0462] Difficulty and reward setting optimization process

[0463] Step 1:

[0464] The device transmits the player's progress and gameplay data to the server.

[0465] Step 2:

[0466] Based on the received player data, the server invokes an algorithm to optimize difficulty and reward settings and analyzes the data.

[0467] Step 3:

[0468] Based on the analysis results, the server generates new difficulty and reward settings and sends them to the device.

[0469] Step 4:

[0470] The device applies the received new setting data to the game and reflects it to the player.

[0471] The above is a specific program processing flow for implementing the invention in combination with an emotion engine. This system provides a personalized gaming experience that incorporates the user's emotions.

[0472] Example 2

[0473] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0474] In conventional game systems, it has been difficult to dynamically adapt game progression and generate interactive characters that reflect the user's individual emotional state. Furthermore, bug detection and game difficulty adjustment are performed manually, resulting in low efficiency and limited improvements to the user experience. New algorithms and technologies are needed to address these challenges.

[0475] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0476] In this invention, the server includes means for generating character images, means for generating movements based on the generated character images, means for synthesizing voice based on dialogue, means for generating conversations and stories, means for recognizing the user's emotional state and dynamically changing game content, means for detecting bugs and generating debugging instructions, and means for optimizing game difficulty and reward settings. This enables a dynamic and personalized game experience that matches the user's emotions, and makes it possible to provide a high-quality game service by automatically detecting bugs and automatically adjusting game balance.

[0477] "Character image" is graphic data of a visual character used in the game, generated based on attribute information selected by the user.

[0478] "Movement data" is a series of data generated based on a character image to represent the movement of the character.

[0479] "Dialogue data" is text data of words spoken by characters that is generated based on the scenario or story within the game.

[0480] A "voice synthesis algorithm" is a calculation procedure or program for generating voice data based on dialogue data.

[0481] A "conversation generation algorithm" is a computational procedure or program for generating dialogue and stories between characters in a game.

[0482] An "emotion engine" is a group of algorithms and programs that analyze a user's facial expression data and voice data to recognize their emotional state.

[0483] "Bug detection AI" is an artificial intelligence that automatically detects glitches (bugs) that occur in games and generates instructions for debugging.

[0484] "Difficulty and reward optimization" is the process of adjusting the balance of a game's difficulty and available rewards based on player progress and gameplay data.

[0485] An "image generation algorithm" is a calculation procedure or program for generating a new character image based on attribute information selected by the user.

[0486] "Dynamic change" refers to changing the content and progress of the game in real time according to the user's emotional state and the game situation.

[0487] MODE FOR CARRYING OUT THE INVENTION

[0488] This invention is a system that utilizes generative AI and an emotion engine to provide users with an endlessly enjoyable gaming experience. To implement the invention, specific hardware and software are required to process and calculate various data.

[0489] Hardware and software used

[0490] This system operates in a network environment that includes user devices (PCs, smartphones, etc.) and servers. Specific software includes image generation algorithms (e.g., StableDiffusion), motion generation algorithms (e.g., MotionDiffuse), speech synthesis algorithms, natural language processing models (e.g., ChatGPT), and emotion engines.

[0491] System Operation Overview

[0492] 1. Character image generation

[0493] The user selects the character's appearance and characteristics on the game start screen. The device sends the selection to the server, which uses an image generation algorithm to generate a character image based on the received attribute information. This image is sent back to the device and displayed.

[0494] 2. Character Movement Generation

[0495] The device sends the generated character image to the server, which uses a motion generation algorithm to generate character movements from the image, and the generated motion data is sent back to the device and applied to the character.

[0496] 3. Speech generation

[0497] Based on the dialogue data generated according to the scenario and story, the server uses a speech synthesis algorithm to generate voice data, which is then sent to the device and played back in the game.

[0498] 4. Conversation and story generation

[0499] When a user talks to an NPC character in a game, the device sends that context data to the server, which uses a natural language processing model to generate a dialogue or story, which is then sent to the device and displayed to the user.

[0500] 5. Emotion Recognition and Dynamic Change of Game Content

[0501] The device collects the user's facial expression and voice data through a camera and microphone and sends it to the server. The server then uses an emotion engine to recognize the user's emotional state. Based on this recognition, in-game events and character behavior are dynamically changed.

[0502] 6. Bug Detection and Debugging Instructions

[0503] If a user discovers a bug during the game, they report the problem to the server via their device. The server uses a bug detection AI to analyze the report and detect bugs in the code. Debug instructions are generated and sent to the development team.

[0504] 7. Optimizing difficulty and reward settings

[0505] The device periodically sends the player's progress and gameplay data to the server. The server analyzes this data and configures the game to optimize the balance between difficulty and rewards. The new configuration data is sent back to the device and applied to the game.

[0506] Specific examples

[0507] For example, if a user selects red hair and armor as their outfit, the server generates a character image wearing red hair and armor based on those attributes. The server then generates movements and sounds for the character, and the emotion engine recognizes the user's emotions and dynamically changes in-game events. For example, if the user shows a tired expression, an event occurs in which a friendly character in the game provides a support item.

[0508] Prompt Sentence Examples

[0509] "Generate an image of a character with red hair and armor."

[0510] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0511] Step 1: User character attribute selection

[0512] Description: The user selects the character's appearance and characteristics (hair color, outfit, etc.) on the game's start screen.

[0513] Input: User-selected character attribute information (e.g., hair color "red", costume "armor").

[0514] Output: Data that sends selected attribute information from the device to the server.

[0515] Specific operation: The user selects "red hair" and "armor," and the device sends that information to the server.

[0516] Step 2: Generate character images

[0517] Description: The server generates a character image using an image generation algorithm based on the received attribute information.

[0518] Input: Character attribute information selected by the user.

[0519] Output: Send the generated character image data to the terminal.

[0520] What it does: The server uses the StableDiffusion algorithm to generate an image of a character with red hair and armor, and sends it back to the device.

[0521] Step 3: Character Movement Generation

[0522] Description: The device uploads the generated character image to the server, and the server generates the character's movements using a movement generation algorithm.

[0523] Input: Generated character image.

[0524] Output: Send the generated motion data to the device.

[0525] How it works: The server uses the MotionDiffuse algorithm to generate character motion data and sends it to the device, which then applies this motion to the character.

[0526] Step 4: Generate audio

[0527] Description: The server generates dialogue data from a scenario or story and generates speech using a speech synthesis algorithm.

[0528] Input: Dialogue data (text format).

[0529] Output: Sends the generated audio data to the device.

[0530] What happens: The server uses a speech synthesis algorithm to generate a voice for the phrase "Hello" and sends it to the device, which then plays it back in the game.

[0531] Step 5: Generate conversations and stories

[0532] Description: When a user talks to an NPC character in a game, the device sends that context data to the server, which then generates the dialogue and story.

[0533] Input: Context data (user-NPC interactions, current scenario, etc.).

[0534] Output: Send the generated conversation data and story data to the device.

[0535] How it works: The server uses the ChatGPT algorithm to generate a conversation saying, "Go down this path and you'll find the treasure," and sends it to the device, which then displays it to the NPC in the game.

[0536] Step 6: Emotion recognition and dynamic change of game content

[0537] Description: The device collects the user's facial expression and voice data and sends it to the server. The server uses an emotion engine to recognize the user's emotional state and dynamically change the game content.

[0538] Input: facial expression data, audio data.

[0539] Output: Dynamically modified game content based on emotion recognition results.

[0540] Specific behavior: The server recognizes that the user is surprised and triggers an event in the game in which an enemy character suddenly appears.

[0541] Step 7: Bug detection and debugging instructions

[0542] Description: When a user discovers a bug during the game, they report the problem to the server through their terminal. The server uses a bug detection AI to detect bugs in the code and generate debug instructions.

[0543] Input: Bug report data.

[0544] Output: Debug instructions, fix patch.

[0545] What happens: The server analyzes the user's report that "the character can go through walls," detects the bug, sends a debug instruction to the development team, and releases a fix patch.

[0546] Step 8: Optimize difficulty and reward settings

[0547] Description: The device sends player progress and gameplay data to the server, which analyzes this data and configures the game to optimize the balance between difficulty and rewards.

[0548] Input: Player progress, gameplay data.

[0549] Output: Optimized difficulty and reward settings.

[0550] What it does: The server analyzes the player's high score, sets more difficult levels and rewards, and sends the new configuration data back to the device to apply it to the game.

[0551] (Application example 2)

[0552] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0553] Conventional security systems operate based on fixed settings, making it difficult to respond flexibly to user emotions and situations. Furthermore, there was no technology that could dynamically change security responses based on emotions in real time. As a result, even if a user is feeling anxious or nervous, it is not possible to provide appropriate safety measures, which could lead to a decline in the quality of security.

[0554] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0555] In this invention, the server includes: [means for generating character images;] [means for generating movements based on the generated character images;] [means for synthesizing voice based on dialogue;] [means for generating conversations and stories;] [means for detecting bugs and generating debugging instructions;] [means for optimizing game difficulty and reward settings;] [means for recognizing user emotions and dynamically changing system settings based on those emotions; and [means for dynamically generating emotion-based security measures using a generative AI model.] This enables appropriate security responses that are in line with user emotions to be provided in real time, improving the quality of security.

[0556] "Means for generating a character image" refers to an algorithm or module that creates an image of a character based on attribute information selected by a user.

[0557] "Means for generating movement based on the generated character image" refers to an algorithm or module that generates movement data from the generated character image.

[0558] "Means for synthesizing voice based on dialogue" refers to a voice synthesis algorithm or module that converts dialogue data from a scenario or story into voice.

[0559] "Means for generating conversations and stories" refers to natural language processing models and algorithms for generating conversations and stories with users.

[0560] "Means for detecting bugs and generating debug instructions" refers to algorithms or modules for analyzing problems reported by users and generating instructions required for debugging.

[0561] "Means for optimizing game difficulty and reward settings" refers to algorithms or modules that analyze users' gameplay data and optimally adjust the balance between difficulty and rewards.

[0562] "Means for recognizing user emotions and dynamically changing system settings based on those emotions" refers to algorithms or modules that analyze emotions from the user's facial expressions and voice and dynamically change system settings based on the results.

[0563] "Means for dynamically generating emotion-based security measures using a generative AI model" refers to an AI model that receives emotion data as input and generates security measures appropriate to those emotions.

[0564] This invention is a system that uses a generative AI model and an emotion engine to provide a dynamic gaming experience and security measures based on user emotions. The system includes multiple algorithms and modules and runs on devices such as smartphones and smart glasses.

[0565] The hardware required to implement the invention is a device equipped with a camera and microphone (e.g., a smartphone or smart glasses). The software uses Python and Django (server-side), an emotion API (e.g., Microsoft® Azure® Emotion API), and a natural language processing model (e.g., ChatGPT).

[0566] 1. Character image generation

[0567] The user uses the device to select the character's appearance and characteristics on the game's opening screen. This selection information is sent from the device to the server. The server generates a character image using an image generation algorithm (e.g., Stable Diffusion) based on the received attribute information, and sends the generated image back to the device. The device then displays this image to the user.

[0568] 2. Generating Character Movements

[0569] The generated character image is uploaded from the device to the server, which analyzes the image and invokes an algorithm (e.g., MotionDiffuse) to generate motion data. The generated motion data is sent to the device, which applies it to the character and displays it to the user.

[0570] 3. Speech Synthesis

[0571] The server generates dialogue data from the scenario and story, and synthesizes voice based on that. The dialogue data is input into a voice synthesis algorithm, and the generated voice data is sent to the device, which then plays it in the game.

[0572] 4. Conversation and story generation

[0573] When a user talks to an NPC character in a game, the device sends context data to the server, which generates a conversation or story based on the context data and sends it to the device, which then displays the generated data to the user.

[0574] 5. User Emotion Recognition and Dynamic System Changes

[0575] The device collects facial expression and voice data provided by the user through the camera and microphone and sends it to the server. The server inputs this data into an emotion engine (e.g., Microsoft Azure Emotion API) to recognize the user's emotional state. System settings are dynamically changed based on the emotion recognition results. For example, if the user feels anxious, security measures are strengthened.

[0576] 6. Emotion data collection using cameras and microphones

[0577] The camera and microphone are used to collect the user's facial expression and voice data. Specifically, the camera captures the user's facial expressions and the microphone records their voice. This data is sent to the emotion API in real time, and the user's emotional state is analyzed.

[0578] 7. Emotion-based security measures

[0579] Based on the emotion data, the server uses a generative AI model (e.g., ChatGPT) to dynamically generate security measures appropriate to the user's emotional state. For example, if the user feels anxious, the generative AI model will suggest security measures such as issuing a warning notification or automatically locking the smart lock.

[0580] Examples of examples and prompts:

[0581] If the user feels anxious at night, the data from the camera and microphone will be analyzed to generate the result "anxious." Based on this, the generative AI will increase the sensitivity of the window and door sensors and automatically generate instructions to lock all the doors in the house.

[0582] Text format

[0583] User emotion: anxious. Suggest measures to enhance home security during the user's anxious state.

[0584] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0585] Step 1:

[0586] The user selects the character's appearance and characteristics.

[0587] Input: Appearance and trait selection data.

[0588] How it works: The user selects the character's appearance and characteristics on the game's opening screen and enters them into the device.

[0589] Output: User-selected attribute information.

[0590] Step 2:

[0591] The terminal transmits the user-selected attribute information to the server.

[0592] Input: User-selected attribute information.

[0593] Operation: The terminal transmits the selected attribute information to the server.

[0594] Output: Attribute information received by the server.

[0595] Step 3:

[0596] The server generates a character image based on the attribute information.

[0597] Input: Attribute information.

[0598] Operation: The server inputs the attribute information into an image generation algorithm (e.g., StableDiffusion) to generate a character image.

[0599] Output: Character image.

[0600] Step 4:

[0601] The server returns the generated character image to the terminal.

[0602] Input: Generated character image.

[0603] Operation: The server sends the generated character image to the device.

[0604] Output: Character image received by the device.

[0605] Step 5:

[0606] The terminal displays the character image to the user.

[0607] Input: Received character image.

[0608] Operation: The device displays the character image on the screen.

[0609] Output: The character image displayed to the user.

[0610] Step 6:

[0611] Movement data is generated based on the character image.

[0612] Input: Character image.

[0613] How it works: The device uploads a character image to the server, which then invokes an algorithm (e.g., MotionDiffuse) that analyzes the image and generates motion data.

[0614] Output: Motion data.

[0615] Step 7:

[0616] The server transmits the generated motion data to the terminal.

[0617] Input: Generated motion data.

[0618] Operation: The server sends the generated motion data to the terminal.

[0619] Output: The motion data received by the device.

[0620] Step 8:

[0621] The device applies the movement data to the character and displays it.

[0622] Input: Received motion data.

[0623] Movement: The device applies the movement data to the character and displays it on the screen.

[0624] Output: A moving character displayed to the user.

[0625] Step 9:

[0626] The server generates dialogue data from the scenario or story and synthesizes the voice.

[0627] Input: A scenario or story.

[0628] How it works: The server generates dialogue data for a scenario or story, inputs it into a speech synthesis algorithm, and synthesizes the speech.

[0629] Output: Audio data.

[0630] Step 10:

[0631] The server transmits the generated voice data to the terminal.

[0632] Input: The generated audio data.

[0633] Operation: The server sends the generated voice data to the terminal.

[0634] Output: The audio data received by the device.

[0635] Step 11:

[0636] The device plays the audio data in the game.

[0637] Input: Received audio data.

[0638] Operation: The device plays the audio data in the game through the playback device.

[0639] Output: The audio played to the user.

[0640] Step 12:

[0641] The server generates conversations with NPC characters.

[0642] Input: A conversation request from the user.

[0643] How it works: When a user talks to an NPC character, the device sends context data to the server, which then uses a natural language processing model (e.g., ChatGPT) to generate a conversation or story.

[0644] Output: The generated conversation data.

[0645] Step 13:

[0646] The terminal displays the generated conversation data to the user.

[0647] Input: Generated conversation data.

[0648] Operation: The device displays the conversation data on the display.

[0649] Output: The conversation as displayed to the user.

[0650] Step 14:

[0651] The device collects the user's emotional data and sends it to the server.

[0652] Input: User's facial expression data, voice data.

[0653] How it works: It uses a camera and microphone to collect facial expression and voice data from the user and sends it to a server.

[0654] Output: Emotion data sent to the server.

[0655] Step 15:

[0656] The server analyzes the emotional data and recognizes the emotional state.

[0657] Input: Collected emotion data.

[0658] How it works: Emotion data is fed into an emotion engine (e.g., Microsoft Azure Emotion API) to analyze the emotional state.

[0659] Output: Parsed emotional state data.

[0660] Step 16:

[0661] The server generates security measures according to the emotional state.

[0662] Input: Emotional state data.

[0663] How it works: Based on the recognized emotional state data, a generative AI model (e.g., ChatGPT) is used to generate emotion-based security measures.

[0664] Output: The generated security measures.

[0665] Step 17:

[0666] The server transmits the generated security measures to the terminal.

[0667] Input: The generated security measures.

[0668] Operation: The server sends the generated security measures to the terminal.

[0669] Output: Security measures data received by the device.

[0670] Step 18:

[0671] The device implements security measures.

[0672] Input: Received security measures data.

[0673] Operation: The device will take necessary security measures based on the received security data, such as increasing the sensitivity of window and door sensors and automatically locking smart locks.

[0674] Output: The security measures taken.

[0675] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0676] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0677] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0678] [Second embodiment]

[0679] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0680] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0681] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0682] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0683] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0684] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0685] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0686] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0687] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0688] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0689] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0690] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0691] This invention relates to a game system that utilizes generative AI to provide endless fun. The system includes multiple algorithms and modules for generating character images, generating movements, synthesizing voices, generating dialogue and stories, detecting and debugging bugs, and optimizing game difficulty and reward settings.

[0692] First, the server receives a request from the user and generates a character image. This uses an image generation algorithm based on attribute information. The user selects the character's appearance and characteristics on the game's start screen, and the device sends this selection to the server. The server creates a character image using an image generation algorithm such as StableDiffusion based on the received attribute information, and sends the generated image back to the device. The device displays this image to the user.

[0693] The server then generates character movement based on the generated character image. This is done using algorithms such as MotionDiffuse. The device uploads the character image to the server, which analyzes the image and generates movement data. The generated movement data is sent to the device, which applies it to the character and displays it.

[0694] Furthermore, for voice synthesis, the server generates voice based on the dialogue data. Dialogue data is generated from the scenario or story, and this is input into a voice synthesis algorithm using LLM. The generated voice data is sent to the device, which then plays it in the game.

[0695] Conversations and stories are generated by the server using an NLP model. When a user interacts with an NPC character in the game, the device sends that context data to the server. The server uses this context data to generate conversations between the user and the NPC and the ongoing story, and sends them to the device. The device then displays the generated data to the user.

[0696] Bug detection and debug instructions are carried out by a bug detection AI on the server. When a user discovers a bug during the game, they report the problem to the server via their device. The server analyzes the report and uses AI to detect bugs in the code. Debug instructions for the detected bug are generated and sent to the development team. The problem is resolved by releasing a patch to fix the problem.

[0697] Finally, the server also optimizes the game's difficulty and reward settings. The device periodically sends the player's progress and gameplay data to the server. The server analyzes this data and sets the optimal balance between difficulty and reward. The new setting data is sent back to the device, which applies it to the game and provides it to the player.

[0698] Through the above process, this invention enables efficient and creative game development, reducing development efforts while providing players with a consistently new and engaging gaming experience.

[0699] The processing flow will be explained below.

[0700] Character generation process

[0701] Step 1:

[0702] The user selects the character's attributes (e.g., hair color, clothing, gender) on the game's start screen.

[0703] Step 2:

[0704] The terminal transmits the selected attribute data to the server.

[0705] Step 3:

[0706] Based on the received attribute data, the server invokes an image generation algorithm such as StableDiffusion to generate a character image.

[0707] Step 4:

[0708] The server transmits the generated character image to the terminal.

[0709] Step 5:

[0710] The terminal displays the received character image to the user.

[0711] Motion generation processing

[0712] Step 1:

[0713] The terminal uploads the generated character image to the server.

[0714] Step 2:

[0715] Based on the received character image, the server invokes algorithms such as MotionDiffuse to generate character movement data.

[0716] Step 3:

[0717] The server transmits the generated motion data to the terminal.

[0718] Step 4:

[0719] The device stores the received movement data locally and applies it to the character for display.

[0720] Speech synthesis processing

[0721] Step 1:

[0722] The server generates dialogue data based on a scenario or story.

[0723] Step 2:

[0724] The server passes the generated dialogue data to a speech synthesis algorithm to generate voice data.

[0725] Step 3:

[0726] The server transmits the generated voice data to the terminal.

[0727] Step 4:

[0728] The device stores the received audio data locally and plays it in the game.

[0729] Conversation / story generation processing

[0730] Step 1:

[0731] The user talks to an NPC character in the game.

[0732] Step 2:

[0733] The device transmits user input and the current game state to the server.

[0734] Step 3:

[0735] Based on the received context data, the server invokes NLP models such as ChatGPT to generate conversations and stories.

[0736] Step 4:

[0737] The server sends the generated conversations and stories to the device.

[0738] Step 5:

[0739] The terminal stores the received conversation data locally and displays it to the user.

[0740] Bug detection and debug instruction handling

[0741] Step 1:

[0742] A user discovers a bug in the game and reports the problem through the "Report a Bug" form.

[0743] Step 2:

[0744] The terminal sends the report to the server.

[0745] Step 3:

[0746] The server analyzes the received report and calls a bug detection AI to detect bugs in the code.

[0747] Step 4:

[0748] The server generates debugging instructions based on the detected bug information and sends them to the development team.

[0749] Step 5:

[0750] The server applies the corrected code or patch to the game and notifies the device of the update.

[0751] Difficulty and reward setting optimization process

[0752] Step 1:

[0753] The device transmits the player's progress and gameplay data to the server.

[0754] Step 2:

[0755] Based on the received player data, the server invokes an algorithm to optimize difficulty and reward settings and analyzes the data.

[0756] Step 3:

[0757] Based on the analysis results, the server generates new difficulty and reward settings and sends them to the device.

[0758] Step 4:

[0759] The device applies the received new setting data to the game and reflects it to the player.

[0760] The above is a specific program processing flow for the embodiment of the invention. This system enables efficient and creative game development, and constantly provides new content to players.

[0761] Example 1

[0762] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0763] In modern games, it is extremely important to be able to highly customize character appearance, movement, voice, dialogue, and story, as well as efficiently detect and fix bugs and adjust difficulty and rewards. However, there is still a lack of a system that can integrate these diverse elements and provide them to users in real time. Furthermore, there is also a lack of a means to automatically and effectively generate and optimize these elements. Therefore, improving the efficiency of game development while providing players with a fresh and engaging experience is a challenge.

[0764] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0765] In this invention, the server includes: [means for generating a character image using digital image generation means based on property information upon receiving a request from a user; [means for generating movement data from the digital image based on the generated character image;] [means for synthesizing voice using a natural language processing model based on lines generated from a scenario;] [means for generating dialogue and a story using a natural language processing model based on user input;] [means including artificial intelligence for detecting system malfunctions and generating correction instructions based thereon; and [means for analyzing user progress and gameplay data to optimize game difficulty and reward settings.] This makes it possible [to generate and optimize a variety of game elements in real time and provide players with an always fresh and engaging game experience].

[0766] "User" refers to the person who operates the system and plays the game.

[0767] "Attribute information" refers to attribute data necessary for generating images and movements, such as a character's appearance and characteristics.

[0768] "Digital image generation means" refers to algorithms or software for generating a digital image based on property information.

[0769] "Character image" refers to digital image data that represents the appearance of a generated character.

[0770] "Movement data" refers to digital data generated to represent the movements of a character.

[0771] "Scenario" refers to text data that includes instructions such as the story and dialogue used in the game.

[0772] "Natural language processing model" refers to a machine learning model used to understand and generate human language.

[0773] "Speech synthesis" refers to the technology of generating voice data from text data.

[0774] "Dialogue and Narrative" refers to the streams of text and dialogue that contain interactions between the user and in-game characters and NPCs.

[0775] "System malfunction" refers to an abnormality or error that occurs in software or hardware.

[0776] "Artificial intelligence" refers to technology that analyzes data, recognizes patterns, and solves problems through learning and inference.

[0777] "Progression" refers to a player's achievements or progress within a game.

[0778] "Gameplay data" refers to information such as player behavior, scores, and play history.

[0779] "Difficulty and reward settings" refers to the balance between the level of challenge in the game and the rewards obtained in return.

[0780] This invention relates to a game system that utilizes generative AI to provide endless fun. The system includes multiple algorithms and modules for generating character images, generating movements, synthesizing voices, generating dialogue and stories, detecting and debugging bugs, and optimizing game difficulty and reward settings.

[0781] Character image generation

[0782] First, the user selects the character's appearance and characteristics on the game's start screen. The specific steps are explained below.

[0783] The user selects the character's appearance and characteristics.

[0784] Example: User selects "Character with blue hair."

[0785] The terminal transmits the user's selection information to the server.

[0786] Example prompt: "Submit blue hair info for character generation."

[0787] The server generates a character image based on the received property information using a digital image generation means such as StableDiffusion.

[0788] The generated character image is sent back to the terminal, which displays this image to the user.

[0789] Example: A blue-haired character appears on the device screen.

[0790] Generating character movements

[0791] Next, a procedure for generating character movements based on the generated character image will be described.

[0792] The terminal uploads the generated character image to the server.

[0793] Example prompt: "Send an image of a blue-haired character to the server."

[0794] The server generates the motion data using an algorithm such as MotionDiffuse.

[0795] Example: The server generates walking motion data for a blue-haired character.

[0796] The generated movement data is sent to the terminal, which applies it to the character and displays it.

[0797] An animation of a blue-haired character walking appears on the device screen.

[0798] Speech synthesis

[0799] The procedure for synthesizing voice based on character lines is explained below.

[0800] The server generates dialogue data from a scenario or story.

[0801] Example: Generate dialogue data for "Hello"

[0802] The server generates the voice using a speech synthesis algorithm that uses LLM.

[0803] Example prompt: "Generate the greeting 'hello' aloud."

[0804] The generated audio data is sent to the terminal, which then plays it back in the game.

[0805] Example: A character in a game says "Hello."

[0806] Conversation and story generation

[0807] The steps for generating conversations between users and NPCs and game stories are explained below.

[0808] The user provides input to an NPC in the game.

[0809] Example: A user asks, "What's the next mission?"

[0810] The terminal transmits the context data to the server.

[0811] Example prompt: Send the question "What's your next mission?"

[0812] The server uses NLP models to generate dialogue and stories.

[0813] Example: Generate a description for the next mission

[0814] The generated data is sent to the terminal, which displays it to the user.

[0815] Example: An NPC in a game explains the details of the next mission.

[0816] Bug detection and debugging instructions

[0817] The procedure for detecting system malfunctions and generating debug instructions is described below.

[0818] When a user discovers a bug during the game, they report the problem to the server via their device.

[0819] Example prompt: "The character walks through the wall."

[0820] The server analyzes the reports and uses AI to detect bugs.

[0821] Example: Server detects bug that allows walking through walls

[0822] The server generates debug instructions and sends them to the development team.

[0823] Example: Generate specific debug instructions for fixing a bug

[0824] The development team makes corrections based on the debugging instructions and releases the corrected version.

[0825] Example: The modified game is provided to the user

[0826] Optimizing game difficulty and reward settings

[0827] The procedure for optimizing the game difficulty and reward settings is described below.

[0828] The device periodically transmits the player's progress and gameplay data to the server.

[0829] Example prompt: "Send data such as game completion time and success rate."

[0830] The server analyzes the received data and optimizes the balance between difficulty and reward.

[0831] Example: Adjusting difficulty based on player performance

[0832] The server sends the new difficulty and reward setting data back to the device.

[0833] Example: Sending optimized configuration data to the device

[0834] The device will apply the new configuration data to the game and reflect it to the player.

[0835] Example: The game restarts with a new difficulty level

[0836] By following the above procedure, the system of the present invention can generate and optimize a variety of game elements in real time, and provide users with a constantly fresh and engaging game experience.

[0837] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0838] Step 1:

[0839] The user selects the character's appearance and characteristics.

[0840] Input: User-selected character appearance and characteristics (e.g., a character with blue hair).

[0841] Specific operation: The user selects the character's attributes on the game's start screen.

[0842] Output: The selected character attribute information is generated on the terminal.

[0843] Step 2:

[0844] The terminal transmits the user's selection information to the server.

[0845] Input: Character selection information (attribute data for a blue-haired character).

[0846] Specific operation: The terminal transmits the selection information to the server via the network.

[0847] Output: The selection information sent to the server.

[0848] Step 3:

[0849] The server generates a character image based on the received property information using a digital image generation algorithm.

[0850] Input: Character attribute data (blue hair information).

[0851] Data processing or data calculation: Using a digital image generation algorithm such as StableDiffusion, a character image is generated based on the specified attribute information.

[0852] Specific operation: The server runs StableDiffusion and creates a character image.

[0853] Output: The generated character image.

[0854] Step 4:

[0855] The server transmits the generated character image to the terminal.

[0856] Input: Generated character image.

[0857] Specific operation: The server sends the generated image back to the terminal via the network.

[0858] Output: Character image sent to the device.

[0859] Step 5:

[0860] The terminal displays the received character image to the user.

[0861] Input: Character image sent from the server.

[0862] Specific operation: Display a character image on the device screen.

[0863] Output: The character image displayed to the user.

[0864] Step 6:

[0865] The terminal uploads the generated character image to the server.

[0866] Input: Character image stored on the device.

[0867] Specific operation: The device uploads the character image to the server via the network.

[0868] Output: Character image uploaded to the server.

[0869] Step 7:

[0870] The server generates movement data based on the character image.

[0871] Input: Uploaded character image.

[0872] Data processing or data calculation: Using algorithms such as MotionDiffuse to analyze character images and generate motion data.

[0873] Specific operation: The server executes MotionDiffuse and creates character movement data.

[0874] Output: Generated character motion data.

[0875] Step 8:

[0876] The server transmits the generated exercise data to the terminal.

[0877] Input: Generated movement data.

[0878] Specific operation: The server sends the exercise data to the terminal via the network.

[0879] Output: Exercise data sent to the device.

[0880] Step 9:

[0881] The device applies the received movement data to the character and displays it.

[0882] Input: Exercise data sent from the server.

[0883] Specific operation: The device applies the movement data to the character and displays it on the screen.

[0884] Output: A moving character displayed on the terminal.

[0885] Step 10:

[0886] The server synthesizes voice based on the dialogue data generated from the scenario.

[0887] Input: Scenario and dialogue data.

[0888] Data processing or data computation: Generate speech data from dialogue data using a natural language processing model.

[0889] Specific operation: The server executes a speech synthesis algorithm to convert the dialogue data into voice data.

[0890] Output: The generated audio data.

[0891] Step 11:

[0892] The server transmits the generated voice data to the terminal.

[0893] Input: The generated audio data.

[0894] Specific operation: The server sends the audio data to the terminal via the network.

[0895] Output: The audio data sent to the device.

[0896] Step 12:

[0897] The device then plays the received audio data as the character's lines.

[0898] Input: Audio data sent from the server.

[0899] Specific operation: The device applies the voice data to the character and plays it in the game.

[0900] Output: Character dialogue played in-game.

[0901] Step 13:

[0902] The context data entered by the user is transmitted from the terminal to the server.

[0903] Input: A question or request that the user types in the game.

[0904] Specific operation: The terminal sends user input to the server.

[0905] Output: The context data sent to the server.

[0906] Step 14:

[0907] The server generates conversations and stories based on the contextual data.

[0908] Input: Context data.

[0909] Data manipulation or data computation: Using natural language processing models to generate conversations or stories.

[0910] What it does: The server runs NLP models and creates conversations and stories.

[0911] Output: Generated conversation or story data.

[0912] Step 15:

[0913] The server sends the generated conversation and story data to the terminal.

[0914] Input: Generated conversation or story data.

[0915] Specific operation: The server sends data to the terminal via the network.

[0916] Output: Conversation or story data sent to your device.

[0917] Step 16:

[0918] The terminal displays the received conversation and story data to the user.

[0919] Input: Conversation or story data sent from the server.

[0920] Specific behavior: The device displays the data on the screen.

[0921] Output: The conversation or story that is displayed to the user.

[0922] Step 17:

[0923] If a user discovers a bug during the game, they report the problem to the server via their device.

[0924] Input: The description of the bug found by the user.

[0925] Specific operation: A user reports a bug to the server through a terminal.

[0926] Output: The bug report sent to the server.

[0927] Step 18:

[0928] The server analyzes the reported bug and uses AI to detect it.

[0929] Input: Bug report data.

[0930] Data manipulation or data manipulation: Using bug detection AI to analyze reports and detect bugs.

[0931] Specific operation: The server runs AI to identify the location and cause of the bug.

[0932] Output: Detected bug data.

[0933] Step 19:

[0934] The server generates debug instructions for the detected bugs and sends them to the development team.

[0935] Input: Detected bug data.

[0936] What happens: The server generates debug instructions and sends them to the development team.

[0937] Output: Debugging instructions sent to the development team.

[0938] Step 20:

[0939] The device periodically transmits the player's progress and gameplay data to the server.

[0940] Input: Player progress data, gameplay data.

[0941] Specific operation: The device sends progress and play data to the server.

[0942] Output: Progress data and play data sent to the server.

[0943] Step 21:

[0944] The server analyzes the received data and optimizes the balance between difficulty and reward.

[0945] Input: Progress data, play data.

[0946] Data processing or data arithmetic: Using data analysis algorithms to calculate the optimal balance between difficulty and reward.

[0947] Specific operation: The server runs the algorithm and adjusts the difficulty and reward.

[0948] Output: New difficulty and reward settings data.

[0949] Step 22:

[0950] The server sends the new difficulty and reward setting data back to the device.

[0951] Input: New difficulty and reward settings data.

[0952] Specific operation: The server sends the setting data to the terminal.

[0953] Output: The new configuration data sent to the terminal.

[0954] Step 23:

[0955] The device will apply the new configuration data to the game and reflect it to the player.

[0956] Input: New difficulty and reward settings data.

[0957] Specific operation: The device applies and reflects the new configuration data in the game.

[0958] Output: The game environment with the new settings.

[0959] (Application example 1)

[0960] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0961] It is necessary to make the virtual shopping experience more engaging and interactive, provide an entertainment-rich shopping experience that users can enjoy endlessly, and increase purchasing motivation by providing product recommendations and navigation in real time based on users' preferences.

[0962] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0963] In this invention, the server includes: [means for generating character images;] [means for generating movements based on the generated character images;] [means for synthesizing voice based on lines;] [means for generating conversations and stories;] [means for detecting bugs and generating debugging instructions;] [means for optimizing game difficulty and reward settings;] [means for recommending products based on user attribute information and displaying them as navigation characters;] [voice synthesis means for explaining products using the voice lines of the generated characters; and [means for generating conversations and dialogues based on interactions with the user. This makes it possible to provide users with an endlessly enjoyable virtual shopping experience and increase their desire to purchase.

[0964] A "character image" is a visual representation of a virtual person or object generated based on the user's attribute information.

[0965] "Movement data" is data on the actions and movements that a character appears to perform based on the generated character image.

[0966] "Speech synthesis" is the process of creating artificially generated speech based on text data.

[0967] "Conversations and stories" refer to dialogues and scenarios created based on user interaction.

[0968] "Bug detection" is the process of discovering errors or defects in software or systems.

[0969] "Debugging instructions" are specific steps or instructions for fixing a detected bug.

[0970] "Optimizing difficulty and reward settings" is the process of adjusting the appropriate challenge and reward levels depending on the progress of a game or experience.

[0971] "User attribute information" is data that includes user preferences, behavioral history, personal settings, etc.

[0972] "Product recommendation" is the process of suggesting highly relevant products based on a user's attribute information.

[0973] A "navigation character" is a virtual character that acts as a guide for users within a virtual store.

[0974] An "interaction" is an interaction between a user and a system.

[0975] This invention relates to a system that provides users with an endlessly enjoyable virtual shopping experience. The system uses a generative AI model and multiple algorithms and modules to generate character images, generate movements, synthesize voices, and generate conversations and stories. It also includes a function that recommends products based on the user's attribute information and displays them as navigation characters.

[0976] The server generates a character image upon receiving a request from the user. This uses StableDiffusion, an image generation algorithm based on the user's attribute information. The user selects the character's appearance and characteristics on their smartphone and sends their selection to the server. The server creates a character image based on the received attribute information and sends the generated image back to the device. The device displays this image to the user.

[0977] The server then uses algorithms such as MotionDiffuse to generate character movement based on the generated character image. The device uploads the character image to the server, which analyzes the image and generates motion data. The generated motion data is then sent to the device, which applies it to the character and displays it.

[0978] The server then generates voice based on the dialogue data. Dialogue data is generated from the scenario or story, and this is input into a speech synthesis algorithm using a large-scale language model (LLM). The generated voice data is sent to the device, which then plays it back to the user.

[0979] The server generates conversations and stories using NLP models. When the user interacts with the navigation character, the device sends the context data to the server. The server generates conversations between the user and the character and the ongoing story based on this context data and sends it to the device. The device then displays the generated data to the user.

[0980] Bug detection and debug instructions are performed by a bug detection AI on the server. When a user finds a bug in the system, they report the problem to the server via their terminal. The server analyzes the report and uses AI to detect bugs in the code. Debug instructions for the detected bug are generated and sent to the development team. The problem is resolved by releasing a patch to fix it.

[0981] Finally, the system periodically transmits user progress and interaction data to the server, which analyzes this data and configures the system to optimize the balance between difficulty and reward. The new configuration data is sent back to the device, which then applies it to the system and reflects it to the user.

[0982] For example, the following conversation might be generated by a generative AI model:

[0983] User: "What are your recent recommendations?"

[0984] AI: "My latest recommendation is a newly released smartwatch. It features a long battery life and comprehensive health tracking."

[0985] In this way, the system can provide users with an endlessly enjoyable virtual shopping experience and increase their desire to purchase.

[0986] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0987] Step 1:

[0988] The user selects the character's appearance and characteristics using a smartphone device.

[0989] Input: User selection information (attribute information)

[0990] Output: Sends user selection information to the server

[0991] Specific operation: The user selects the character's appearance and characteristics on the application interface and sends that information to the server.

[0992] Step 2:

[0993] The server generates a character image using StableDiffusion based on the attribute information received from the user.

[0994] Input: User attribute information

[0995] Output: Generated character image

[0996] Specific operation: The server uses the StableDiffusion algorithm to generate a character image from the user's attribute information.

[0997] Step 3:

[0998] The server returns the generated character image to the terminal.

[0999] Input: Generated character image

[1000] Output: Character image returned to the device

[1001] Specific operation: The server sends the generated character image to the terminal, and the terminal displays this image.

[1002] Step 4:

[1003] The device uploads the character image to the server, and the server generates the motion data using MotionDiffuse.

[1004] Input: Character image

[1005] Output: Generated motion data

[1006] Specific operation: The device uploads the character image to the server, and the server generates motion data using the MotionDiffuse algorithm.

[1007] Step 5:

[1008] The server transmits the generated motion data to the terminal.

[1009] Input: Generated motion data

[1010] Output: Movement data sent to the device

[1011] Specific operation: The server sends the generated movement data to the device, which then applies it to the character and displays it.

[1012] Step 6:

[1013] The server synthesizes voice based on the scenario or story.

[1014] Input: Scenario and story dialogue data

[1015] Output: Generated audio data

[1016] Specific operation: The server generates speech from dialogue data using a speech synthesis algorithm based on a large-scale language model (LLM) and sends it to the terminal.

[1017] Step 7:

[1018] The terminal reproduces the generated voice data and lets the user hear it.

[1019] Input: Generated audio data

[1020] Output: The audio played to the user

[1021] Specific operation: The device plays the audio data received from the server.

[1022] Step 8:

[1023] The user interacts with the navigation character and the terminal transmits the context data to the server.

[1024] Input: User interaction

[1025] Output: Context data sent to the server

[1026] Specific operation: The user interacts with the navigation character, and the content of the interaction is sent from the terminal to the server as context data.

[1027] Step 9:

[1028] The server uses NLP models to generate conversations and stories and send them to the device.

[1029] Input: Context data

[1030] Output: Generated conversations and story data

[1031] Specific operation: The server analyzes the context data using an NLP model, generates a conversation or story, and sends it to the device.

[1032] Step 10:

[1033] The terminal displays the generated conversations and stories to the user.

[1034] Input: Generated conversation or story data

[1035] Output: The conversation or story that is displayed to the user

[1036] Specific operation: The device displays the conversation and story data received from the server and shows it to the user.

[1037] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1038] This invention relates to a system that utilizes generative AI and an emotion engine to provide users with an endlessly enjoyable gaming experience. This system includes multiple algorithms and modules that generate character images and create movements, voices, dialogue, and stories. It also incorporates an emotion engine that recognizes the user's emotions and dynamically changes the game content.

[1039] First, the system generates a character image based on the attributes of the character selected by the user. The user selects the character's appearance and characteristics on the game's opening screen, and the device sends that selection to the server. The server uses an image generation algorithm (e.g., StableDiffusion) to create a character image based on the received attribute information, and sends the generated image back to the device. The device then displays this image to the user.

[1040] Next, the server generates character movement based on the generated character image. The device uploads the character image to the server, and the server analyzes the image and invokes an algorithm (such as MotionDiffuse) to generate motion data. The generated motion data is sent to the device, which applies it to the character and displays it.

[1041] For voice synthesis, the server generates voice based on dialogue data. Dialogue data is generated from the scenario or story, and this is input into the voice synthesis algorithm. The generated voice data is sent to the device, which plays it in the game.

[1042] Conversations and stories are generated by an NLP model (e.g., ChatGPT) on the server. When a user talks to an NPC character in the game, the device sends context data to the server. The server generates conversations and stories based on this context data and sends them to the device. The device then displays the generated data to the user.

[1043] The emotion engine has the ability to recognize emotions by analyzing the user's facial expression and voice data. The device collects data provided by the user through the camera and microphone and sends it to the server. The server inputs the received data into the emotion engine and recognizes the user's emotional state. The game content is dynamically changed based on the emotion recognition results. For example, if the user is surprised, an enemy character in the game can suddenly appear.

[1044] Bug detection and debug instructions are carried out by a bug detection AI on the server. When a user discovers a bug during the game, they report the problem to the server via their device. The server analyzes the report and uses AI to detect bugs in the code. Debug instructions for the detected bug are generated and sent to the development team. The problem is resolved by releasing a patch to fix the problem.

[1045] Finally, the server also optimizes the game's difficulty and reward settings. The device periodically sends the player's progress and gameplay data to the server. The server analyzes this data and sets the optimal balance between difficulty and reward. The new setting data is sent back to the device, which applies it to the game and provides it to the player.

[1046] For example, if a user selects red hair and armor as their costume during character creation, the server generates an image of the character wearing red hair and armor based on those attributes. The server then generates movements and sounds for the character, and the emotion engine recognizes the user's emotions and dynamically changes in-game events. For example, if the user looks tired, an event will occur in which a friendly NPC in the game offers a support item.

[1047] Through the above process, this invention enables efficient and creative game development, reducing development effort while providing players with a consistently new and engaging gaming experience. Furthermore, incorporating the user's emotional state can provide a more personalized gaming experience.

[1048] The processing flow will be explained below.

[1049] Character generation process

[1050] Step 1:

[1051] The user selects the character's attributes (e.g., hair color, clothing, gender) on the game's start screen.

[1052] Step 2:

[1053] The terminal transmits the selected attribute data to the server.

[1054] Step 3:

[1055] Based on the received attribute data, the server invokes an image generation algorithm (e.g., StableDiffusion) to generate a character image.

[1056] Step 4:

[1057] The server transmits the generated character image to the terminal.

[1058] Step 5:

[1059] The terminal displays the received character image to the user.

[1060] Motion generation processing

[1061] Step 1:

[1062] The terminal uploads the generated character image to the server.

[1063] Step 2:

[1064] The server calls an algorithm (e.g., MotionDiffuse) that generates motion data based on the received character image.

[1065] Step 3:

[1066] The server transmits the generated motion data to the terminal.

[1067] Step 4:

[1068] The terminal applies the received motion data to the character and displays the motion to the user.

[1069] Speech synthesis processing

[1070] Step 1:

[1071] The server generates dialogue data based on a scenario or story.

[1072] Step 2:

[1073] The server passes the dialogue data to a speech synthesis algorithm to generate speech data.

[1074] Step 3:

[1075] The server transmits the generated voice data to the terminal.

[1076] Step 4:

[1077] The device plays the received audio data in the game.

[1078] Conversation / story generation processing

[1079] Step 1:

[1080] The user talks to an NPC character in the game.

[1081] Step 2:

[1082] The device transmits user input and the current game state to the server.

[1083] Step 3:

[1084] The server invokes an NLP model (e.g., ChatGPT) based on the received context data to generate a conversation or story.

[1085] Step 4:

[1086] The server sends the generated conversations and stories to the device.

[1087] Step 5:

[1088] The terminal displays the received conversation data to the user.

[1089] Emotion Recognition Processing

[1090] Step 1:

[1091] The user provides facial expression data and voice data through a camera and microphone while playing the game.

[1092] Step 2:

[1093] The terminal transmits the collected facial expression data and voice data to the server.

[1094] Step 3:

[1095] The server inputs the received data into an emotion engine to recognize the user's emotion.

[1096] Step 4:

[1097] The server dynamically adjusts in-game events and conversations based on the emotion recognition results.

[1098] Step 5:

[1099] The device displays the adjusted events and conversations in the game and reflects them to the user.

[1100] Bug detection and debug instruction handling

[1101] Step 1:

[1102] A user discovers a bug in the game and reports the problem through a bug report form.

[1103] Step 2:

[1104] The terminal sends the report to the server.

[1105] Step 3:

[1106] The server analyzes the received report and calls a bug detection AI to detect bugs in the code.

[1107] Step 4:

[1108] The server generates debug instructions for the detected bugs and sends them to the development team.

[1109] Step 5:

[1110] The server applies the corrected code or patch to the game and notifies the device of the update.

[1111] Difficulty and reward setting optimization process

[1112] Step 1:

[1113] The device transmits the player's progress and gameplay data to the server.

[1114] Step 2:

[1115] Based on the received player data, the server invokes an algorithm to optimize difficulty and reward settings and analyzes the data.

[1116] Step 3:

[1117] Based on the analysis results, the server generates new difficulty and reward settings and sends them to the device.

[1118] Step 4:

[1119] The device applies the received new setting data to the game and reflects it to the player.

[1120] The above is a specific program processing flow for implementing the invention in combination with an emotion engine. This system provides a personalized gaming experience that incorporates the user's emotions.

[1121] Example 2

[1122] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1123] In conventional game systems, it has been difficult to dynamically adapt game progression and generate interactive characters that reflect the user's individual emotional state. Furthermore, bug detection and game difficulty adjustment are performed manually, resulting in low efficiency and limited improvements to the user experience. New algorithms and technologies are needed to address these challenges.

[1124] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1125] In this invention, the server includes means for generating character images, means for generating movements based on the generated character images, means for synthesizing voice based on dialogue, means for generating conversations and stories, means for recognizing the user's emotional state and dynamically changing game content, means for detecting bugs and generating debugging instructions, and means for optimizing game difficulty and reward settings. This enables a dynamic and personalized game experience that matches the user's emotions, and makes it possible to provide a high-quality game service by automatically detecting bugs and automatically adjusting game balance.

[1126] "Character image" is graphic data of a visual character used in the game, generated based on attribute information selected by the user.

[1127] "Movement data" is a series of data generated based on a character image to represent the movement of the character.

[1128] "Dialogue data" is text data of words spoken by characters that is generated based on the scenario or story within the game.

[1129] A "voice synthesis algorithm" is a calculation procedure or program for generating voice data based on dialogue data.

[1130] A "conversation generation algorithm" is a computational procedure or program for generating dialogue and stories between characters in a game.

[1131] An "emotion engine" is a group of algorithms and programs that analyze a user's facial expression data and voice data to recognize their emotional state.

[1132] "Bug detection AI" is an artificial intelligence that automatically detects glitches (bugs) that occur in games and generates instructions for debugging.

[1133] "Difficulty and reward optimization" is the process of adjusting the balance of a game's difficulty and available rewards based on player progress and gameplay data.

[1134] An "image generation algorithm" is a calculation procedure or program for generating a new character image based on attribute information selected by the user.

[1135] "Dynamic change" refers to changing the content and progress of the game in real time according to the user's emotional state and the game situation.

[1136] MODE FOR CARRYING OUT THE INVENTION

[1137] This invention is a system that utilizes generative AI and an emotion engine to provide users with an endlessly enjoyable gaming experience. To implement the invention, specific hardware and software are required to process and calculate various data.

[1138] Hardware and software used

[1139] This system operates in a network environment that includes user devices (PCs, smartphones, etc.) and servers. Specific software includes image generation algorithms (e.g., StableDiffusion), motion generation algorithms (e.g., MotionDiffuse), speech synthesis algorithms, natural language processing models (e.g., ChatGPT), and emotion engines.

[1140] System Operation Overview

[1141] 1. Character image generation

[1142] The user selects the character's appearance and characteristics on the game start screen. The device sends the selection to the server, which uses an image generation algorithm to generate a character image based on the received attribute information. This image is sent back to the device and displayed.

[1143] 2. Character Movement Generation

[1144] The device sends the generated character image to the server, which uses a motion generation algorithm to generate character movements from the image, and the generated motion data is sent back to the device and applied to the character.

[1145] 3. Speech generation

[1146] Based on the dialogue data generated according to the scenario and story, the server uses a speech synthesis algorithm to generate voice data, which is then sent to the device and played back in the game.

[1147] 4. Conversation and story generation

[1148] When a user talks to an NPC character in a game, the device sends that context data to the server, which uses a natural language processing model to generate a dialogue or story, which is then sent to the device and displayed to the user.

[1149] 5. Emotion Recognition and Dynamic Change of Game Content

[1150] The device collects the user's facial expression and voice data through a camera and microphone and sends it to the server. The server then uses an emotion engine to recognize the user's emotional state. Based on this recognition, in-game events and character behavior are dynamically changed.

[1151] 6. Bug Detection and Debugging Instructions

[1152] If a user discovers a bug during the game, they report the problem to the server via their device. The server uses a bug detection AI to analyze the report and detect bugs in the code. Debug instructions are generated and sent to the development team.

[1153] 7. Optimizing difficulty and reward settings

[1154] The device periodically sends the player's progress and gameplay data to the server. The server analyzes this data and configures the game to optimize the balance between difficulty and rewards. The new configuration data is sent back to the device and applied to the game.

[1155] Specific examples

[1156] For example, if a user selects red hair and armor as their outfit, the server generates a character image wearing red hair and armor based on those attributes. The server then generates movements and sounds for the character, and the emotion engine recognizes the user's emotions and dynamically changes in-game events. For example, if the user shows a tired expression, an event occurs in which a friendly character in the game provides a support item.

[1157] Prompt Sentence Examples

[1158] "Generate an image of a character with red hair and armor."

[1159] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1160] Step 1: User character attribute selection

[1161] Description: The user selects the character's appearance and characteristics (hair color, outfit, etc.) on the game's start screen.

[1162] Input: User-selected character attribute information (e.g., hair color "red", costume "armor").

[1163] Output: Data that sends selected attribute information from the device to the server.

[1164] Specific operation: The user selects "red hair" and "armor," and the device sends that information to the server.

[1165] Step 2: Generate character images

[1166] Description: The server generates a character image using an image generation algorithm based on the received attribute information.

[1167] Input: Character attribute information selected by the user.

[1168] Output: Send the generated character image data to the terminal.

[1169] What it does: The server uses the StableDiffusion algorithm to generate an image of a character with red hair and armor, and sends it back to the device.

[1170] Step 3: Character Movement Generation

[1171] Description: The device uploads the generated character image to the server, and the server generates the character's movements using a movement generation algorithm.

[1172] Input: Generated character image.

[1173] Output: Send the generated motion data to the device.

[1174] How it works: The server uses the MotionDiffuse algorithm to generate character motion data and sends it to the device, which then applies this motion to the character.

[1175] Step 4: Generate audio

[1176] Description: The server generates dialogue data from a scenario or story and generates speech using a speech synthesis algorithm.

[1177] Input: Dialogue data (text format).

[1178] Output: Sends the generated audio data to the device.

[1179] What happens: The server uses a speech synthesis algorithm to generate a voice for the phrase "Hello" and sends it to the device, which then plays it back in the game.

[1180] Step 5: Generate conversations and stories

[1181] Description: When a user talks to an NPC character in a game, the device sends that context data to the server, which then generates the dialogue and story.

[1182] Input: Context data (user-NPC interactions, current scenario, etc.).

[1183] Output: Send the generated conversation data and story data to the device.

[1184] How it works: The server uses the ChatGPT algorithm to generate a conversation saying, "Go down this path and you'll find the treasure," and sends it to the device, which then displays it to the NPC in the game.

[1185] Step 6: Emotion recognition and dynamic change of game content

[1186] Description: The device collects the user's facial expression and voice data and sends it to the server. The server uses an emotion engine to recognize the user's emotional state and dynamically change the game content.

[1187] Input: facial expression data, audio data.

[1188] Output: Dynamically modified game content based on emotion recognition results.

[1189] Specific behavior: The server recognizes that the user is surprised and triggers an event in the game in which an enemy character suddenly appears.

[1190] Step 7: Bug detection and debugging instructions

[1191] Description: When a user discovers a bug during the game, they report the problem to the server through their terminal. The server uses a bug detection AI to detect bugs in the code and generate debug instructions.

[1192] Input: Bug report data.

[1193] Output: Debug instructions, fix patch.

[1194] What happens: The server analyzes the user's report that "the character can go through walls," detects the bug, sends a debug instruction to the development team, and releases a fix patch.

[1195] Step 8: Optimize difficulty and reward settings

[1196] Description: The device sends player progress and gameplay data to the server, which analyzes this data and configures the game to optimize the balance between difficulty and rewards.

[1197] Input: Player progress, gameplay data.

[1198] Output: Optimized difficulty and reward settings.

[1199] What it does: The server analyzes the player's high score, sets more difficult levels and rewards, and sends the new configuration data back to the device to apply it to the game.

[1200] (Application example 2)

[1201] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1202] Conventional security systems operate based on fixed settings, making it difficult to respond flexibly to user emotions and situations. Furthermore, there was no technology that could dynamically change security responses based on emotions in real time. As a result, even if a user is feeling anxious or nervous, it is not possible to provide appropriate safety measures, which could lead to a decline in the quality of security.

[1203] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1204] In this invention, the server includes: [means for generating character images;] [means for generating movements based on the generated character images;] [means for synthesizing voice based on dialogue;] [means for generating conversations and stories;] [means for detecting bugs and generating debugging instructions;] [means for optimizing game difficulty and reward settings;] [means for recognizing user emotions and dynamically changing system settings based on those emotions; and [means for dynamically generating emotion-based security measures using a generative AI model.] This enables appropriate security responses that are in line with user emotions to be provided in real time, improving the quality of security.

[1205] "Means for generating a character image" refers to an algorithm or module that creates an image of a character based on attribute information selected by a user.

[1206] "Means for generating movement based on the generated character image" refers to an algorithm or module that generates movement data from the generated character image.

[1207] "Means for synthesizing voice based on dialogue" refers to a voice synthesis algorithm or module that converts dialogue data from a scenario or story into voice.

[1208] "Means for generating conversations and stories" refers to natural language processing models and algorithms for generating conversations and stories with users.

[1209] "Means for detecting bugs and generating debug instructions" refers to algorithms or modules for analyzing problems reported by users and generating instructions required for debugging.

[1210] "Means for optimizing game difficulty and reward settings" refers to algorithms or modules that analyze users' gameplay data and optimally adjust the balance between difficulty and rewards.

[1211] "Means for recognizing user emotions and dynamically changing system settings based on those emotions" refers to algorithms or modules that analyze emotions from the user's facial expressions and voice and dynamically change system settings based on the results.

[1212] "Means for dynamically generating emotion-based security measures using a generative AI model" refers to an AI model that receives emotion data as input and generates security measures appropriate to those emotions.

[1213] This invention is a system that uses a generative AI model and an emotion engine to provide a dynamic gaming experience and security measures based on user emotions. The system includes multiple algorithms and modules and runs on devices such as smartphones and smart glasses.

[1214] The hardware required to implement the invention is a device equipped with a camera and microphone (e.g., a smartphone or smart glasses). The software uses Python and Django (server-side), an emotion API (e.g., Microsoft Azure Emotion API), and a natural language processing model (e.g., ChatGPT).

[1215] 1. Character image generation

[1216] The user uses the device to select the character's appearance and characteristics on the game's opening screen. This selection information is sent from the device to the server. The server generates a character image using an image generation algorithm (e.g., Stable Diffusion) based on the received attribute information, and sends the generated image back to the device. The device then displays this image to the user.

[1217] 2. Generating Character Movements

[1218] The generated character image is uploaded from the device to the server, which analyzes the image and invokes an algorithm (e.g., MotionDiffuse) to generate motion data. The generated motion data is sent to the device, which applies it to the character and displays it to the user.

[1219] 3. Speech Synthesis

[1220] The server generates dialogue data from the scenario and story, and synthesizes voice based on that. The dialogue data is input into a voice synthesis algorithm, and the generated voice data is sent to the device, which then plays it in the game.

[1221] 4. Conversation and story generation

[1222] When a user talks to an NPC character in a game, the device sends context data to the server, which generates a conversation or story based on the context data and sends it to the device, which then displays the generated data to the user.

[1223] 5. User Emotion Recognition and Dynamic System Changes

[1224] The device collects facial expression and voice data provided by the user through the camera and microphone and sends it to the server. The server inputs this data into an emotion engine (e.g., Microsoft Azure Emotion API) to recognize the user's emotional state. System settings are dynamically changed based on the emotion recognition results. For example, if the user feels anxious, security measures are strengthened.

[1225] 6. Emotion data collection using cameras and microphones

[1226] The camera and microphone are used to collect the user's facial expression and voice data. Specifically, the camera captures the user's facial expressions and the microphone records their voice. This data is sent to the emotion API in real time, and the user's emotional state is analyzed.

[1227] 7. Emotion-based security measures

[1228] Based on the emotion data, the server uses a generative AI model (e.g., ChatGPT) to dynamically generate security measures appropriate to the user's emotional state. For example, if the user feels anxious, the generative AI model will suggest security measures such as issuing a warning notification or automatically locking the smart lock.

[1229] Examples of examples and prompts:

[1230] If the user feels anxious at night, the data from the camera and microphone will be analyzed to generate the result "anxious." Based on this, the generative AI will increase the sensitivity of the window and door sensors and automatically generate instructions to lock all the doors in the house.

[1231] Text format

[1232] User emotion: anxious. Suggest measures to enhance home security during the user's anxious state.

[1233] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1234] Step 1:

[1235] The user selects the character's appearance and characteristics.

[1236] Input: Appearance and trait selection data.

[1237] How it works: The user selects the character's appearance and characteristics on the game's opening screen and enters them into the device.

[1238] Output: User-selected attribute information.

[1239] Step 2:

[1240] The terminal transmits the user-selected attribute information to the server.

[1241] Input: User-selected attribute information.

[1242] Operation: The terminal transmits the selected attribute information to the server.

[1243] Output: Attribute information received by the server.

[1244] Step 3:

[1245] The server generates a character image based on the attribute information.

[1246] Input: Attribute information.

[1247] Operation: The server inputs the attribute information into an image generation algorithm (e.g., StableDiffusion) to generate a character image.

[1248] Output: Character image.

[1249] Step 4:

[1250] The server returns the generated character image to the terminal.

[1251] Input: Generated character image.

[1252] Operation: The server sends the generated character image to the device.

[1253] Output: Character image received by the device.

[1254] Step 5:

[1255] The terminal displays the character image to the user.

[1256] Input: Received character image.

[1257] Operation: The device displays the character image on the screen.

[1258] Output: The character image displayed to the user.

[1259] Step 6:

[1260] Movement data is generated based on the character image.

[1261] Input: Character image.

[1262] How it works: The device uploads a character image to the server, which then invokes an algorithm (e.g., MotionDiffuse) that analyzes the image and generates motion data.

[1263] Output: Motion data.

[1264] Step 7:

[1265] The server transmits the generated motion data to the terminal.

[1266] Input: Generated motion data.

[1267] Operation: The server sends the generated motion data to the terminal.

[1268] Output: The motion data received by the device.

[1269] Step 8:

[1270] The device applies the movement data to the character and displays it.

[1271] Input: Received motion data.

[1272] Movement: The device applies the movement data to the character and displays it on the screen.

[1273] Output: A moving character displayed to the user.

[1274] Step 9:

[1275] The server generates dialogue data from the scenario or story and synthesizes the voice.

[1276] Input: A scenario or story.

[1277] How it works: The server generates dialogue data for a scenario or story, inputs it into a speech synthesis algorithm, and synthesizes the speech.

[1278] Output: Audio data.

[1279] Step 10:

[1280] The server transmits the generated voice data to the terminal.

[1281] Input: The generated audio data.

[1282] Operation: The server sends the generated voice data to the terminal.

[1283] Output: The audio data received by the device.

[1284] Step 11:

[1285] The device plays the audio data in the game.

[1286] Input: Received audio data.

[1287] Operation: The device plays the audio data in the game through the playback device.

[1288] Output: The audio played to the user.

[1289] Step 12:

[1290] The server generates conversations with NPC characters.

[1291] Input: A conversation request from the user.

[1292] How it works: When a user talks to an NPC character, the device sends context data to the server, which then uses a natural language processing model (e.g., ChatGPT) to generate a conversation or story.

[1293] Output: The generated conversation data.

[1294] Step 13:

[1295] The terminal displays the generated conversation data to the user.

[1296] Input: Generated conversation data.

[1297] Operation: The device displays the conversation data on the display.

[1298] Output: The conversation as displayed to the user.

[1299] Step 14:

[1300] The device collects the user's emotional data and sends it to the server.

[1301] Input: User's facial expression data, voice data.

[1302] How it works: It uses a camera and microphone to collect facial expression and voice data from the user and sends it to a server.

[1303] Output: Emotion data sent to the server.

[1304] Step 15:

[1305] The server analyzes the emotional data and recognizes the emotional state.

[1306] Input: Collected emotion data.

[1307] How it works: Emotion data is fed into an emotion engine (e.g., Microsoft Azure Emotion API) to analyze the emotional state.

[1308] Output: Parsed emotional state data.

[1309] Step 16:

[1310] The server generates security measures according to the emotional state.

[1311] Input: Emotional state data.

[1312] How it works: Based on the recognized emotional state data, a generative AI model (e.g., ChatGPT) is used to generate emotion-based security measures.

[1313] Output: The generated security measures.

[1314] Step 17:

[1315] The server transmits the generated security measures to the terminal.

[1316] Input: The generated security measures.

[1317] Operation: The server sends the generated security measures to the terminal.

[1318] Output: Security measures data received by the device.

[1319] Step 18:

[1320] The device implements security measures.

[1321] Input: Received security measures data.

[1322] Operation: The device will take necessary security measures based on the received security data, such as increasing the sensitivity of window and door sensors and automatically locking smart locks.

[1323] Output: The security measures taken.

[1324] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1325] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1326] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1327] [Third embodiment]

[1328] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1329] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1330] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1331] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1332] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1333] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1334] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1335] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1336] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1337] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1338] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1339] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1340] This invention relates to a game system that utilizes generative AI to provide endless fun. The system includes multiple algorithms and modules for generating character images, generating movements, synthesizing voices, generating dialogue and stories, detecting and debugging bugs, and optimizing game difficulty and reward settings.

[1341] First, the server receives a request from the user and generates a character image. This uses an image generation algorithm based on attribute information. The user selects the character's appearance and characteristics on the game's start screen, and the device sends this selection to the server. The server creates a character image using an image generation algorithm such as StableDiffusion based on the received attribute information, and sends the generated image back to the device. The device displays this image to the user.

[1342] The server then generates character movement based on the generated character image. This is done using algorithms such as MotionDiffuse. The device uploads the character image to the server, which analyzes the image and generates movement data. The generated movement data is sent to the device, which applies it to the character and displays it.

[1343] Furthermore, for voice synthesis, the server generates voice based on the dialogue data. Dialogue data is generated from the scenario or story, and this is input into a voice synthesis algorithm using LLM. The generated voice data is sent to the device, which then plays it in the game.

[1344] Conversations and stories are generated by the server using an NLP model. When a user interacts with an NPC character in the game, the device sends that context data to the server. The server uses this context data to generate conversations between the user and the NPC and the ongoing story, and sends them to the device. The device then displays the generated data to the user.

[1345] Bug detection and debug instructions are carried out by a bug detection AI on the server. When a user discovers a bug during the game, they report the problem to the server via their device. The server analyzes the report and uses AI to detect bugs in the code. Debug instructions for the detected bug are generated and sent to the development team. The problem is resolved by releasing a patch to fix the problem.

[1346] Finally, the server also optimizes the game's difficulty and reward settings. The device periodically sends the player's progress and gameplay data to the server. The server analyzes this data and sets the optimal balance between difficulty and reward. The new setting data is sent back to the device, which applies it to the game and provides it to the player.

[1347] Through the above process, this invention enables efficient and creative game development, reducing development efforts while providing players with a consistently new and engaging gaming experience.

[1348] The processing flow will be explained below.

[1349] Character generation process

[1350] Step 1:

[1351] The user selects the character's attributes (e.g., hair color, clothing, gender) on the game's start screen.

[1352] Step 2:

[1353] The terminal transmits the selected attribute data to the server.

[1354] Step 3:

[1355] Based on the received attribute data, the server invokes an image generation algorithm such as StableDiffusion to generate a character image.

[1356] Step 4:

[1357] The server transmits the generated character image to the terminal.

[1358] Step 5:

[1359] The terminal displays the received character image to the user.

[1360] Motion generation processing

[1361] Step 1:

[1362] The terminal uploads the generated character image to the server.

[1363] Step 2:

[1364] Based on the received character image, the server invokes algorithms such as MotionDiffuse to generate character movement data.

[1365] Step 3:

[1366] The server transmits the generated motion data to the terminal.

[1367] Step 4:

[1368] The device stores the received movement data locally and applies it to the character for display.

[1369] Speech synthesis processing

[1370] Step 1:

[1371] The server generates dialogue data based on a scenario or story.

[1372] Step 2:

[1373] The server passes the generated dialogue data to a speech synthesis algorithm to generate voice data.

[1374] Step 3:

[1375] The server transmits the generated voice data to the terminal.

[1376] Step 4:

[1377] The device stores the received audio data locally and plays it in the game.

[1378] Conversation / story generation processing

[1379] Step 1:

[1380] The user talks to an NPC character in the game.

[1381] Step 2:

[1382] The device transmits user input and the current game state to the server.

[1383] Step 3:

[1384] Based on the received context data, the server invokes NLP models such as ChatGPT to generate conversations and stories.

[1385] Step 4:

[1386] The server sends the generated conversations and stories to the device.

[1387] Step 5:

[1388] The terminal stores the received conversation data locally and displays it to the user.

[1389] Bug detection and debug instruction handling

[1390] Step 1:

[1391] A user discovers a bug in the game and reports the problem through the "Report a Bug" form.

[1392] Step 2:

[1393] The terminal sends the report to the server.

[1394] Step 3:

[1395] The server analyzes the received report and calls a bug detection AI to detect bugs in the code.

[1396] Step 4:

[1397] The server generates debugging instructions based on the detected bug information and sends them to the development team.

[1398] Step 5:

[1399] The server applies the corrected code or patch to the game and notifies the device of the update.

[1400] Difficulty and reward setting optimization process

[1401] Step 1:

[1402] The device transmits the player's progress and gameplay data to the server.

[1403] Step 2:

[1404] Based on the received player data, the server invokes an algorithm to optimize difficulty and reward settings and analyzes the data.

[1405] Step 3:

[1406] Based on the analysis results, the server generates new difficulty and reward settings and sends them to the device.

[1407] Step 4:

[1408] The device applies the received new setting data to the game and reflects it to the player.

[1409] The above is a specific program processing flow for the embodiment of the invention. This system enables efficient and creative game development, and constantly provides new content to players.

[1410] Example 1

[1411] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1412] In modern games, it is extremely important to be able to highly customize character appearance, movement, voice, dialogue, and story, as well as efficiently detect and fix bugs and adjust difficulty and rewards. However, there is still a lack of a system that can integrate these diverse elements and provide them to users in real time. Furthermore, there is also a lack of a means to automatically and effectively generate and optimize these elements. Therefore, improving the efficiency of game development while providing players with a fresh and engaging experience is a challenge.

[1413] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1414] In this invention, the server includes: [means for generating a character image using digital image generation means based on property information upon receiving a request from a user; [means for generating movement data from the digital image based on the generated character image;] [means for synthesizing voice using a natural language processing model based on lines generated from a scenario;] [means for generating dialogue and a story using a natural language processing model based on user input;] [means including artificial intelligence for detecting system malfunctions and generating correction instructions based thereon; and [means for analyzing user progress and gameplay data to optimize game difficulty and reward settings.] This makes it possible [to generate and optimize a variety of game elements in real time and provide players with an always fresh and engaging game experience].

[1415] "User" refers to the person who operates the system and plays the game.

[1416] "Attribute information" refers to attribute data necessary for generating images and movements, such as a character's appearance and characteristics.

[1417] "Digital image generation means" refers to algorithms or software for generating a digital image based on property information.

[1418] "Character image" refers to digital image data that represents the appearance of a generated character.

[1419] "Movement data" refers to digital data generated to represent the movements of a character.

[1420] "Scenario" refers to text data that includes instructions such as the story and dialogue used in the game.

[1421] "Natural language processing model" refers to a machine learning model used to understand and generate human language.

[1422] "Speech synthesis" refers to the technology of generating voice data from text data.

[1423] "Dialogue and Narrative" refers to the streams of text and dialogue that contain interactions between the user and in-game characters and NPCs.

[1424] "System malfunction" refers to an abnormality or error that occurs in software or hardware.

[1425] "Artificial intelligence" refers to technology that analyzes data, recognizes patterns, and solves problems through learning and inference.

[1426] "Progression" refers to a player's achievements or progress within a game.

[1427] "Gameplay data" refers to information such as player behavior, scores, and play history.

[1428] "Difficulty and reward settings" refers to the balance between the level of challenge in the game and the rewards obtained in return.

[1429] This invention relates to a game system that utilizes generative AI to provide endless fun. The system includes multiple algorithms and modules for generating character images, generating movements, synthesizing voices, generating dialogue and stories, detecting and debugging bugs, and optimizing game difficulty and reward settings.

[1430] Character image generation

[1431] First, the user selects the character's appearance and characteristics on the game's start screen. The specific steps are explained below.

[1432] The user selects the character's appearance and characteristics.

[1433] Example: User selects "Character with blue hair."

[1434] The terminal transmits the user's selection information to the server.

[1435] Example prompt: "Submit blue hair info for character generation."

[1436] The server generates a character image based on the received property information using a digital image generation means such as StableDiffusion.

[1437] The generated character image is sent back to the terminal, which displays this image to the user.

[1438] Example: A blue-haired character appears on the device screen.

[1439] Generating character movements

[1440] Next, a procedure for generating character movements based on the generated character image will be described.

[1441] The terminal uploads the generated character image to the server.

[1442] Example prompt: "Send an image of a blue-haired character to the server."

[1443] The server generates the motion data using an algorithm such as MotionDiffuse.

[1444] Example: The server generates walking motion data for a blue-haired character.

[1445] The generated movement data is sent to the terminal, which applies it to the character and displays it.

[1446] An animation of a blue-haired character walking appears on the device screen.

[1447] Speech synthesis

[1448] The procedure for synthesizing voice based on character lines is explained below.

[1449] The server generates dialogue data from a scenario or story.

[1450] Example: Generate dialogue data for "Hello"

[1451] The server generates the voice using a speech synthesis algorithm that uses LLM.

[1452] Example prompt: "Generate the greeting 'hello' aloud."

[1453] The generated audio data is sent to the terminal, which then plays it back in the game.

[1454] Example: A character in a game says "Hello."

[1455] Conversation and story generation

[1456] The steps for generating conversations between users and NPCs and game stories are explained below.

[1457] The user provides input to an NPC in the game.

[1458] Example: A user asks, "What's the next mission?"

[1459] The terminal transmits the context data to the server.

[1460] Example prompt: Send the question "What's your next mission?"

[1461] The server uses NLP models to generate dialogue and stories.

[1462] Example: Generate a description for the next mission

[1463] The generated data is sent to the terminal, which displays it to the user.

[1464] Example: An NPC in a game explains the details of the next mission.

[1465] Bug detection and debugging instructions

[1466] The procedure for detecting system malfunctions and generating debug instructions is described below.

[1467] When a user discovers a bug during the game, they report the problem to the server via their device.

[1468] Example prompt: "The character walks through the wall."

[1469] The server analyzes the reports and uses AI to detect bugs.

[1470] Example: Server detects bug that allows walking through walls

[1471] The server generates debug instructions and sends them to the development team.

[1472] Example: Generate specific debug instructions for fixing a bug

[1473] The development team makes corrections based on the debugging instructions and releases the corrected version.

[1474] Example: The modified game is provided to the user

[1475] Optimizing game difficulty and reward settings

[1476] The procedure for optimizing the game difficulty and reward settings is described below.

[1477] The device periodically transmits the player's progress and gameplay data to the server.

[1478] Example prompt: "Send data such as game completion time and success rate."

[1479] The server analyzes the received data and optimizes the balance between difficulty and reward.

[1480] Example: Adjusting difficulty based on player performance

[1481] The server sends the new difficulty and reward setting data back to the device.

[1482] Example: Sending optimized configuration data to the device

[1483] The device will apply the new configuration data to the game and reflect it to the player.

[1484] Example: The game restarts with a new difficulty level

[1485] By following the above procedure, the system of the present invention can generate and optimize a variety of game elements in real time, and provide users with a constantly fresh and engaging game experience.

[1486] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1487] Step 1:

[1488] The user selects the character's appearance and characteristics.

[1489] Input: User-selected character appearance and characteristics (e.g., a character with blue hair).

[1490] Specific operation: The user selects the character's attributes on the game's start screen.

[1491] Output: The selected character attribute information is generated on the terminal.

[1492] Step 2:

[1493] The terminal transmits the user's selection information to the server.

[1494] Input: Character selection information (attribute data for a blue-haired character).

[1495] Specific operation: The terminal transmits the selection information to the server via the network.

[1496] Output: The selection information sent to the server.

[1497] Step 3:

[1498] The server generates a character image based on the received property information using a digital image generation algorithm.

[1499] Input: Character attribute data (blue hair information).

[1500] Data processing or data calculation: Using a digital image generation algorithm such as StableDiffusion, a character image is generated based on the specified attribute information.

[1501] Specific operation: The server runs StableDiffusion and creates a character image.

[1502] Output: The generated character image.

[1503] Step 4:

[1504] The server transmits the generated character image to the terminal.

[1505] Input: Generated character image.

[1506] Specific operation: The server sends the generated image back to the terminal via the network.

[1507] Output: Character image sent to the device.

[1508] Step 5:

[1509] The terminal displays the received character image to the user.

[1510] Input: Character image sent from the server.

[1511] Specific operation: Display a character image on the device screen.

[1512] Output: The character image displayed to the user.

[1513] Step 6:

[1514] The terminal uploads the generated character image to the server.

[1515] Input: Character image stored on the device.

[1516] Specific operation: The device uploads the character image to the server via the network.

[1517] Output: Character image uploaded to the server.

[1518] Step 7:

[1519] The server generates movement data based on the character image.

[1520] Input: Uploaded character image.

[1521] Data processing or data calculation: Using algorithms such as MotionDiffuse to analyze character images and generate motion data.

[1522] Specific operation: The server executes MotionDiffuse and creates character movement data.

[1523] Output: Generated character motion data.

[1524] Step 8:

[1525] The server transmits the generated exercise data to the terminal.

[1526] Input: Generated movement data.

[1527] Specific operation: The server sends the exercise data to the terminal via the network.

[1528] Output: Exercise data sent to the device.

[1529] Step 9:

[1530] The device applies the received movement data to the character and displays it.

[1531] Input: Exercise data sent from the server.

[1532] Specific operation: The device applies the movement data to the character and displays it on the screen.

[1533] Output: A moving character displayed on the terminal.

[1534] Step 10:

[1535] The server synthesizes voice based on the dialogue data generated from the scenario.

[1536] Input: Scenario and dialogue data.

[1537] Data processing or data computation: Generate speech data from dialogue data using a natural language processing model.

[1538] Specific operation: The server executes a speech synthesis algorithm to convert the dialogue data into voice data.

[1539] Output: The generated audio data.

[1540] Step 11:

[1541] The server transmits the generated voice data to the terminal.

[1542] Input: The generated audio data.

[1543] Specific operation: The server sends the audio data to the terminal via the network.

[1544] Output: The audio data sent to the device.

[1545] Step 12:

[1546] The device then plays the received audio data as the character's lines.

[1547] Input: Audio data sent from the server.

[1548] Specific operation: The device applies the voice data to the character and plays it in the game.

[1549] Output: Character dialogue played in-game.

[1550] Step 13:

[1551] The context data entered by the user is transmitted from the terminal to the server.

[1552] Input: A question or request that the user types in the game.

[1553] Specific operation: The terminal sends user input to the server.

[1554] Output: The context data sent to the server.

[1555] Step 14:

[1556] The server generates conversations and stories based on the contextual data.

[1557] Input: Context data.

[1558] Data manipulation or data computation: Using natural language processing models to generate conversations or stories.

[1559] What it does: The server runs NLP models and creates conversations and stories.

[1560] Output: Generated conversation or story data.

[1561] Step 15:

[1562] The server sends the generated conversation and story data to the terminal.

[1563] Input: Generated conversation or story data.

[1564] Specific operation: The server sends data to the terminal via the network.

[1565] Output: Conversation or story data sent to your device.

[1566] Step 16:

[1567] The terminal displays the received conversation and story data to the user.

[1568] Input: Conversation or story data sent from the server.

[1569] Specific behavior: The device displays the data on the screen.

[1570] Output: The conversation or story that is displayed to the user.

[1571] Step 17:

[1572] If a user discovers a bug during the game, they report the problem to the server via their device.

[1573] Input: The description of the bug found by the user.

[1574] Specific operation: A user reports a bug to the server through a terminal.

[1575] Output: The bug report sent to the server.

[1576] Step 18:

[1577] The server analyzes the reported bug and uses AI to detect it.

[1578] Input: Bug report data.

[1579] Data manipulation or data manipulation: Using bug detection AI to analyze reports and detect bugs.

[1580] Specific operation: The server runs AI to identify the location and cause of the bug.

[1581] Output: Detected bug data.

[1582] Step 19:

[1583] The server generates debug instructions for the detected bugs and sends them to the development team.

[1584] Input: Detected bug data.

[1585] What happens: The server generates debug instructions and sends them to the development team.

[1586] Output: Debugging instructions sent to the development team.

[1587] Step 20:

[1588] The device periodically transmits the player's progress and gameplay data to the server.

[1589] Input: Player progress data, gameplay data.

[1590] Specific operation: The device sends progress and play data to the server.

[1591] Output: Progress data and play data sent to the server.

[1592] Step 21:

[1593] The server analyzes the received data and optimizes the balance between difficulty and reward.

[1594] Input: Progress data, play data.

[1595] Data processing or data arithmetic: Using data analysis algorithms to calculate the optimal balance between difficulty and reward.

[1596] Specific operation: The server runs the algorithm and adjusts the difficulty and reward.

[1597] Output: New difficulty and reward settings data.

[1598] Step 22:

[1599] The server sends the new difficulty and reward setting data back to the device.

[1600] Input: New difficulty and reward settings data.

[1601] Specific operation: The server sends the setting data to the terminal.

[1602] Output: The new configuration data sent to the terminal.

[1603] Step 23:

[1604] The device will apply the new configuration data to the game and reflect it to the player.

[1605] Input: New difficulty and reward settings data.

[1606] Specific operation: The device applies and reflects the new configuration data in the game.

[1607] Output: The game environment with the new settings.

[1608] (Application example 1)

[1609] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1610] It is necessary to make the virtual shopping experience more engaging and interactive, provide an entertainment-rich shopping experience that users can enjoy endlessly, and increase purchasing motivation by providing product recommendations and navigation in real time based on users' preferences.

[1611] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1612] In this invention, the server includes: [means for generating character images;] [means for generating movements based on the generated character images;] [means for synthesizing voice based on lines;] [means for generating conversations and stories;] [means for detecting bugs and generating debugging instructions;] [means for optimizing game difficulty and reward settings;] [means for recommending products based on user attribute information and displaying them as navigation characters;] [voice synthesis means for explaining products using the voice lines of the generated characters; and [means for generating conversations and dialogues based on interactions with the user. This makes it possible to provide users with an endlessly enjoyable virtual shopping experience and increase their desire to purchase.

[1613] A "character image" is a visual representation of a virtual person or object generated based on the user's attribute information.

[1614] "Movement data" is data on the actions and movements that a character appears to perform based on the generated character image.

[1615] "Speech synthesis" is the process of creating artificially generated speech based on text data.

[1616] "Conversations and stories" refer to dialogues and scenarios created based on user interaction.

[1617] "Bug detection" is the process of discovering errors or defects in software or systems.

[1618] "Debugging instructions" are specific steps or instructions for fixing a detected bug.

[1619] "Optimizing difficulty and reward settings" is the process of adjusting the appropriate challenge and reward levels depending on the progress of a game or experience.

[1620] "User attribute information" is data that includes user preferences, behavioral history, personal settings, etc.

[1621] "Product recommendation" is the process of suggesting highly relevant products based on a user's attribute information.

[1622] A "navigation character" is a virtual character that acts as a guide for users within a virtual store.

[1623] An "interaction" is an interaction between a user and a system.

[1624] This invention relates to a system that provides users with an endlessly enjoyable virtual shopping experience. The system uses a generative AI model and multiple algorithms and modules to generate character images, generate movements, synthesize voices, and generate conversations and stories. It also includes a function that recommends products based on the user's attribute information and displays them as navigation characters.

[1625] The server generates a character image upon receiving a request from the user. This uses StableDiffusion, an image generation algorithm based on the user's attribute information. The user selects the character's appearance and characteristics on their smartphone and sends their selection to the server. The server creates a character image based on the received attribute information and sends the generated image back to the device. The device displays this image to the user.

[1626] The server then uses algorithms such as MotionDiffuse to generate character movement based on the generated character image. The device uploads the character image to the server, which analyzes the image and generates motion data. The generated motion data is then sent to the device, which applies it to the character and displays it.

[1627] The server then generates voice based on the dialogue data. Dialogue data is generated from the scenario or story, and this is input into a speech synthesis algorithm using a large-scale language model (LLM). The generated voice data is sent to the device, which then plays it back to the user.

[1628] The server generates conversations and stories using NLP models. When the user interacts with the navigation character, the device sends the context data to the server. The server generates conversations between the user and the character and the ongoing story based on this context data and sends it to the device. The device then displays the generated data to the user.

[1629] Bug detection and debug instructions are performed by a bug detection AI on the server. When a user finds a bug in the system, they report the problem to the server via their terminal. The server analyzes the report and uses AI to detect bugs in the code. Debug instructions for the detected bug are generated and sent to the development team. The problem is resolved by releasing a patch to fix it.

[1630] Finally, the system periodically transmits user progress and interaction data to the server, which analyzes this data and configures the system to optimize the balance between difficulty and reward. The new configuration data is sent back to the device, which then applies it to the system and reflects it to the user.

[1631] For example, the following conversation might be generated by a generative AI model:

[1632] User: "What are your recent recommendations?"

[1633] AI: "My latest recommendation is a newly released smartwatch. It features a long battery life and comprehensive health tracking."

[1634] In this way, the system can provide users with an endlessly enjoyable virtual shopping experience and increase their desire to purchase.

[1635] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1636] Step 1:

[1637] The user selects the character's appearance and characteristics using a smartphone device.

[1638] Input: User selection information (attribute information)

[1639] Output: Sends user selection information to the server

[1640] Specific operation: The user selects the character's appearance and characteristics on the application interface and sends that information to the server.

[1641] Step 2:

[1642] The server generates a character image using StableDiffusion based on the attribute information received from the user.

[1643] Input: User attribute information

[1644] Output: Generated character image

[1645] Specific operation: The server uses the StableDiffusion algorithm to generate a character image from the user's attribute information.

[1646] Step 3:

[1647] The server returns the generated character image to the terminal.

[1648] Input: Generated character image

[1649] Output: Character image returned to the device

[1650] Specific operation: The server sends the generated character image to the terminal, and the terminal displays this image.

[1651] Step 4:

[1652] The device uploads the character image to the server, and the server generates the motion data using MotionDiffuse.

[1653] Input: Character image

[1654] Output: Generated motion data

[1655] Specific operation: The device uploads the character image to the server, and the server generates motion data using the MotionDiffuse algorithm.

[1656] Step 5:

[1657] The server transmits the generated motion data to the terminal.

[1658] Input: Generated motion data

[1659] Output: Movement data sent to the device

[1660] Specific operation: The server sends the generated movement data to the device, which then applies it to the character and displays it.

[1661] Step 6:

[1662] The server synthesizes voice based on the scenario or story.

[1663] Input: Scenario and story dialogue data

[1664] Output: Generated audio data

[1665] Specific operation: The server generates speech from dialogue data using a speech synthesis algorithm based on a large-scale language model (LLM) and sends it to the terminal.

[1666] Step 7:

[1667] The terminal reproduces the generated voice data and lets the user hear it.

[1668] Input: Generated audio data

[1669] Output: The audio played to the user

[1670] Specific operation: The device plays the audio data received from the server.

[1671] Step 8:

[1672] The user interacts with the navigation character and the terminal transmits the context data to the server.

[1673] Input: User interaction

[1674] Output: Context data sent to the server

[1675] Specific operation: The user interacts with the navigation character, and the content of the interaction is sent from the terminal to the server as context data.

[1676] Step 9:

[1677] The server uses NLP models to generate conversations and stories and send them to the device.

[1678] Input: Context data

[1679] Output: Generated conversations and story data

[1680] Specific operation: The server analyzes the context data using an NLP model, generates a conversation or story, and sends it to the device.

[1681] Step 10:

[1682] The terminal displays the generated conversations and stories to the user.

[1683] Input: Generated conversation or story data

[1684] Output: The conversation or story that is displayed to the user

[1685] Specific operation: The device displays the conversation and story data received from the server and shows it to the user.

[1686] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1687] This invention relates to a system that utilizes generative AI and an emotion engine to provide users with an endlessly enjoyable gaming experience. This system includes multiple algorithms and modules that generate character images and create movements, voices, dialogue, and stories. It also incorporates an emotion engine that recognizes the user's emotions and dynamically changes the game content.

[1688] First, the system generates a character image based on the attributes of the character selected by the user. The user selects the character's appearance and characteristics on the game's opening screen, and the device sends that selection to the server. The server uses an image generation algorithm (e.g., StableDiffusion) to create a character image based on the received attribute information, and sends the generated image back to the device. The device then displays this image to the user.

[1689] Next, the server generates character movement based on the generated character image. The device uploads the character image to the server, and the server analyzes the image and invokes an algorithm (such as MotionDiffuse) to generate motion data. The generated motion data is sent to the device, which applies it to the character and displays it.

[1690] For voice synthesis, the server generates voice based on dialogue data. Dialogue data is generated from the scenario or story, and this is input into the voice synthesis algorithm. The generated voice data is sent to the device, which plays it in the game.

[1691] Conversations and stories are generated by an NLP model (e.g., ChatGPT) on the server. When a user talks to an NPC character in the game, the device sends context data to the server. The server generates conversations and stories based on this context data and sends them to the device. The device then displays the generated data to the user.

[1692] The emotion engine has the ability to recognize emotions by analyzing the user's facial expression and voice data. The device collects data provided by the user through the camera and microphone and sends it to the server. The server inputs the received data into the emotion engine and recognizes the user's emotional state. The game content is dynamically changed based on the emotion recognition results. For example, if the user is surprised, an enemy character in the game can suddenly appear.

[1693] Bug detection and debug instructions are carried out by a bug detection AI on the server. When a user discovers a bug during the game, they report the problem to the server via their device. The server analyzes the report and uses AI to detect bugs in the code. Debug instructions for the detected bug are generated and sent to the development team. The problem is resolved by releasing a patch to fix the problem.

[1694] Finally, the server also optimizes the game's difficulty and reward settings. The device periodically sends the player's progress and gameplay data to the server. The server analyzes this data and sets the optimal balance between difficulty and reward. The new setting data is sent back to the device, which applies it to the game and provides it to the player.

[1695] For example, if a user selects red hair and armor as their costume during character creation, the server generates an image of the character wearing red hair and armor based on those attributes. The server then generates movements and sounds for the character, and the emotion engine recognizes the user's emotions and dynamically changes in-game events. For example, if the user looks tired, an event will occur in which a friendly NPC in the game offers a support item.

[1696] Through the above process, this invention enables efficient and creative game development, reducing development effort while providing players with a consistently new and engaging gaming experience. Furthermore, incorporating the user's emotional state can provide a more personalized gaming experience.

[1697] The processing flow will be explained below.

[1698] Character generation process

[1699] Step 1:

[1700] The user selects the character's attributes (e.g., hair color, clothing, gender) on the game's start screen.

[1701] Step 2:

[1702] The terminal transmits the selected attribute data to the server.

[1703] Step 3:

[1704] Based on the received attribute data, the server invokes an image generation algorithm (e.g., StableDiffusion) to generate a character image.

[1705] Step 4:

[1706] The server transmits the generated character image to the terminal.

[1707] Step 5:

[1708] The terminal displays the received character image to the user.

[1709] Motion generation processing

[1710] Step 1:

[1711] The terminal uploads the generated character image to the server.

[1712] Step 2:

[1713] The server calls an algorithm (e.g., MotionDiffuse) that generates motion data based on the received character image.

[1714] Step 3:

[1715] The server transmits the generated motion data to the terminal.

[1716] Step 4:

[1717] The terminal applies the received motion data to the character and displays the motion to the user.

[1718] Speech synthesis processing

[1719] Step 1:

[1720] The server generates dialogue data based on a scenario or story.

[1721] Step 2:

[1722] The server passes the dialogue data to a speech synthesis algorithm to generate speech data.

[1723] Step 3:

[1724] The server transmits the generated voice data to the terminal.

[1725] Step 4:

[1726] The device plays the received audio data in the game.

[1727] Conversation / story generation processing

[1728] Step 1:

[1729] The user talks to an NPC character in the game.

[1730] Step 2:

[1731] The device transmits user input and the current game state to the server.

[1732] Step 3:

[1733] The server invokes an NLP model (e.g., ChatGPT) based on the received context data to generate a conversation or story.

[1734] Step 4:

[1735] The server sends the generated conversations and stories to the device.

[1736] Step 5:

[1737] The terminal displays the received conversation data to the user.

[1738] Emotion Recognition Processing

[1739] Step 1:

[1740] The user provides facial expression data and voice data through a camera and microphone while playing the game.

[1741] Step 2:

[1742] The terminal transmits the collected facial expression data and voice data to the server.

[1743] Step 3:

[1744] The server inputs the received data into an emotion engine to recognize the user's emotion.

[1745] Step 4:

[1746] The server dynamically adjusts in-game events and conversations based on the emotion recognition results.

[1747] Step 5:

[1748] The device displays the adjusted events and conversations in the game and reflects them to the user.

[1749] Bug detection and debug instruction handling

[1750] Step 1:

[1751] A user discovers a bug in the game and reports the problem through a bug report form.

[1752] Step 2:

[1753] The terminal sends the report to the server.

[1754] Step 3:

[1755] The server analyzes the received report and calls a bug detection AI to detect bugs in the code.

[1756] Step 4:

[1757] The server generates debug instructions for the detected bugs and sends them to the development team.

[1758] Step 5:

[1759] The server applies the corrected code or patch to the game and notifies the device of the update.

[1760] Difficulty and reward setting optimization process

[1761] Step 1:

[1762] The device transmits the player's progress and gameplay data to the server.

[1763] Step 2:

[1764] Based on the received player data, the server invokes an algorithm to optimize difficulty and reward settings and analyzes the data.

[1765] Step 3:

[1766] Based on the analysis results, the server generates new difficulty and reward settings and sends them to the device.

[1767] Step 4:

[1768] The device applies the received new setting data to the game and reflects it to the player.

[1769] The above is a specific program processing flow for implementing the invention in combination with an emotion engine. This system provides a personalized gaming experience that incorporates the user's emotions.

[1770] Example 2

[1771] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1772] In conventional game systems, it has been difficult to dynamically adapt game progression and generate interactive characters that reflect the user's individual emotional state. Furthermore, bug detection and game difficulty adjustment are performed manually, resulting in low efficiency and limited improvements to the user experience. New algorithms and technologies are needed to address these challenges.

[1773] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1774] In this invention, the server includes means for generating character images, means for generating movements based on the generated character images, means for synthesizing voice based on dialogue, means for generating conversations and stories, means for recognizing the user's emotional state and dynamically changing game content, means for detecting bugs and generating debugging instructions, and means for optimizing game difficulty and reward settings. This enables a dynamic and personalized game experience that matches the user's emotions, and makes it possible to provide a high-quality game service by automatically detecting bugs and automatically adjusting game balance.

[1775] "Character image" is graphic data of a visual character used in the game, generated based on attribute information selected by the user.

[1776] "Movement data" is a series of data generated based on a character image to represent the movement of the character.

[1777] "Dialogue data" is text data of words spoken by characters that is generated based on the scenario or story within the game.

[1778] A "voice synthesis algorithm" is a calculation procedure or program for generating voice data based on dialogue data.

[1779] A "conversation generation algorithm" is a computational procedure or program for generating dialogue and stories between characters in a game.

[1780] An "emotion engine" is a group of algorithms and programs that analyze a user's facial expression data and voice data to recognize their emotional state.

[1781] "Bug detection AI" is an artificial intelligence that automatically detects glitches (bugs) that occur in games and generates instructions for debugging.

[1782] "Difficulty and reward optimization" is the process of adjusting the balance of a game's difficulty and available rewards based on player progress and gameplay data.

[1783] An "image generation algorithm" is a calculation procedure or program for generating a new character image based on attribute information selected by the user.

[1784] "Dynamic change" refers to changing the content and progress of the game in real time according to the user's emotional state and the game situation.

[1785] MODE FOR CARRYING OUT THE INVENTION

[1786] This invention is a system that utilizes generative AI and an emotion engine to provide users with an endlessly enjoyable gaming experience. To implement the invention, specific hardware and software are required to process and calculate various data.

[1787] Hardware and software used

[1788] This system operates in a network environment that includes user devices (PCs, smartphones, etc.) and servers. Specific software includes image generation algorithms (e.g., StableDiffusion), motion generation algorithms (e.g., MotionDiffuse), speech synthesis algorithms, natural language processing models (e.g., ChatGPT), and emotion engines.

[1789] System Operation Overview

[1790] 1. Character image generation

[1791] The user selects the character's appearance and characteristics on the game start screen. The device sends the selection to the server, which uses an image generation algorithm to generate a character image based on the received attribute information. This image is sent back to the device and displayed.

[1792] 2. Character Movement Generation

[1793] The device sends the generated character image to the server, which uses a motion generation algorithm to generate character movements from the image, and the generated motion data is sent back to the device and applied to the character.

[1794] 3. Speech generation

[1795] Based on the dialogue data generated according to the scenario and story, the server uses a speech synthesis algorithm to generate voice data, which is then sent to the device and played back in the game.

[1796] 4. Conversation and story generation

[1797] When a user talks to an NPC character in a game, the device sends that context data to the server, which uses a natural language processing model to generate a dialogue or story, which is then sent to the device and displayed to the user.

[1798] 5. Emotion Recognition and Dynamic Change of Game Content

[1799] The device collects the user's facial expression and voice data through a camera and microphone and sends it to the server. The server then uses an emotion engine to recognize the user's emotional state. Based on this recognition, in-game events and character behavior are dynamically changed.

[1800] 6. Bug Detection and Debugging Instructions

[1801] If a user discovers a bug during the game, they report the problem to the server via their device. The server uses a bug detection AI to analyze the report and detect bugs in the code. Debug instructions are generated and sent to the development team.

[1802] 7. Optimizing difficulty and reward settings

[1803] The device periodically sends the player's progress and gameplay data to the server. The server analyzes this data and configures the game to optimize the balance between difficulty and rewards. The new configuration data is sent back to the device and applied to the game.

[1804] Specific examples

[1805] For example, if a user selects red hair and armor as their outfit, the server generates a character image wearing red hair and armor based on those attributes. The server then generates movements and sounds for the character, and the emotion engine recognizes the user's emotions and dynamically changes in-game events. For example, if the user shows a tired expression, an event occurs in which a friendly character in the game provides a support item.

[1806] Prompt Sentence Examples

[1807] "Generate an image of a character with red hair and armor."

[1808] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1809] Step 1: User character attribute selection

[1810] Description: The user selects the character's appearance and characteristics (hair color, outfit, etc.) on the game's start screen.

[1811] Input: User-selected character attribute information (e.g., hair color "red", costume "armor").

[1812] Output: Data that sends selected attribute information from the device to the server.

[1813] Specific operation: The user selects "red hair" and "armor," and the device sends that information to the server.

[1814] Step 2: Generate character images

[1815] Description: The server generates a character image using an image generation algorithm based on the received attribute information.

[1816] Input: Character attribute information selected by the user.

[1817] Output: Send the generated character image data to the terminal.

[1818] What it does: The server uses the StableDiffusion algorithm to generate an image of a character with red hair and armor, and sends it back to the device.

[1819] Step 3: Character Movement Generation

[1820] Description: The device uploads the generated character image to the server, and the server generates the character's movements using a movement generation algorithm.

[1821] Input: Generated character image.

[1822] Output: Send the generated motion data to the device.

[1823] How it works: The server uses the MotionDiffuse algorithm to generate character motion data and sends it to the device, which then applies this motion to the character.

[1824] Step 4: Generate audio

[1825] Description: The server generates dialogue data from a scenario or story and generates speech using a speech synthesis algorithm.

[1826] Input: Dialogue data (text format).

[1827] Output: Sends the generated audio data to the device.

[1828] What happens: The server uses a speech synthesis algorithm to generate a voice for the phrase "Hello" and sends it to the device, which then plays it back in the game.

[1829] Step 5: Generate conversations and stories

[1830] Description: When a user talks to an NPC character in a game, the device sends that context data to the server, which then generates the dialogue and story.

[1831] Input: Context data (user-NPC interactions, current scenario, etc.).

[1832] Output: Send the generated conversation data and story data to the device.

[1833] How it works: The server uses the ChatGPT algorithm to generate a conversation saying, "Go down this path and you'll find the treasure," and sends it to the device, which then displays it to the NPC in the game.

[1834] Step 6: Emotion recognition and dynamic change of game content

[1835] Description: The device collects the user's facial expression and voice data and sends it to the server. The server uses an emotion engine to recognize the user's emotional state and dynamically change the game content.

[1836] Input: facial expression data, audio data.

[1837] Output: Dynamically modified game content based on emotion recognition results.

[1838] Specific behavior: The server recognizes that the user is surprised and triggers an event in the game in which an enemy character suddenly appears.

[1839] Step 7: Bug detection and debugging instructions

[1840] Description: When a user discovers a bug during the game, they report the problem to the server through their terminal. The server uses a bug detection AI to detect bugs in the code and generate debug instructions.

[1841] Input: Bug report data.

[1842] Output: Debug instructions, fix patch.

[1843] What happens: The server analyzes the user's report that "the character can go through walls," detects the bug, sends a debug instruction to the development team, and releases a fix patch.

[1844] Step 8: Optimize difficulty and reward settings

[1845] Description: The device sends player progress and gameplay data to the server, which analyzes this data and configures the game to optimize the balance between difficulty and rewards.

[1846] Input: Player progress, gameplay data.

[1847] Output: Optimized difficulty and reward settings.

[1848] What it does: The server analyzes the player's high score, sets more difficult levels and rewards, and sends the new configuration data back to the device to apply it to the game.

[1849] (Application example 2)

[1850] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1851] Conventional security systems operate based on fixed settings, making it difficult to respond flexibly to user emotions and situations. Furthermore, there was no technology that could dynamically change security responses based on emotions in real time. As a result, even if a user is feeling anxious or nervous, it is not possible to provide appropriate safety measures, which could lead to a decline in the quality of security.

[1852] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1853] In this invention, the server includes: [means for generating character images;] [means for generating movements based on the generated character images;] [means for synthesizing voice based on dialogue;] [means for generating conversations and stories;] [means for detecting bugs and generating debugging instructions;] [means for optimizing game difficulty and reward settings;] [means for recognizing user emotions and dynamically changing system settings based on those emotions; and [means for dynamically generating emotion-based security measures using a generative AI model.] This enables appropriate security responses that are in line with user emotions to be provided in real time, improving the quality of security.

[1854] "Means for generating a character image" refers to an algorithm or module that creates an image of a character based on attribute information selected by a user.

[1855] "Means for generating movement based on the generated character image" refers to an algorithm or module that generates movement data from the generated character image.

[1856] "Means for synthesizing voice based on dialogue" refers to a voice synthesis algorithm or module that converts dialogue data from a scenario or story into voice.

[1857] "Means for generating conversations and stories" refers to natural language processing models and algorithms for generating conversations and stories with users.

[1858] "Means for detecting bugs and generating debug instructions" refers to algorithms or modules for analyzing problems reported by users and generating instructions required for debugging.

[1859] "Means for optimizing game difficulty and reward settings" refers to algorithms or modules that analyze users' gameplay data and optimally adjust the balance between difficulty and rewards.

[1860] "Means for recognizing user emotions and dynamically changing system settings based on those emotions" refers to algorithms or modules that analyze emotions from the user's facial expressions and voice and dynamically change system settings based on the results.

[1861] "Means for dynamically generating emotion-based security measures using a generative AI model" refers to an AI model that receives emotion data as input and generates security measures appropriate to those emotions.

[1862] This invention is a system that uses a generative AI model and an emotion engine to provide a dynamic gaming experience and security measures based on user emotions. The system includes multiple algorithms and modules and runs on devices such as smartphones and smart glasses.

[1863] The hardware required to implement the invention is a device equipped with a camera and microphone (e.g., a smartphone or smart glasses). The software uses Python and Django (server-side), an emotion API (e.g., Microsoft Azure Emotion API), and a natural language processing model (e.g., ChatGPT).

[1864] 1. Character image generation

[1865] The user uses the device to select the character's appearance and characteristics on the game's opening screen. This selection information is sent from the device to the server. The server generates a character image using an image generation algorithm (e.g., Stable Diffusion) based on the received attribute information, and sends the generated image back to the device. The device then displays this image to the user.

[1866] 2. Generating Character Movements

[1867] The generated character image is uploaded from the device to the server, which analyzes the image and invokes an algorithm (e.g., MotionDiffuse) to generate motion data. The generated motion data is sent to the device, which applies it to the character and displays it to the user.

[1868] 3. Speech Synthesis

[1869] The server generates dialogue data from the scenario and story, and synthesizes voice based on that. The dialogue data is input into a voice synthesis algorithm, and the generated voice data is sent to the device, which then plays it in the game.

[1870] 4. Conversation and story generation

[1871] When a user talks to an NPC character in a game, the device sends context data to the server, which generates a conversation or story based on the context data and sends it to the device, which then displays the generated data to the user.

[1872] 5. User Emotion Recognition and Dynamic System Changes

[1873] The device collects facial expression and voice data provided by the user through the camera and microphone and sends it to the server. The server inputs this data into an emotion engine (e.g., Microsoft Azure Emotion API) to recognize the user's emotional state. System settings are dynamically changed based on the emotion recognition results. For example, if the user feels anxious, security measures are strengthened.

[1874] 6. Emotion data collection using cameras and microphones

[1875] The camera and microphone are used to collect the user's facial expression and voice data. Specifically, the camera captures the user's facial expressions and the microphone records their voice. This data is sent to the emotion API in real time, and the user's emotional state is analyzed.

[1876] 7. Emotion-based security measures

[1877] Based on the emotion data, the server uses a generative AI model (e.g., ChatGPT) to dynamically generate security measures appropriate to the user's emotional state. For example, if the user feels anxious, the generative AI model will suggest security measures such as issuing a warning notification or automatically locking the smart lock.

[1878] Examples of examples and prompts:

[1879] If the user feels anxious at night, the data from the camera and microphone will be analyzed to generate the result "anxious." Based on this, the generative AI will increase the sensitivity of the window and door sensors and automatically generate instructions to lock all the doors in the house.

[1880] Text format

[1881] User emotion: anxious. Suggest measures to enhance home security during the user's anxious state.

[1882] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1883] Step 1:

[1884] The user selects the character's appearance and characteristics.

[1885] Input: Appearance and trait selection data.

[1886] How it works: The user selects the character's appearance and characteristics on the game's opening screen and enters them into the device.

[1887] Output: User-selected attribute information.

[1888] Step 2:

[1889] The terminal transmits the user-selected attribute information to the server.

[1890] Input: User-selected attribute information.

[1891] Operation: The terminal transmits the selected attribute information to the server.

[1892] Output: Attribute information received by the server.

[1893] Step 3:

[1894] The server generates a character image based on the attribute information.

[1895] Input: Attribute information.

[1896] Operation: The server inputs the attribute information into an image generation algorithm (e.g., StableDiffusion) to generate a character image.

[1897] Output: Character image.

[1898] Step 4:

[1899] The server returns the generated character image to the terminal.

[1900] Input: Generated character image.

[1901] Operation: The server sends the generated character image to the device.

[1902] Output: Character image received by the device.

[1903] Step 5:

[1904] The terminal displays the character image to the user.

[1905] Input: Received character image.

[1906] Operation: The device displays the character image on the screen.

[1907] Output: The character image displayed to the user.

[1908] Step 6:

[1909] Movement data is generated based on the character image.

[1910] Input: Character image.

[1911] How it works: The device uploads a character image to the server, which then invokes an algorithm (e.g., MotionDiffuse) that analyzes the image and generates motion data.

[1912] Output: Motion data.

[1913] Step 7:

[1914] The server transmits the generated motion data to the terminal.

[1915] Input: Generated motion data.

[1916] Operation: The server sends the generated motion data to the terminal.

[1917] Output: The motion data received by the device.

[1918] Step 8:

[1919] The device applies the movement data to the character and displays it.

[1920] Input: Received motion data.

[1921] Movement: The device applies the movement data to the character and displays it on the screen.

[1922] Output: A moving character displayed to the user.

[1923] Step 9:

[1924] The server generates dialogue data from the scenario or story and synthesizes the voice.

[1925] Input: A scenario or story.

[1926] How it works: The server generates dialogue data for a scenario or story, inputs it into a speech synthesis algorithm, and synthesizes the speech.

[1927] Output: Audio data.

[1928] Step 10:

[1929] The server transmits the generated voice data to the terminal.

[1930] Input: The generated audio data.

[1931] Operation: The server sends the generated voice data to the terminal.

[1932] Output: The audio data received by the device.

[1933] Step 11:

[1934] The device plays the audio data in the game.

[1935] Input: Received audio data.

[1936] Operation: The device plays the audio data in the game through the playback device.

[1937] Output: The audio played to the user.

[1938] Step 12:

[1939] The server generates conversations with NPC characters.

[1940] Input: A conversation request from the user.

[1941] How it works: When a user talks to an NPC character, the device sends context data to the server, which then uses a natural language processing model (e.g., ChatGPT) to generate a conversation or story.

[1942] Output: The generated conversation data.

[1943] Step 13:

[1944] The terminal displays the generated conversation data to the user.

[1945] Input: Generated conversation data.

[1946] Operation: The device displays the conversation data on the display.

[1947] Output: The conversation as displayed to the user.

[1948] Step 14:

[1949] The device collects the user's emotional data and sends it to the server.

[1950] Input: User's facial expression data, voice data.

[1951] How it works: It uses a camera and microphone to collect facial expression and voice data from the user and sends it to a server.

[1952] Output: Emotion data sent to the server.

[1953] Step 15:

[1954] The server analyzes the emotional data and recognizes the emotional state.

[1955] Input: Collected emotion data.

[1956] How it works: Emotion data is fed into an emotion engine (e.g., Microsoft Azure Emotion API) to analyze the emotional state.

[1957] Output: Parsed emotional state data.

[1958] Step 16:

[1959] The server generates security measures according to the emotional state.

[1960] Input: Emotional state data.

[1961] How it works: Based on the recognized emotional state data, a generative AI model (e.g., ChatGPT) is used to generate emotion-based security measures.

[1962] Output: The generated security measures.

[1963] Step 17:

[1964] The server transmits the generated security measures to the terminal.

[1965] Input: The generated security measures.

[1966] Operation: The server sends the generated security measures to the terminal.

[1967] Output: Security measures data received by the device.

[1968] Step 18:

[1969] The device implements security measures.

[1970] Input: Received security measures data.

[1971] Operation: The device will take necessary security measures based on the received security data, such as increasing the sensitivity of window and door sensors and automatically locking smart locks.

[1972] Output: The security measures taken.

[1973] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1974] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1975] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1976] [Fourth embodiment]

[1977] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1978] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1979] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1980] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1981] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1982] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1983] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1984] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1985] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1986] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1987] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1988] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1989] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1990] This invention relates to a game system that utilizes generative AI to provide endless fun. The system includes multiple algorithms and modules for generating character images, generating movements, synthesizing voices, generating dialogue and stories, detecting and debugging bugs, and optimizing game difficulty and reward settings.

[1991] First, the server receives a request from the user and generates a character image. This uses an image generation algorithm based on attribute information. The user selects the character's appearance and characteristics on the game's start screen, and the device sends this selection to the server. The server creates a character image using an image generation algorithm such as StableDiffusion based on the received attribute information, and sends the generated image back to the device. The device displays this image to the user.

[1992] The server then generates character movement based on the generated character image. This is done using algorithms such as MotionDiffuse. The device uploads the character image to the server, which analyzes the image and generates movement data. The generated movement data is sent to the device, which applies it to the character and displays it.

[1993] Furthermore, for voice synthesis, the server generates voice based on the dialogue data. Dialogue data is generated from the scenario or story, and this is input into a voice synthesis algorithm using LLM. The generated voice data is sent to the device, which then plays it in the game.

[1994] Conversations and stories are generated by the server using an NLP model. When a user interacts with an NPC character in the game, the device sends that context data to the server. The server uses this context data to generate conversations between the user and the NPC and the ongoing story, and sends them to the device. The device then displays the generated data to the user.

[1995] Bug detection and debug instructions are carried out by a bug detection AI on the server. When a user discovers a bug during the game, they report the problem to the server via their device. The server analyzes the report and uses AI to detect bugs in the code. Debug instructions for the detected bug are generated and sent to the development team. The problem is resolved by releasing a patch to fix the problem.

[1996] Finally, the server also optimizes the game's difficulty and reward settings. The device periodically sends the player's progress and gameplay data to the server. The server analyzes this data and sets the optimal balance between difficulty and reward. The new setting data is sent back to the device, which applies it to the game and provides it to the player.

[1997] Through the above process, this invention enables efficient and creative game development, reducing development efforts while providing players with a consistently new and engaging gaming experience.

[1998] The processing flow will be explained below.

[1999] Character generation process

[2000] Step 1:

[2001] The user selects the character's attributes (e.g., hair color, clothing, gender) on the game's start screen.

[2002] Step 2:

[2003] The terminal transmits the selected attribute data to the server.

[2004] Step 3:

[2005] Based on the received attribute data, the server invokes an image generation algorithm such as StableDiffusion to generate a character image.

[2006] Step 4:

[2007] The server transmits the generated character image to the terminal.

[2008] Step 5:

[2009] The terminal displays the received character image to the user.

[2010] Motion generation processing

[2011] Step 1:

[2012] The terminal uploads the generated character image to the server.

[2013] Step 2:

[2014] Based on the received character image, the server invokes algorithms such as MotionDiffuse to generate character movement data.

[2015] Step 3:

[2016] The server transmits the generated motion data to the terminal.

[2017] Step 4:

[2018] The device stores the received movement data locally and applies it to the character for display.

[2019] Speech synthesis processing

[2020] Step 1:

[2021] The server generates dialogue data based on a scenario or story.

[2022] Step 2:

[2023] The server passes the generated dialogue data to a speech synthesis algorithm to generate voice data.

[2024] Step 3:

[2025] The server transmits the generated voice data to the terminal.

[2026] Step 4:

[2027] The device stores the received audio data locally and plays it in the game.

[2028] Conversation / story generation processing

[2029] Step 1:

[2030] The user talks to an NPC character in the game.

[2031] Step 2:

[2032] The device transmits user input and the current game state to the server.

[2033] Step 3:

[2034] Based on the received context data, the server invokes NLP models such as ChatGPT to generate conversations and stories.

[2035] Step 4:

[2036] The server sends the generated conversations and stories to the device.

[2037] Step 5:

[2038] The terminal stores the received conversation data locally and displays it to the user.

[2039] Bug detection and debug instruction handling

[2040] Step 1:

[2041] A user discovers a bug in the game and reports the problem through the "Report a Bug" form.

[2042] Step 2:

[2043] The terminal sends the report to the server.

[2044] Step 3:

[2045] The server analyzes the received report and calls a bug detection AI to detect bugs in the code.

[2046] Step 4:

[2047] The server generates debugging instructions based on the detected bug information and sends them to the development team.

[2048] Step 5:

[2049] The server applies the corrected code or patch to the game and notifies the device of the update.

[2050] Difficulty and reward setting optimization process

[2051] Step 1:

[2052] The device transmits the player's progress and gameplay data to the server.

[2053] Step 2:

[2054] Based on the received player data, the server invokes an algorithm to optimize difficulty and reward settings and analyzes the data.

[2055] Step 3:

[2056] Based on the analysis results, the server generates new difficulty and reward settings and sends them to the device.

[2057] Step 4:

[2058] The device applies the received new setting data to the game and reflects it to the player.

[2059] The above is a specific program processing flow for the embodiment of the invention. This system enables efficient and creative game development, and constantly provides new content to players.

[2060] Example 1

[2061] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2062] In modern games, it is extremely important to be able to highly customize character appearance, movement, voice, dialogue, and story, as well as efficiently detect and fix bugs and adjust difficulty and rewards. However, there is still a lack of a system that can integrate these diverse elements and provide them to users in real time. Furthermore, there is also a lack of a means to automatically and effectively generate and optimize these elements. Therefore, improving the efficiency of game development while providing players with a fresh and engaging experience is a challenge.

[2063] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2064] In this invention, the server includes: [means for generating a character image using digital image generation means based on property information upon receiving a request from a user; [means for generating movement data from the digital image based on the generated character image;] [means for synthesizing voice using a natural language processing model based on lines generated from a scenario;] [means for generating dialogue and a story using a natural language processing model based on user input;] [means including artificial intelligence for detecting system malfunctions and generating correction instructions based thereon; and [means for analyzing user progress and gameplay data to optimize game difficulty and reward settings.] This makes it possible [to generate and optimize a variety of game elements in real time and provide players with an always fresh and engaging game experience].

[2065] "User" refers to the person who operates the system and plays the game.

[2066] "Attribute information" refers to attribute data necessary for generating images and movements, such as a character's appearance and characteristics.

[2067] "Digital image generation means" refers to algorithms or software for generating a digital image based on property information.

[2068] "Character image" refers to digital image data that represents the appearance of a generated character.

[2069] "Movement data" refers to digital data generated to represent the movements of a character.

[2070] "Scenario" refers to text data that includes instructions such as the story and dialogue used in the game.

[2071] "Natural language processing model" refers to a machine learning model used to understand and generate human language.

[2072] "Speech synthesis" refers to the technology of generating voice data from text data.

[2073] "Dialogue and Narrative" refers to the streams of text and dialogue that contain interactions between the user and in-game characters and NPCs.

[2074] "System malfunction" refers to an abnormality or error that occurs in software or hardware.

[2075] "Artificial intelligence" refers to technology that analyzes data, recognizes patterns, and solves problems through learning and inference.

[2076] "Progression" refers to a player's achievements or progress within a game.

[2077] "Gameplay data" refers to information such as player behavior, scores, and play history.

[2078] "Difficulty and reward settings" refers to the balance between the level of challenge in the game and the rewards obtained in return.

[2079] This invention relates to a game system that utilizes generative AI to provide endless fun. The system includes multiple algorithms and modules for generating character images, generating movements, synthesizing voices, generating dialogue and stories, detecting and debugging bugs, and optimizing game difficulty and reward settings.

[2080] Character image generation

[2081] First, the user selects the character's appearance and characteristics on the game's start screen. The specific steps are explained below.

[2082] The user selects the character's appearance and characteristics.

[2083] Example: User selects "Character with blue hair."

[2084] The terminal transmits the user's selection information to the server.

[2085] Example prompt: "Submit blue hair info for character generation."

[2086] The server generates a character image based on the received property information using a digital image generation means such as StableDiffusion.

[2087] The generated character image is sent back to the terminal, which displays this image to the user.

[2088] Example: A blue-haired character appears on the device screen.

[2089] Generating character movements

[2090] Next, a procedure for generating character movements based on the generated character image will be described.

[2091] The terminal uploads the generated character image to the server.

[2092] Example prompt: "Send an image of a blue-haired character to the server."

[2093] The server generates the motion data using an algorithm such as MotionDiffuse.

[2094] Example: The server generates walking motion data for a blue-haired character.

[2095] The generated movement data is sent to the terminal, which applies it to the character and displays it.

[2096] An animation of a blue-haired character walking appears on the device screen.

[2097] Speech synthesis

[2098] The procedure for synthesizing voice based on character lines is explained below.

[2099] The server generates dialogue data from a scenario or story.

[2100] Example: Generate dialogue data for "Hello"

[2101] The server generates the voice using a speech synthesis algorithm that uses LLM.

[2102] Example prompt: "Generate the greeting 'hello' aloud."

[2103] The generated audio data is sent to the terminal, which then plays it back in the game.

[2104] Example: A character in a game says "Hello."

[2105] Conversation and story generation

[2106] The steps for generating conversations between users and NPCs and game stories are explained below.

[2107] The user provides input to an NPC in the game.

[2108] Example: A user asks, "What's the next mission?"

[2109] The terminal transmits the context data to the server.

[2110] Example prompt: Send the question "What's your next mission?"

[2111] The server uses NLP models to generate dialogue and stories.

[2112] Example: Generate a description for the next mission

[2113] The generated data is sent to the terminal, which displays it to the user.

[2114] Example: An NPC in a game explains the details of the next mission.

[2115] Bug detection and debugging instructions

[2116] The procedure for detecting system malfunctions and generating debug instructions is described below.

[2117] When a user discovers a bug during the game, they report the problem to the server via their device.

[2118] Example prompt: "The character walks through the wall."

[2119] The server analyzes the reports and uses AI to detect bugs.

[2120] Example: Server detects bug that allows walking through walls

[2121] The server generates debug instructions and sends them to the development team.

[2122] Example: Generate specific debug instructions for fixing a bug

[2123] The development team makes corrections based on the debugging instructions and releases the corrected version.

[2124] Example: The modified game is provided to the user

[2125] Optimizing game difficulty and reward settings

[2126] The procedure for optimizing the game difficulty and reward settings is described below.

[2127] The device periodically transmits the player's progress and gameplay data to the server.

[2128] Example prompt: "Send data such as game completion time and success rate."

[2129] The server analyzes the received data and optimizes the balance between difficulty and reward.

[2130] Example: Adjusting difficulty based on player performance

[2131] The server sends the new difficulty and reward setting data back to the device.

[2132] Example: Sending optimized configuration data to the device

[2133] The device will apply the new configuration data to the game and reflect it to the player.

[2134] Example: The game restarts with a new difficulty level

[2135] By following the above procedure, the system of the present invention can generate and optimize a variety of game elements in real time, and provide users with a constantly fresh and engaging game experience.

[2136] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2137] Step 1:

[2138] The user selects the character's appearance and characteristics.

[2139] Input: User-selected character appearance and characteristics (e.g., a character with blue hair).

[2140] Specific operation: The user selects the character's attributes on the game's start screen.

[2141] Output: The selected character attribute information is generated on the terminal.

[2142] Step 2:

[2143] The terminal transmits the user's selection information to the server.

[2144] Input: Character selection information (attribute data for a blue-haired character).

[2145] Specific operation: The terminal transmits the selection information to the server via the network.

[2146] Output: The selection information sent to the server.

[2147] Step 3:

[2148] The server generates a character image based on the received property information using a digital image generation algorithm.

[2149] Input: Character attribute data (blue hair information).

[2150] Data processing or data calculation: Using a digital image generation algorithm such as StableDiffusion, a character image is generated based on the specified attribute information.

[2151] Specific operation: The server runs StableDiffusion and creates a character image.

[2152] Output: The generated character image.

[2153] Step 4:

[2154] The server transmits the generated character image to the terminal.

[2155] Input: Generated character image.

[2156] Specific operation: The server sends the generated image back to the terminal via the network.

[2157] Output: Character image sent to the device.

[2158] Step 5:

[2159] The terminal displays the received character image to the user.

[2160] Input: Character image sent from the server.

[2161] Specific operation: Display a character image on the device screen.

[2162] Output: The character image displayed to the user.

[2163] Step 6:

[2164] The terminal uploads the generated character image to the server.

[2165] Input: Character image stored on the device.

[2166] Specific operation: The device uploads the character image to the server via the network.

[2167] Output: Character image uploaded to the server.

[2168] Step 7:

[2169] The server generates movement data based on the character image.

[2170] Input: Uploaded character image.

[2171] Data processing or data calculation: Using algorithms such as MotionDiffuse to analyze character images and generate motion data.

[2172] Specific operation: The server executes MotionDiffuse and creates character movement data.

[2173] Output: Generated character motion data.

[2174] Step 8:

[2175] The server transmits the generated exercise data to the terminal.

[2176] Input: Generated movement data.

[2177] Specific operation: The server sends the exercise data to the terminal via the network.

[2178] Output: Exercise data sent to the device.

[2179] Step 9:

[2180] The device applies the received movement data to the character and displays it.

[2181] Input: Exercise data sent from the server.

[2182] Specific operation: The device applies the movement data to the character and displays it on the screen.

[2183] Output: A moving character displayed on the terminal.

[2184] Step 10:

[2185] The server synthesizes voice based on the dialogue data generated from the scenario.

[2186] Input: Scenario and dialogue data.

[2187] Data processing or data computation: Generate speech data from dialogue data using a natural language processing model.

[2188] Specific operation: The server executes a speech synthesis algorithm to convert the dialogue data into voice data.

[2189] Output: The generated audio data.

[2190] Step 11:

[2191] The server transmits the generated voice data to the terminal.

[2192] Input: The generated audio data.

[2193] Specific operation: The server sends the audio data to the terminal via the network.

[2194] Output: The audio data sent to the device.

[2195] Step 12:

[2196] The device then plays the received audio data as the character's lines.

[2197] Input: Audio data sent from the server.

[2198] Specific operation: The device applies the voice data to the character and plays it in the game.

[2199] Output: Character dialogue played in-game.

[2200] Step 13:

[2201] The context data entered by the user is transmitted from the terminal to the server.

[2202] Input: A question or request that the user types in the game.

[2203] Specific operation: The terminal sends user input to the server.

[2204] Output: The context data sent to the server.

[2205] Step 14:

[2206] The server generates conversations and stories based on the contextual data.

[2207] Input: Context data.

[2208] Data manipulation or data computation: Using natural language processing models to generate conversations or stories.

[2209] What it does: The server runs NLP models and creates conversations and stories.

[2210] Output: Generated conversation or story data.

[2211] Step 15:

[2212] The server sends the generated conversation and story data to the terminal.

[2213] Input: Generated conversation or story data.

[2214] Specific operation: The server sends data to the terminal via the network.

[2215] Output: Conversation or story data sent to your device.

[2216] Step 16:

[2217] The terminal displays the received conversation and story data to the user.

[2218] Input: Conversation or story data sent from the server.

[2219] Specific behavior: The device displays the data on the screen.

[2220] Output: The conversation or story that is displayed to the user.

[2221] Step 17:

[2222] If a user discovers a bug during the game, they report the problem to the server via their device.

[2223] Input: The description of the bug found by the user.

[2224] Specific operation: A user reports a bug to the server through a terminal.

[2225] Output: The bug report sent to the server.

[2226] Step 18:

[2227] The server analyzes the reported bug and uses AI to detect it.

[2228] Input: Bug report data.

[2229] Data manipulation or data manipulation: Using bug detection AI to analyze reports and detect bugs.

[2230] Specific operation: The server runs AI to identify the location and cause of the bug.

[2231] Output: Detected bug data.

[2232] Step 19:

[2233] The server generates debug instructions for the detected bugs and sends them to the development team.

[2234] Input: Detected bug data.

[2235] What happens: The server generates debug instructions and sends them to the development team.

[2236] Output: Debugging instructions sent to the development team.

[2237] Step 20:

[2238] The device periodically transmits the player's progress and gameplay data to the server.

[2239] Input: Player progress data, gameplay data.

[2240] Specific operation: The device sends progress and play data to the server.

[2241] Output: Progress data and play data sent to the server.

[2242] Step 21:

[2243] The server analyzes the received data and optimizes the balance between difficulty and reward.

[2244] Input: Progress data, play data.

[2245] Data processing or data arithmetic: Using data analysis algorithms to calculate the optimal balance between difficulty and reward.

[2246] Specific operation: The server runs the algorithm and adjusts the difficulty and reward.

[2247] Output: New difficulty and reward settings data.

[2248] Step 22:

[2249] The server sends the new difficulty and reward setting data back to the device.

[2250] Input: New difficulty and reward settings data.

[2251] Specific operation: The server sends the setting data to the terminal.

[2252] Output: The new configuration data sent to the terminal.

[2253] Step 23:

[2254] The device will apply the new configuration data to the game and reflect it to the player.

[2255] Input: New difficulty and reward settings data.

[2256] Specific operation: The device applies and reflects the new configuration data in the game.

[2257] Output: The game environment with the new settings.

[2258] (Application example 1)

[2259] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2260] It is necessary to make the virtual shopping experience more engaging and interactive, provide an entertainment-rich shopping experience that users can enjoy endlessly, and increase purchasing motivation by providing product recommendations and navigation in real time based on users' preferences.

[2261] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2262] In this invention, the server includes: [means for generating character images;] [means for generating movements based on the generated character images;] [means for synthesizing voice based on lines;] [means for generating conversations and stories;] [means for detecting bugs and generating debugging instructions;] [means for optimizing game difficulty and reward settings;] [means for recommending products based on user attribute information and displaying them as navigation characters;] [voice synthesis means for explaining products using the voice lines of the generated characters; and [means for generating conversations and dialogues based on interactions with the user. This makes it possible to provide users with an endlessly enjoyable virtual shopping experience and increase their desire to purchase.

[2263] A "character image" is a visual representation of a virtual person or object generated based on the user's attribute information.

[2264] "Movement data" is data on the actions and movements that a character appears to perform based on the generated character image.

[2265] "Speech synthesis" is the process of creating artificially generated speech based on text data.

[2266] "Conversations and stories" refer to dialogues and scenarios created based on user interaction.

[2267] "Bug detection" is the process of discovering errors or defects in software or systems.

[2268] "Debugging instructions" are specific steps or instructions for fixing a detected bug.

[2269] "Optimizing difficulty and reward settings" is the process of adjusting the appropriate challenge and reward levels depending on the progress of a game or experience.

[2270] "User attribute information" is data that includes user preferences, behavioral history, personal settings, etc.

[2271] "Product recommendation" is the process of suggesting highly relevant products based on a user's attribute information.

[2272] A "navigation character" is a virtual character that acts as a guide for users within a virtual store.

[2273] An "interaction" is an interaction between a user and a system.

[2274] This invention relates to a system that provides users with an endlessly enjoyable virtual shopping experience. The system uses a generative AI model and multiple algorithms and modules to generate character images, generate movements, synthesize voices, and generate conversations and stories. It also includes a function that recommends products based on the user's attribute information and displays them as navigation characters.

[2275] The server generates a character image upon receiving a request from the user. This uses StableDiffusion, an image generation algorithm based on the user's attribute information. The user selects the character's appearance and characteristics on their smartphone and sends their selection to the server. The server creates a character image based on the received attribute information and sends the generated image back to the device. The device displays this image to the user.

[2276] The server then uses algorithms such as MotionDiffuse to generate character movement based on the generated character image. The device uploads the character image to the server, which analyzes the image and generates motion data. The generated motion data is then sent to the device, which applies it to the character and displays it.

[2277] The server then generates voice based on the dialogue data. Dialogue data is generated from the scenario or story, and this is input into a speech synthesis algorithm using a large-scale language model (LLM). The generated voice data is sent to the device, which then plays it back to the user.

[2278] The server generates conversations and stories using NLP models. When the user interacts with the navigation character, the device sends the context data to the server. The server generates conversations between the user and the character and the ongoing story based on this context data and sends it to the device. The device then displays the generated data to the user.

[2279] Bug detection and debug instructions are performed by a bug detection AI on the server. When a user finds a bug in the system, they report the problem to the server via their terminal. The server analyzes the report and uses AI to detect bugs in the code. Debug instructions for the detected bug are generated and sent to the development team. The problem is resolved by releasing a patch to fix it.

[2280] Finally, the system periodically transmits user progress and interaction data to the server, which analyzes this data and configures the system to optimize the balance between difficulty and reward. The new configuration data is sent back to the device, which then applies it to the system and reflects it to the user.

[2281] For example, the following conversation might be generated by a generative AI model:

[2282] User: "What are your recent recommendations?"

[2283] AI: "My latest recommendation is a newly released smartwatch. It features a long battery life and comprehensive health tracking."

[2284] In this way, the system can provide users with an endlessly enjoyable virtual shopping experience and increase their desire to purchase.

[2285] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2286] Step 1:

[2287] The user selects the character's appearance and characteristics using a smartphone device.

[2288] Input: User selection information (attribute information)

[2289] Output: Sends user selection information to the server

[2290] Specific operation: The user selects the character's appearance and characteristics on the application interface and sends that information to the server.

[2291] Step 2:

[2292] The server generates a character image using StableDiffusion based on the attribute information received from the user.

[2293] Input: User attribute information

[2294] Output: Generated character image

[2295] Specific operation: The server uses the StableDiffusion algorithm to generate a character image from the user's attribute information.

[2296] Step 3:

[2297] The server returns the generated character image to the terminal.

[2298] Input: Generated character image

[2299] Output: Character image returned to the device

[2300] Specific operation: The server sends the generated character image to the terminal, and the terminal displays this image.

[2301] Step 4:

[2302] The device uploads the character image to the server, and the server generates the motion data using MotionDiffuse.

[2303] Input: Character image

[2304] Output: Generated motion data

[2305] Specific operation: The device uploads the character image to the server, and the server generates motion data using the MotionDiffuse algorithm.

[2306] Step 5:

[2307] The server transmits the generated motion data to the terminal.

[2308] Input: Generated motion data

[2309] Output: Movement data sent to the device

[2310] Specific operation: The server sends the generated movement data to the device, which then applies it to the character and displays it.

[2311] Step 6:

[2312] The server synthesizes voice based on the scenario or story.

[2313] Input: Scenario and story dialogue data

[2314] Output: Generated audio data

[2315] Specific operation: The server generates speech from dialogue data using a speech synthesis algorithm based on a large-scale language model (LLM) and sends it to the terminal.

[2316] Step 7:

[2317] The terminal reproduces the generated voice data and lets the user hear it.

[2318] Input: Generated audio data

[2319] Output: The audio played to the user

[2320] Specific operation: The device plays the audio data received from the server.

[2321] Step 8:

[2322] The user interacts with the navigation character and the terminal transmits the context data to the server.

[2323] Input: User interaction

[2324] Output: Context data sent to the server

[2325] Specific operation: The user interacts with the navigation character, and the content of the interaction is sent from the terminal to the server as context data.

[2326] Step 9:

[2327] The server uses NLP models to generate conversations and stories and send them to the device.

[2328] Input: Context data

[2329] Output: Generated conversations and story data

[2330] Specific operation: The server analyzes the context data using an NLP model, generates a conversation or story, and sends it to the device.

[2331] Step 10:

[2332] The terminal displays the generated conversations and stories to the user.

[2333] Input: Generated conversation or story data

[2334] Output: The conversation or story that is displayed to the user

[2335] Specific operation: The device displays the conversation and story data received from the server and shows it to the user.

[2336] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2337] This invention relates to a system that utilizes generative AI and an emotion engine to provide users with an endlessly enjoyable gaming experience. This system includes multiple algorithms and modules that generate character images and create movements, voices, dialogue, and stories. It also incorporates an emotion engine that recognizes the user's emotions and dynamically changes the game content.

[2338] First, the system generates a character image based on the attributes of the character selected by the user. The user selects the character's appearance and characteristics on the game's opening screen, and the device sends that selection to the server. The server uses an image generation algorithm (e.g., StableDiffusion) to create a character image based on the received attribute information, and sends the generated image back to the device. The device then displays this image to the user.

[2339] Next, the server generates character movement based on the generated character image. The device uploads the character image to the server, and the server analyzes the image and invokes an algorithm (such as MotionDiffuse) to generate motion data. The generated motion data is sent to the device, which applies it to the character and displays it.

[2340] For voice synthesis, the server generates voice based on dialogue data. Dialogue data is generated from the scenario or story, and this is input into the voice synthesis algorithm. The generated voice data is sent to the device, which plays it in the game.

[2341] Conversations and stories are generated by an NLP model (e.g., ChatGPT) on the server. When a user talks to an NPC character in the game, the device sends context data to the server. The server generates conversations and stories based on this context data and sends them to the device. The device then displays the generated data to the user.

[2342] The emotion engine has the ability to recognize emotions by analyzing the user's facial expression and voice data. The device collects data provided by the user through the camera and microphone and sends it to the server. The server inputs the received data into the emotion engine and recognizes the user's emotional state. The game content is dynamically changed based on the emotion recognition results. For example, if the user is surprised, an enemy character in the game can suddenly appear.

[2343] Bug detection and debug instructions are carried out by a bug detection AI on the server. When a user discovers a bug during the game, they report the problem to the server via their device. The server analyzes the report and uses AI to detect bugs in the code. Debug instructions for the detected bug are generated and sent to the development team. The problem is resolved by releasing a patch to fix the problem.

[2344] Finally, the server also optimizes the game's difficulty and reward settings. The device periodically sends the player's progress and gameplay data to the server. The server analyzes this data and sets the optimal balance between difficulty and reward. The new setting data is sent back to the device, which applies it to the game and provides it to the player.

[2345] For example, if a user selects red hair and armor as their costume during character creation, the server generates an image of the character wearing red hair and armor based on those attributes. The server then generates movements and sounds for the character, and the emotion engine recognizes the user's emotions and dynamically changes in-game events. For example, if the user looks tired, an event will occur in which a friendly NPC in the game offers a support item.

[2346] Through the above process, this invention enables efficient and creative game development, reducing development effort while providing players with a consistently new and engaging gaming experience. Furthermore, incorporating the user's emotional state can provide a more personalized gaming experience.

[2347] The processing flow will be explained below.

[2348] Character generation process

[2349] Step 1:

[2350] The user selects the character's attributes (e.g., hair color, clothing, gender) on the game's start screen.

[2351] Step 2:

[2352] The terminal transmits the selected attribute data to the server.

[2353] Step 3:

[2354] Based on the received attribute data, the server invokes an image generation algorithm (e.g., StableDiffusion) to generate a character image.

[2355] Step 4:

[2356] The server transmits the generated character image to the terminal.

[2357] Step 5:

[2358] The terminal displays the received character image to the user.

[2359] Motion generation processing

[2360] Step 1:

[2361] The terminal uploads the generated character image to the server.

[2362] Step 2:

[2363] The server calls an algorithm (e.g., MotionDiffuse) that generates motion data based on the received character image.

[2364] Step 3:

[2365] The server transmits the generated motion data to the terminal.

[2366] Step 4:

[2367] The terminal applies the received motion data to the character and displays the motion to the user.

[2368] Speech synthesis processing

[2369] Step 1:

[2370] The server generates dialogue data based on a scenario or story.

[2371] Step 2:

[2372] The server passes the dialogue data to a speech synthesis algorithm to generate speech data.

[2373] Step 3:

[2374] The server transmits the generated voice data to the terminal.

[2375] Step 4:

[2376] The device plays the received audio data in the game.

[2377] Conversation / story generation processing

[2378] Step 1:

[2379] The user talks to an NPC character in the game.

[2380] Step 2:

[2381] The device transmits user input and the current game state to the server.

[2382] Step 3:

[2383] The server invokes an NLP model (e.g., ChatGPT) based on the received context data to generate a conversation or story.

[2384] Step 4:

[2385] The server sends the generated conversations and stories to the device.

[2386] Step 5:

[2387] The terminal displays the received conversation data to the user.

[2388] Emotion Recognition Processing

[2389] Step 1:

[2390] The user provides facial expression data and voice data through a camera and microphone while playing the game.

[2391] Step 2:

[2392] The terminal transmits the collected facial expression data and voice data to the server.

[2393] Step 3:

[2394] The server inputs the received data into an emotion engine to recognize the user's emotion.

[2395] Step 4:

[2396] The server dynamically adjusts in-game events and conversations based on the emotion recognition results.

[2397] Step 5:

[2398] The device displays the adjusted events and conversations in the game and reflects them to the user.

[2399] Bug detection and debug instruction handling

[2400] Step 1:

[2401] A user discovers a bug in the game and reports the problem through a bug report form.

[2402] Step 2:

[2403] The terminal sends the report to the server.

[2404] Step 3:

[2405] The server analyzes the received report and calls a bug detection AI to detect bugs in the code.

[2406] Step 4:

[2407] The server generates debug instructions for the detected bugs and sends them to the development team.

[2408] Step 5:

[2409] The server applies the corrected code or patch to the game and notifies the device of the update.

[2410] Difficulty and reward setting optimization process

[2411] Step 1:

[2412] The device transmits the player's progress and gameplay data to the server.

[2413] Step 2:

[2414] Based on the received player data, the server invokes an algorithm to optimize difficulty and reward settings and analyzes the data.

[2415] Step 3:

[2416] Based on the analysis results, the server generates new difficulty and reward settings and sends them to the device.

[2417] Step 4:

[2418] The device applies the received new setting data to the game and reflects it to the player.

[2419] The above is a specific program processing flow for implementing the invention in combination with an emotion engine. This system provides a personalized gaming experience that incorporates the user's emotions.

[2420] Example 2

[2421] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2422] In conventional game systems, it has been difficult to dynamically adapt game progression and generate interactive characters that reflect the user's individual emotional state. Furthermore, bug detection and game difficulty adjustment are performed manually, resulting in low efficiency and limited improvements to the user experience. New algorithms and technologies are needed to address these challenges.

[2423] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2424] In this invention, the server includes means for generating character images, means for generating movements based on the generated character images, means for synthesizing voice based on dialogue, means for generating conversations and stories, means for recognizing the user's emotional state and dynamically changing game content, means for detecting bugs and generating debugging instructions, and means for optimizing game difficulty and reward settings. This enables a dynamic and personalized game experience that matches the user's emotions, and makes it possible to provide a high-quality game service by automatically detecting bugs and automatically adjusting game balance.

[2425] "Character image" is graphic data of a visual character used in the game, generated based on attribute information selected by the user.

[2426] "Movement data" is a series of data generated based on a character image to represent the movement of the character.

[2427] "Dialogue data" is text data of words spoken by characters that is generated based on the scenario or story within the game.

[2428] A "voice synthesis algorithm" is a calculation procedure or program for generating voice data based on dialogue data.

[2429] A "conversation generation algorithm" is a computational procedure or program for generating dialogue and stories between characters in a game.

[2430] An "emotion engine" is a group of algorithms and programs that analyze a user's facial expression data and voice data to recognize their emotional state.

[2431] "Bug detection AI" is an artificial intelligence that automatically detects glitches (bugs) that occur in games and generates instructions for debugging.

[2432] "Difficulty and reward optimization" is the process of adjusting the balance of a game's difficulty and available rewards based on player progress and gameplay data.

[2433] An "image generation algorithm" is a calculation procedure or program for generating a new character image based on attribute information selected by the user.

[2434] "Dynamic change" refers to changing the content and progress of the game in real time according to the user's emotional state and the game situation.

[2435] MODE FOR CARRYING OUT THE INVENTION

[2436] This invention is a system that utilizes generative AI and an emotion engine to provide users with an endlessly enjoyable gaming experience. To implement the invention, specific hardware and software are required to process and calculate various data.

[2437] Hardware and software used

[2438] This system operates in a network environment that includes user devices (PCs, smartphones, etc.) and servers. Specific software includes image generation algorithms (e.g., StableDiffusion), motion generation algorithms (e.g., MotionDiffuse), speech synthesis algorithms, natural language processing models (e.g., ChatGPT), and emotion engines.

[2439] System Operation Overview

[2440] 1. Character image generation

[2441] The user selects the character's appearance and characteristics on the game start screen. The device sends the selection to the server, which uses an image generation algorithm to generate a character image based on the received attribute information. This image is sent back to the device and displayed.

[2442] 2. Character Movement Generation

[2443] The device sends the generated character image to the server, which uses a motion generation algorithm to generate character movements from the image, and the generated motion data is sent back to the device and applied to the character.

[2444] 3. Speech generation

[2445] Based on the dialogue data generated according to the scenario and story, the server uses a speech synthesis algorithm to generate voice data, which is then sent to the device and played back in the game.

[2446] 4. Conversation and story generation

[2447] When a user talks to an NPC character in a game, the device sends that context data to the server, which uses a natural language processing model to generate a dialogue or story, which is then sent to the device and displayed to the user.

[2448] 5. Emotion Recognition and Dynamic Change of Game Content

[2449] The device collects the user's facial expression and voice data through a camera and microphone and sends it to the server. The server then uses an emotion engine to recognize the user's emotional state. Based on this recognition, in-game events and character behavior are dynamically changed.

[2450] 6. Bug Detection and Debugging Instructions

[2451] If a user discovers a bug during the game, they report the problem to the server via their device. The server uses a bug detection AI to analyze the report and detect bugs in the code. Debug instructions are generated and sent to the development team.

[2452] 7. Optimizing difficulty and reward settings

[2453] The device periodically sends the player's progress and gameplay data to the server. The server analyzes this data and configures the game to optimize the balance between difficulty and rewards. The new configuration data is sent back to the device and applied to the game.

[2454] Specific examples

[2455] For example, if a user selects red hair and armor as their outfit, the server generates a character image wearing red hair and armor based on those attributes. The server then generates movements and sounds for the character, and the emotion engine recognizes the user's emotions and dynamically changes in-game events. For example, if the user shows a tired expression, an event occurs in which a friendly character in the game provides a support item.

[2456] Prompt Sentence Examples

[2457] "Generate an image of a character with red hair and armor."

[2458] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2459] Step 1: User character attribute selection

[2460] Description: The user selects the character's appearance and characteristics (hair color, outfit, etc.) on the game's start screen.

[2461] Input: User-selected character attribute information (e.g., hair color "red", costume "armor").

[2462] Output: Data that sends selected attribute information from the device to the server.

[2463] Specific operation: The user selects "red hair" and "armor," and the device sends that information to the server.

[2464] Step 2: Generate character images

[2465] Description: The server generates a character image using an image generation algorithm based on the received attribute information.

[2466] Input: Character attribute information selected by the user.

[2467] Output: Send the generated character image data to the terminal.

[2468] What it does: The server uses the StableDiffusion algorithm to generate an image of a character with red hair and armor, and sends it back to the device.

[2469] Step 3: Character Movement Generation

[2470] Description: The device uploads the generated character image to the server, and the server generates the character's movements using a movement generation algorithm.

[2471] Input: Generated character image.

[2472] Output: Send the generated motion data to the device.

[2473] How it works: The server uses the MotionDiffuse algorithm to generate character motion data and sends it to the device, which then applies this motion to the character.

[2474] Step 4: Generate audio

[2475] Description: The server generates dialogue data from a scenario or story and generates speech using a speech synthesis algorithm.

[2476] Input: Dialogue data (text format).

[2477] Output: Sends the generated audio data to the device.

[2478] What happens: The server uses a speech synthesis algorithm to generate a voice for the phrase "Hello" and sends it to the device, which then plays it back in the game.

[2479] Step 5: Generate conversations and stories

[2480] Description: When a user talks to an NPC character in a game, the device sends that context data to the server, which then generates the dialogue and story.

[2481] Input: Context data (user-NPC interactions, current scenario, etc.).

[2482] Output: Send the generated conversation data and story data to the device.

[2483] How it works: The server uses the ChatGPT algorithm to generate a conversation saying, "Go down this path and you'll find the treasure," and sends it to the device, which then displays it to the NPC in the game.

[2484] Step 6: Emotion recognition and dynamic change of game content

[2485] Description: The device collects the user's facial expression and voice data and sends it to the server. The server uses an emotion engine to recognize the user's emotional state and dynamically change the game content.

[2486] Input: facial expression data, audio data.

[2487] Output: Dynamically modified game content based on emotion recognition results.

[2488] Specific behavior: The server recognizes that the user is surprised and triggers an event in the game in which an enemy character suddenly appears.

[2489] Step 7: Bug detection and debugging instructions

[2490] Description: When a user discovers a bug during the game, they report the problem to the server through their terminal. The server uses a bug detection AI to detect bugs in the code and generate debug instructions.

[2491] Input: Bug report data.

[2492] Output: Debug instructions, fix patch.

[2493] What happens: The server analyzes the user's report that "the character can go through walls," detects the bug, sends a debug instruction to the development team, and releases a fix patch.

[2494] Step 8: Optimize difficulty and reward settings

[2495] Description: The device sends player progress and gameplay data to the server, which analyzes this data and configures the game to optimize the balance between difficulty and rewards.

[2496] Input: Player progress, gameplay data.

[2497] Output: Optimized difficulty and reward settings.

[2498] What it does: The server analyzes the player's high score, sets more difficult levels and rewards, and sends the new configuration data back to the device to apply it to the game.

[2499] (Application example 2)

[2500] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2501] Conventional security systems operate based on fixed settings, making it difficult to respond flexibly to user emotions and situations. Furthermore, there was no technology that could dynamically change security responses based on emotions in real time. As a result, even if a user is feeling anxious or nervous, it is not possible to provide appropriate safety measures, which could lead to a decline in the quality of security.

[2502] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2503] In this invention, the server includes: [means for generating character images;] [means for generating movements based on the generated character images;] [means for synthesizing voice based on dialogue;] [means for generating conversations and stories;] [means for detecting bugs and generating debugging instructions;] [means for optimizing game difficulty and reward settings;] [means for recognizing user emotions and dynamically changing system settings based on those emotions; and [means for dynamically generating emotion-based security measures using a generative AI model.] This enables appropriate security responses that are in line with user emotions to be provided in real time, improving the quality of security.

[2504] "Means for generating a character image" refers to an algorithm or module that creates an image of a character based on attribute information selected by a user.

[2505] "Means for generating movement based on the generated character image" refers to an algorithm or module that generates movement data from the generated character image.

[2506] "Means for synthesizing voice based on dialogue" refers to a voice synthesis algorithm or module that converts dialogue data from a scenario or story into voice.

[2507] "Means for generating conversations and stories" refers to natural language processing models and algorithms for generating conversations and stories with users.

[2508] "Means for detecting bugs and generating debug instructions" refers to algorithms or modules for analyzing problems reported by users and generating instructions required for debugging.

[2509] "Means for optimizing game difficulty and reward settings" refers to algorithms or modules that analyze users' gameplay data and optimally adjust the balance between difficulty and rewards.

[2510] "Means for recognizing user emotions and dynamically changing system settings based on those emotions" refers to algorithms or modules that analyze emotions from the user's facial expressions and voice and dynamically change system settings based on the results.

[2511] "Means for dynamically generating emotion-based security measures using a generative AI model" refers to an AI model that receives emotion data as input and generates security measures appropriate to those emotions.

[2512] This invention is a system that uses a generative AI model and an emotion engine to provide a dynamic gaming experience and security measures based on user emotions. The system includes multiple algorithms and modules and runs on devices such as smartphones and smart glasses.

[2513] The hardware required to implement the invention is a device equipped with a camera and microphone (e.g., a smartphone or smart glasses). The software uses Python and Django (server-side), an emotion API (e.g., Microsoft Azure Emotion API), and a natural language processing model (e.g., ChatGPT).

[2514] 1. Character image generation

[2515] The user uses the device to select the character's appearance and characteristics on the game's opening screen. This selection information is sent from the device to the server. The server generates a character image using an image generation algorithm (e.g., Stable Diffusion) based on the received attribute information, and sends the generated image back to the device. The device then displays this image to the user.

[2516] 2. Generating Character Movements

[2517] The generated character image is uploaded from the device to the server, which analyzes the image and invokes an algorithm (e.g., MotionDiffuse) to generate motion data. The generated motion data is sent to the device, which applies it to the character and displays it to the user.

[2518] 3. Speech Synthesis

[2519] The server generates dialogue data from the scenario and story, and synthesizes voice based on that. The dialogue data is input into a voice synthesis algorithm, and the generated voice data is sent to the device, which then plays it in the game.

[2520] 4. Conversation and story generation

[2521] When a user talks to an NPC character in a game, the device sends context data to the server, which generates a conversation or story based on the context data and sends it to the device, which then displays the generated data to the user.

[2522] 5. User Emotion Recognition and Dynamic System Changes

[2523] The device collects facial expression and voice data provided by the user through the camera and microphone and sends it to the server. The server inputs this data into an emotion engine (e.g., Microsoft Azure Emotion API) to recognize the user's emotional state. System settings are dynamically changed based on the emotion recognition results. For example, if the user feels anxious, security measures are strengthened.

[2524] 6. Emotion data collection using cameras and microphones

[2525] The camera and microphone are used to collect the user's facial expression and voice data. Specifically, the camera captures the user's facial expressions and the microphone records their voice. This data is sent to the emotion API in real time, and the user's emotional state is analyzed.

[2526] 7. Emotion-based security measures

[2527] Based on the emotion data, the server uses a generative AI model (e.g., ChatGPT) to dynamically generate security measures appropriate to the user's emotional state. For example, if the user feels anxious, the generative AI model will suggest security measures such as issuing a warning notification or automatically locking the smart lock.

[2528] Examples of examples and prompts:

[2529] If the user feels anxious at night, the data from the camera and microphone will be analyzed to generate the result "anxious." Based on this, the generative AI will increase the sensitivity of the window and door sensors and automatically generate instructions to lock all the doors in the house.

[2530] Text format

[2531] User emotion: anxious. Suggest measures to enhance home security during the user's anxious state.

[2532] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2533] Step 1:

[2534] The user selects the character's appearance and characteristics.

[2535] Input: Appearance and trait selection data.

[2536] How it works: The user selects the character's appearance and characteristics on the game's opening screen and enters them into the device.

[2537] Output: User-selected attribute information.

[2538] Step 2:

[2539] The terminal transmits the user-selected attribute information to the server.

[2540] Input: User-selected attribute information.

[2541] Operation: The terminal transmits the selected attribute information to the server.

[2542] Output: Attribute information received by the server.

[2543] Step 3:

[2544] The server generates a character image based on the attribute information.

[2545] Input: Attribute information.

[2546] Operation: The server inputs the attribute information into an image generation algorithm (e.g., StableDiffusion) to generate a character image.

[2547] Output: Character image.

[2548] Step 4:

[2549] The server returns the generated character image to t...

Claims

1. A means for generating a character image; A means for generating a movement based on the generated character image; A means of synthesizing voice based on dialogue, A means of generating conversations and stories; means for detecting bugs and generating debug instructions; A means to optimize game difficulty and reward settings; A system including:

2. 2. The system according to claim 1, wherein the means for generating the character image uses an image generation algorithm based on attribute information.

3. 2. The system according to claim 1, wherein the means for generating the movement uses an algorithm for generating movement data based on a character image.

Images