system

Abstract

Provide a system. An image capturing means; image analysis means for identifying captured images; A generating means for generating a monster based on the identified object; a display means for displaying the generated monster; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Previous monster generation games lacked the ability to automatically generate unique monsters from images provided by the user, limiting the user's creativity. Furthermore, monster generation required individual settings, making the process complicated. Furthermore, the variations of monsters generated were limited, making it difficult for users to constantly enjoy fresh experiences. [Means for solving the problem]

[0005] The present invention provides a system that includes an image capturing means, an image analysis means for identifying the captured image, a generation means for generating a monster based on the identified object, and a display means for displaying the generated monster. Furthermore, by including a status determination means for determining the status and abilities of the monster based on the characteristics of the identified object, it becomes possible to automatically generate a unique monster using any object photographed by a user. Furthermore, by including a gameplay means for providing adventure or battle using the monster depending on the mode selected by the user, it is possible to provide a diverse gaming experience using the generated monster. This makes it possible to realize a system that utilizes the user's creativity, is simple to operate, and always provides a fresh experience.

[0006] The "image capturing means" is a device or function that allows a user to capture an image of an object, and acquires image data of the captured image.

[0007] "Image analysis means" refers to a device or function for analyzing captured image data, identifying objects in the image, and extracting their features.

[0008] "Generation means" is a device or function for automatically generating a unique monster based on the characteristics of an identified object.

[0009] "Display means" refers to a device or function for visually displaying the appearance and status information of the generated monster to the user.

[0010] The "status determining means" is a device or function for determining the status (HP, attack power, defense power, etc.) and special abilities of the monster to be generated based on the characteristic information of the identified object.

[0011] "Gameplay means" refers to a device or function that provides an interactive gaming experience for users to engage in adventures and battles using the monsters they have created.

[0012] "User" refers to the player or end user who operates this system. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] The system for implementing the present invention allows a user to take a photograph of an object through a smartphone application and generate an original monster based on the photograph. The system uses the following means.

[0035] System Configuration

[0036] The system includes (1) an image capturing means, (2) an image analyzing means, (3) a generating means, (4) a display means, (5) a status determining means, and (6) a game playing means.

[0037] Program processing

[0038] 1. Image capture method

[0039] The user launches the smartphone app and uses the app's camera to take a photo of any object, such as stationery, furniture, or an illustration.

[0040] 2. Image analysis methods

[0041] The device sends the captured image to a server, which uses computer vision technology to analyze the received image and identify the object in the image. For example, if the captured object is a pen, it extracts its shape, color, texture, and other characteristics.

[0042] 3. Generation means

[0043] The server runs a monster generation algorithm based on the characteristic information of the identified object. Based on the characteristics of the object, the monster's appearance, status, and special abilities are automatically generated. For example, if a pen is photographed and identified, a monster with a long, slender body is generated and configured with the ability to write.

[0044] 4. Status Determination Method

[0045] The server determines the stats (HP, attack power, defense power, etc.) of the monsters that are generated. This reflects the size, color, material, and other characteristics of the object. For example, a red pen might be set to have a high attack power.

[0046] 5. Display means

[0047] The terminal receives the monster data sent from the server and visually displays it to the user, allowing the user to check the monster's appearance and status information.

[0048] 6. Gameplay Methods

[0049] Users can use the monsters they create to participate in in-game adventures and battle modes. In single-player mode, users control monsters to progress through stages in the game. In multiplayer mode, users can battle against other users.

[0050] Specific examples

[0051] For example, suppose a user takes a photo of a pen on their desk with their smartphone. The device sends the image to a server, which analyzes the image and identifies the object as a "pen." The server then generates a monster with a long, thin shape based on the pen, and determines that this monster has a special ability called "ink attack."

[0052] The server sets the monster's status (e.g., HP 100, attack power 50, defense power 30) and sends the data to the device. The device displays the monster to the user, allowing the user to enjoy adventures and battles using the monster.

[0053] This system allows users to use everyday objects as monsters in the game, providing a constantly new experience.

[0054] The processing flow will be explained below.

[0055] Step 1:

[0056] The user starts the smartphone app and uses the camera function to take a picture of any object, such as a pen, stuffed animal, or piece of furniture.

[0057] Step 2:

[0058] The device acquires the captured image data and sends the image data and the user's identification information (such as the user ID) to the server.

[0059] Step 3:

[0060] The server analyzes the image data received from the device. Specifically, it uses computer vision technology to identify objects in the image and extract their features (shape, color, texture, etc.). For example, if a pen is photographed, the shape and color of the pen are identified.

[0061] Step 4:

[0062] The server inputs characteristic information into a monster generation algorithm based on the object identification results. The server then automatically generates the monster's appearance, status, and special abilities based on the input characteristic information. For example, based on the characteristics of a pen, it generates a long, thin monster with the ability to write.

[0063] Step 5:

[0064] The server determines the stats (HP, attack power, defense power, etc.) of the generated monster. The stats are reflected in the characteristics of the object, such as its color and shape. For example, a red pen might have a high attack power.

[0065] Step 6:

[0066] The server sends the generated monster data (image of appearance, status, special abilities, etc.) to the terminal.

[0067] Step 7:

[0068] The device analyzes the monster data received from the server and visually displays it to the user, who can then check detailed information about the monster (appearance, status, special abilities, etc.) on their smartphone screen.

[0069] Step 8:

[0070] The user selects a game mode within the app, for example, adventure mode or battle mode.

[0071] Step 9:

[0072] The terminal requests the server for the necessary data depending on the selected game mode. For example, if battle mode is selected, the terminal requests information about the opponent's monsters.

[0073] Step 10:

[0074] The device displays the battle scene based on the battle data received from the server, and the user controls their own monster to fight.

[0075] Step 11:

[0076] The server calculates the results of the battle and sends the results (win / loss, experience points and items gained, etc.) to the device.

[0077] Step 12:

[0078] The terminal displays the battle results received from the server to the user, allowing the user to check, for example, whether they won or lost, the items they acquired, and the experience points they gained.

[0079] Through this series of steps, users can create original monsters based on everyday objects they photograph, and then enjoy adventures and battles using those monsters.

[0080] Example 1

[0081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0082] In conventional monster generation games, the monsters provided in the game are fixed, which means that the game lacks freshness and originality for users. Furthermore, there is a limitation in that users cannot customize the game experience using everyday objects. This makes it difficult to maintain interest in the game and limits the user experience.

[0083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0084] In this invention, the server includes an image analysis means, a generation means, and a status determination means. This allows users to generate unique monsters using images they have taken, and to adjust the monster's status and special abilities. This allows users to enjoy a diverse and new gaming experience based on familiar objects.

[0085] The "image capturing means" is a means for a user to capture an image of an object using the camera function within a smartphone application.

[0086] A "terminal" is a device that transmits captured image data to a server via a communication line.

[0087] The "image analysis means" is a means for analyzing the image received by the server using computer vision technology and extracting feature information such as the shape, color, and texture of objects in the image.

[0088] The "generation means" is a means for generating a monster using a generation AI model based on the extracted feature information.

[0089] The "display means" is a means for transmitting the generated monster to the terminal and visually displaying the monster on the terminal.

[0090] The "status determining means" is a means for determining the status and special abilities of a monster based on the characteristics of the identified object.

[0091] "Gameplay means" refers to the means by which a user can engage in adventures and battles using the monsters they have created.

[0092] A system for implementing this invention allows a user to photograph an object using a smartphone application and then generate a unique monster based on that object. The system includes an image capturing unit, a terminal, an image analysis unit, a generation unit, a status determination unit, a display unit, and a gameplay unit. The specific operation and technologies used are described below.

[0093] The user launches the smartphone application "Monster Creator" and uses the app's camera function to take a photo of any object. This object can be anything, such as stationery, furniture, or an illustration. The device then sends the captured image data to a cloud server. This communication uses an internet connection.

[0094] The server uses computer vision technology to analyze the received images. This analysis utilizes image processing libraries and models, such as the open-source Tensorflow®. The server identifies objects in the image and extracts feature information such as their shape, color, and texture. For example, if the captured image shows a pen, the pen's shape, color, and texture are analyzed.

[0095] Next, the server uses a generative AI model to generate a unique monster based on the extracted feature information. This algorithm is based on OpenAI's GPT model and determines the monster's appearance, status, and special abilities based on the object's characteristics. For example, if a pen is photographed, a long, thin monster is generated with a special ability called ink attack.

[0096] The status of the generated monsters (HP, attack power, defense power, etc.) is also determined on the server. This process is affected by the color and material of the object, so for example, a red pen has high attack power, a blue pen has high defense power, and so on.

[0097] The server then sends data about the generated monster to the device. The device uses the received data to visually display the monster within the app. This display uses real-time 3D display technology such as the Unity engine. Here, users can check the appearance and status of the generated monster.

[0098] Finally, users can use the monsters they create to take part in in-game adventures and battle modes. In single-player mode, they can strengthen their monsters by progressing through stages, and in multiplayer mode, they can enjoy battles with other users.

[0099] Specific examples

[0100] For example, a user takes a picture of a blue pen on their desk. In this case, the device sends the image data to the server, which analyzes the image and identifies the object as a pen. Based on the characteristics of the pen, the server generates a monster with a long, thin shape and the ability to perform ink attacks. The monster's stats are set to HP 100, attack power 50, and defense power 30. The device displays the monster in 3D based on the received data, and the user can use that monster to progress through the game.

[0101] Prompt Sentence Examples

[0102] "Please take a photo of the blue pen on your desk. The server will analyze the image and create a special monster based on the pen's characteristics. This monster will have ink attack abilities and its stats will be HP 100, Attack Power 50, and Defense Power 30."

[0103] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0104] Step 1:

[0105] The user starts the smartphone application "Monster Creator" and takes a picture of the object using the camera function in the application. The input is the image data taken by the user. For example, let's say the user takes a picture of a blue pen on a desk.

[0106] Step 2:

[0107] The device sends the captured image data to the cloud server via a communication line. The input is the image data obtained in step 1, which is sent to the cloud server via an Internet connection. The output is the image data received by the cloud server.

[0108] Step 3:

[0109] The server analyzes the received image data using computer vision technology. For the analysis, it uses image processing libraries such as TensorFlow to identify and extract the shape, color, and texture of objects from the input image. The input is the image data sent from the device, and the output is the extracted object feature information. Specifically, the shape, color, and texture of the pen are analyzed.

[0110] Step 4:

[0111] The server generates monsters using a generative AI model based on the extracted feature information. This generation procedure uses an algorithm based on OpenAI's GPT model. The input is the feature information extracted in step 3, and the output is the data of the generated monster. For example, based on the pen, a long, thin monster is generated and given a special ability called an ink attack.

[0112] Step 5:

[0113] The server determines the status of the generated monster (HP, attack power, defense power, etc.). The input is the basic data of the generated monster, and the shape and color of the object are reflected in the status determination procedure. For example, a blue pen will be set as having high defense power. The output is the final status of the monster.

[0114] Step 6:

[0115] The server sends the data of the generated monster to the terminal. The input is the monster data determined in step 5, and this data is sent back to the terminal via the communication line. The output is the monster data received by the terminal.

[0116] Step 7:

[0117] The device visually displays the monster within the app based on the received monster data. Here, 3D display is performed in real time using the Unity engine. The input is the monster data received from the server, and the output is the monster visually displayed for the user. The user can check the monster's appearance and status within the app.

[0118] Step 8:

[0119] Users control the generated monsters to participate in the game's adventures and battle modes. The input is the user's actions and the monster's status data, and the output is the user's progress in the game. In single-player mode, users can strengthen their monsters, and in multiplayer mode, they can battle other users.

[0120] (Application example 1)

[0121] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0122] In modern society, an increasing number of people are seeking healthy meals and new cooking ideas. However, there are limited ways to easily access recipes based on everyday ingredients and information about the health benefits of each ingredient. This makes it difficult to easily plan meals that suit one's health condition and preferences.

[0123] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0124] In this invention, the server includes an image capturing means, an image analysis means, a generating means for generating a recipe and a healthy report based on the identified object, and a display means for displaying the generated recipe and healthy report, thereby enabling a user to easily capture an image of a food using a smartphone and instantly obtain healthy recipes and health information based on that food.

[0125] The "image capturing means" is a means for a user to capture an image of an object using a smartphone application.

[0126] The "image analysis means" is a means for analyzing a captured image and identifying an object in the image.

[0127] The "generation means" is a means for generating related recipes and healthy reports based on the identified object information.

[0128] The "display means" is a means for visually presenting the generated recipe and healthy report to the user.

[0129] The "status determining means" is a means for determining detailed recipe and health information based on characteristic information of the identified object.

[0130] The "service means" is a means for executing a service to provide recipes and healthy reports in response to a user's selection.

[0131] This invention allows a user to take a picture of a food using a smartphone application, and generates an original recipe and a healthy report based on the food. The system includes the following means.

[0132] 1. Use of Hardware and Software

[0133] Camera (hardware): Used to capture images.

[0134] OpenCV (software): A library for image processing.

[0135] Image recognition model (software): A model for analyzing objects in an image.

[0136] Python runtime environment (software): The infrastructure for running programs.

[0137] 2. Contents of data processing and data calculation

[0138] The user launches the smartphone application and takes a picture of the food with the camera.

[0139] The device processes the captured image using OpenCV and identifies the object using an image recognition model. The feature information of the identified object is then sent to the server.

[0140] The server operates algorithms to generate recipes and healthy reports based on the identified object information.

[0141] The terminal receives the recipe and healthy report sent from the server and displays them to the user.

[0142] 3. Specific Examples

[0143] User input: The user takes a picture of a tomato with their smartphone.

[0144] Output: The following information will be generated:

[0145] Recommended recipe: Tomato and basil pasta

[0146] Health Report: Rich in Vitamin C and has antioxidant properties.

[0147] 4. Examples of prompts

[0148] Prompt: "Extract features from the captured image and generate appropriate cooking recipes and health reports based on those features."

[0149] This system allows users to quickly and easily obtain health information about everyday ingredients and new recipe suggestions, helping them improve the quality of their diet.

[0150] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0151] Step 1:

[0152] A user starts the smartphone application and takes a picture of food with the camera. The user's input is the image of the food. This image data is input.

[0153] Step 2:

[0154] The device processes the captured image using OpenCV and identifies the object using an image recognition model. The input is the image captured by the user. Specifically, the image is preprocessed (resized, noise removed, etc.), and then the image recognition model is applied to identify the food. The output is feature information of the identified food.

[0155] Step 3:

[0156] The device sends the characteristic information of the identified food to the server. The input is the characteristic information of the identified food, and the output is a data packet containing that information. Specifically, the device's data transmission API is called to send the data to the server.

[0157] Step 4:

[0158] The server uses a means for generating recipes and health reports based on the received food characteristic information. The input is the food characteristic information sent from the terminal. Specifically, the server retrieves the corresponding recipe information from the database, analyzes health-related statistical data, and generates a health report. The output is the generated recipe and health report data.

[0159] Step 5:

[0160] The server sends the generated recipe and healthy report data to the terminal. The input is the generated recipe and healthy report data, and the output is a data packet. Specifically, the server calls the data transmission API to send the data to the terminal.

[0161] Step 6:

[0162] The terminal displays the recipe and healthy report data received from the server. The input is the recipe and healthy report data sent from the server. This data is visualized and displayed on the display in a format that is easy for the user to view. The output is visual information that the user can confirm.

[0163] This process flow allows users to easily take a picture of a food item using their smartphone and instantly obtain healthy recipes and health information based on that food.

[0164] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0165] A system embodying the present invention allows a user to photograph an object through a smartphone application and generate an original monster based on that object. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to generate monsters and provide a gaming experience that reflects the user's emotions. This system uses the following means:

[0166] System Configuration

[0167] The system includes (1) an image capturing means, (2) an image analyzing means, (3) a generating means, (4) a displaying means, (5) a status determining means, (6) a game playing means, and (7) an emotion engine.

[0168] Program processing

[0169] 1. Image capture method

[0170] The user starts the smartphone app and uses the camera function to take a picture of any object, such as a pen, stuffed animal, or piece of furniture.

[0171] 2. Image analysis methods

[0172] The device acquires the captured image data and sends the image data and the user's identification information (such as the user ID) to the server.

[0173] 3. Image Classification and Feature Extraction

[0174] The server analyzes the image data received from the device. Specifically, it uses computer vision technology to identify objects in the image and extract their features (shape, color, texture, etc.). For example, if a pen is photographed, the shape and color of the pen are identified.

[0175] 4. Monster Generation

[0176] The server inputs characteristic information into a monster generation algorithm based on the object identification results. The server then automatically generates the monster's appearance, status, and special abilities based on the input characteristic information. For example, based on the characteristics of a pen, it generates a long, thin monster with the ability to write.

[0177] 5. Emotion Recognition and Affect

[0178] The device's built-in emotion engine analyzes the user's facial expressions and voice in real time to identify the user's emotions (joy, sadness, anger, etc.).

[0179] 6. Reflecting monster emotions

[0180] The server receives emotional data from the emotion engine and changes the monster's appearance and status according to the emotion. For example, if the user is happy, the monster will be set to have bright colors and high attack power.

[0181] 7. Status check and display

[0182] The device receives the monster data sent from the server and visually displays it to the user, who can then check detailed information about the monster (appearance, status, special abilities, etc.) on their smartphone screen.

[0183] 8. Start playing the game

[0184] The user selects a game mode within the app, for example, adventure mode or battle mode.

[0185] 9. Data Acquisition Depending on the Mode

[0186] The terminal requests the server for the necessary data depending on the selected game mode. For example, if battle mode is selected, the terminal requests information about the opponent's monsters.

[0187] 10. Displaying battle scenes

[0188] The device displays the battle scene based on the battle data received from the server, and the user controls their own monster to fight.

[0189] 11. Calculation and notification of battle results

[0190] The server calculates the results of the battle and sends the results (win / loss, experience points and items gained, etc.) to the device.

[0191] 12. Display of battle results

[0192] The terminal displays the battle results received from the server to the user, allowing the user to check, for example, whether they won or lost, the items they acquired, and the experience points they gained.

[0193] Specific examples

[0194] For example, a user takes a photo of a pen on their desk with their smartphone. The device sends the image to a server, which analyzes the image and identifies the object as a "pen." The server then generates a monster with a long, thin shape based on the pen, and determines that the monster has special abilities such as "ink attack."

[0195] At the same time, the emotion engine recognizes that the user is smiling and sends that data to the server, which uses this information to make the monsters it generates more vibrant in color and increase their attack power.

[0196] The server sets the monster's status (for example, HP 100, attack power 70, defense power 30) and sends the data to the device. The device displays the monster to the user, and the user can enjoy adventures and battles using the monster.

[0197] This system allows users to use everyday objects as monsters in the game, and their own emotional state will affect their gaming experience, providing a more personalized experience.

[0198] The processing flow will be explained below.

[0199] Step 1:

[0200] The user starts the smartphone app and uses the camera function to take a picture of any object, such as a pen, stuffed animal, or piece of furniture.

[0201] Step 2:

[0202] The device acquires the captured image data and sends the image data and the user's identification information (such as the user ID) to the server.

[0203] Step 3:

[0204] The server analyzes the image data received from the device. Specifically, it uses computer vision technology to identify objects in the image and extract their features (shape, color, texture, etc.). For example, if a pen is photographed, the shape and color of the pen are identified.

[0205] Step 4:

[0206] The server inputs characteristic information into a monster generation algorithm based on the object identification results. The server then automatically generates the monster's appearance, status, and special abilities based on the input characteristic information. For example, based on the characteristics of a pen, it generates a long, thin monster with the ability to write.

[0207] Step 5:

[0208] The device's built-in emotion engine analyzes the user's facial expressions and voice in real time to identify the user's emotions (joy, sadness, anger, etc.).

[0209] Step 6:

[0210] The server receives emotional data from the emotion engine and changes the monster's appearance and status according to the emotion. For example, if the user is happy, the monster will be set to have bright colors and high attack power.

[0211] Step 7:

[0212] The server determines the stats (HP, attack power, defense power, etc.) of the generated monster. The stats are reflected in the characteristics of the object, such as its color and shape. For example, a red pen might have a high attack power.

[0213] Step 8:

[0214] The server sends the generated monster data (image of appearance, status, special abilities, etc.) to the terminal.

[0215] Step 9:

[0216] The device analyzes the monster data received from the server and visually displays it to the user, who can then check detailed information about the monster (appearance, status, special abilities, etc.) on their smartphone screen.

[0217] Step 10:

[0218] The user selects a game mode within the app, for example, adventure mode or battle mode.

[0219] Step 11:

[0220] The terminal requests the server for the necessary data depending on the selected game mode. For example, if battle mode is selected, the terminal requests information about the opponent's monsters.

[0221] Step 12:

[0222] The device displays the battle scene based on the battle data received from the server, and the user controls their own monster to fight.

[0223] Step 13:

[0224] The server calculates the results of the battle and sends the results (win / loss, experience points and items gained, etc.) to the device.

[0225] Step 14:

[0226] The terminal displays the battle results received from the server to the user, allowing the user to check, for example, whether they won or lost, the items they acquired, and the experience points they gained.

[0227] Through this series of steps, users can create original monsters based on everyday objects they photograph, and then enjoy adventures and battles with those monsters. Furthermore, the emotion engine reflects the user's emotions in the game experience, providing a more personalized experience.

[0228] Example 2

[0229] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0230] Modern gaming systems lack personalized elements that allow users to have individualized experiences. In particular, few systems allow users to create unique characters using everyday objects and reflect the user's emotional state in the game. Furthermore, there is a lack of technology to analyze users' emotions in real time and directly link the results to the game experience.

[0231] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0232] In this invention, the server includes an image analysis means for transmitting image data to the server, a feature extraction means for the server to analyze the image data and extract features, and a generation means for the server to generate a monster based on the extracted features. This allows a user to take pictures of everyday objects and generate a personalized monster, providing a unique game experience that reflects the user's emotions.

[0233] "Image capturing means" refers to a device or function that a user uses to capture an image of an object, and typically refers to the camera function of a smartphone, tablet, or the like.

[0234] The "image analysis means" is a device or function for analyzing captured image data and transmitting it to a server.

[0235] The "feature extraction means" refers to a technology or algorithm for analyzing and identifying features such as the shape, color, and texture of an object from the image data received by the server.

[0236] The "generation means" is a device or algorithm for generating a monster based on the extracted characteristic information.

[0237] "Display means" refers to a device or function for visually displaying the generated monsters and their status to the user, and typically refers to the display of a smartphone or tablet.

[0238] A "status determining means" is a device or algorithm for determining the status and capabilities of a monster based on the characteristics of an identified object.

[0239] The "emotion reflection means" is a device or function that uses an emotion engine to analyze the user's emotional state and change the monster's appearance or status based on the analysis results.

[0240] "Game play means" refers to a device or function for providing game play using monsters in accordance with the game mode (adventure mode, battle mode, etc.) selected by the user.

[0241] The "battle result calculation means" is a device or algorithm for calculating the battle result and transmitting the calculation result to the terminal.

[0242] An "emotion engine" is a technology or system that recognizes a user's emotions (joy, sadness, anger, etc.) in real time by analyzing the user's facial expressions and voice.

[0243] The present invention is a system that allows a user to take a photograph of an object using a smartphone app and generate an original character based on that object. This system uses the following means.

[0244] First, the user launches the smartphone app and uses the device's camera to take a photo of an object of their choice. This photo can be of an everyday object (e.g., a pen, a stuffed toy, or a piece of furniture). The device then acquires the captured image data and sends it to a server. This transmission uses a secure communication protocol (e.g., HTTPS).

[0245] The server then analyzes the image data received from the device. This analysis uses computer vision algorithms (e.g., OpenCV) and other image analysis techniques. As a result of analyzing the image data, features such as the shape, color, and texture of the object are extracted. For example, if a user takes a photo of a pen on a desk, the pen's long, slender shape and unique color are analyzed.

[0246] The server then uses the extracted features to generate a monster using a generative AI model (e.g., GAN, Transformer). This determines the monster's appearance, status, and special abilities. For example, the pen's features generate a long, thin, blue monster with an "ink attack" ability.

[0247] Furthermore, the device is equipped with an emotion engine (e.g., Microsoft® Azure® Emotion API) that analyzes the user's facial expressions and voice in real time to identify emotions (joy, sadness, anger, etc.). The identified emotion data is sent to a server, which then dynamically changes the monster's appearance and status based on that data. For example, if the user is smiling, the monster's colors become more vibrant and its attack power increases.

[0248] Finally, the device visually displays the monster information received from the server to the user. This display allows the user to check the appearance, status, and special abilities of the created monster. The user then selects a game mode available within the app (e.g., adventure mode or battle mode) and enjoys gameplay using the monster. In battle mode, the server calculates the battle results and sends them to the device. The device then displays the results to the user. This allows the user to check the outcome of the battle, the experience points and items acquired, etc.

[0249] As a concrete example, a user takes a photo of a pen on their desk and sends the data to a server. The server analyzes the pen's characteristics (long, thin shape, blue color, etc.) and generates a monster based on them. At the same time, the server analyzes the user's emotions (e.g., smiling) and changes the color and status of the monster. The user can then enjoy the game using the generated monster.

[0250] Prompt Sentence Examples

[0251] "Please tell me the steps to take a picture of a pen on a desk to generate a monster, and how to reflect the user's emotions on that monster."

[0252] In this way, the present invention allows users to create their own unique monsters using everyday objects, providing a personalized gaming experience that reflects their own emotional state.

[0253] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0254] Step 1:

[0255] A user launches a smartphone app and uses the camera function to take a photo of an object. The input here is the object selected by the user, and the output is the captured image data. The specific action is for the user to tap the camera button in the app to take a photo. For example, the user takes a photo of a pen on a desk with the camera.

[0256] Step 2:

[0257] The device acquires the captured image data and sends the image data and the user's identification information (such as the user ID) to the server. The input is the captured image data and the user ID, and the output is the data to be sent to the server. Specifically, the device sends the image data and the user ID to the server using the HTTPS protocol.

[0258] Step 3:

[0259] The server analyzes the image data it receives. The input is the image data sent from the device, and the output is the feature information (shape, color, texture, etc.) of the objects in the image. Specifically, it uses computer vision technology (e.g., OpenCV) to identify the objects in the image and extract their features. For example, it identifies that the pen is long and thin and blue in color.

[0260] Step 4:

[0261] The server generates monsters based on the feature information of the object. The input is the extracted feature information, and the output is the data of the generated monster (appearance, status, special abilities). Specifically, the feature information is input into a generative AI model (e.g., GAN, Transformer), and a monster is automatically generated. For example, a long, thin blue monster is generated and set to have the special ability of "ink attack."

[0262] Step 5:

[0263] The emotion engine installed on the device analyzes the user's facial expressions and voice in real time to identify the user's emotions. The input is the user's real-time facial expressions and voice, and the output is the identified emotion data. The specific operation is to use the emotion engine (e.g., Microsoft Azure Emotion API) to analyze emotions from the user's facial expressions. For example, if the user is smiling, the emotion is identified as "joy."

[0264] Step 6:

[0265] The server receives emotion data from the emotion engine and changes the monster's appearance and status according to that emotion. The input is emotion data, and the output is the modified monster data. Specifically, the server makes the monster's color more vivid and increases its attack power.

[0266] Step 7:

[0267] The device receives the monster data sent from the server and visually displays it to the user. The input is the monster data sent from the server, and the output is the monster information displayed to the user. Specifically, the device displays the monster's image and status information on the app screen.

[0268] Step 8:

[0269] The user selects a game mode within the app. The input is the mode selection made by the user, and the output is the selected mode information. The specific operation is that the user selects adventure mode or battle mode from the menu within the app.

[0270] Step 9:

[0271] The terminal requests the necessary data from the server according to the selected game mode. The input is the selected mode information, and the output is a request to the server. Specifically, when battle mode is selected, the terminal requests opponent monster information from the server.

[0272] Step 10:

[0273] The server sends the requested data to the terminal. The input is the request from the terminal, and the output is the data to be sent to the terminal (such as information about the opponent's monster). The specific operation is that the server collects the requested data and sends it to the terminal.

[0274] Step 11:

[0275] The device displays a battle scene based on the game data it receives. The input is the game data sent from the server, and the output is the displayed battle scene. Specifically, the device displays the battle scene on the app screen, and the user can control their own monster to fight.

[0276] Step 12:

[0277] The server calculates the battle results and sends the results to the device. The input is detailed battle data, and the output is the calculated results (win / lose, experience points, acquired items, etc.). Specifically, the server analyzes the progress of the battle and calculates the results.

[0278] Step 13:

[0279] The device receives the battle results from the server and displays them to the user. The input is the calculated battle result, and the output is the result information displayed to the user. Specifically, the device displays the battle results, experience points gained, items, etc. on the app screen.

[0280] Prompt Sentence Examples

[0281] "Please tell me the steps to take a picture of a pen on a desk to generate a monster, and how to reflect the user's emotions on that monster."

[0282] (Application example 2)

[0283] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0284] Conventional game systems cannot generate monsters based on objects photographed by the user, nor can they change the game experience based on the user's emotions. This limits the game experience for users, making it difficult to provide a personalized experience.

[0285] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an image capturing means, an image analysis means, a generation means, a display means, an emotion recognition means for recognizing the user's emotion, and an attribute change means for changing the characteristics of a monster based on the recognized emotion. This makes it possible to generate an original monster based on an object photographed by the user, and further to dynamically change the monster's characteristics and game experience according to the user's emotion.

[0286] The "image capturing means" is a function that allows a user to capture an image of an object using a smartphone, digital camera, or the like.

[0287] "Image analysis means" refers to technology for processing captured image data and identifying objects in the image.

[0288] "Generation means" refers to an algorithm or program for automatically generating monsters based on the results of image analysis.

[0289] "Display means" refers to a screen or display for visually presenting the generated monsters and their characteristics to the user.

[0290] "Emotion recognition means" refers to a function for analyzing and identifying emotions from the user's facial expressions, voice, etc.

[0291] The "characteristics changing means" refers to a mechanism for dynamically changing the characteristics (color, ability, status, etc.) of a monster based on the recognized user's emotions.

[0292] "Status determination means" refers to an algorithm or program for determining the status and abilities of a monster according to the characteristics of the identified object.

[0293] "Gameplay means" refers to the interactive features that allow users to enjoy adventures and battles using monsters they create.

[0294] The system embodying this invention allows users to take a photograph of an object through a smartphone application and generate an original monster based on that object. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system generates monsters and provides a gaming experience that responds to the user's emotions.

[0295] System Configuration

[0296] The system includes the following elements:

[0297] 1. Image capture method:

[0298] The user uses the camera function of their smartphone to take a picture of any object, such as a pen on a desk, a stuffed animal, or a piece of furniture.

[0299] 2. Image analysis methods:

[0300] The device captures the captured image data and sends it to a server, which uses computer vision technology to identify the object and extract its features (shape, color, texture, etc.).

[0301] 3. Generation means:

[0302] Based on the object identification results, the server inputs characteristic information into a monster generation algorithm to determine the appearance, status, and special abilities of the generated monster.

[0303] 4. Display means:

[0304] The device receives the data of the generated monster and displays it visually to the user, who can then check detailed information about the monster (appearance, status, special abilities, etc.) on their smartphone screen.

[0305] 5. Emotion recognition means:

[0306] The device's built-in emotion engine analyzes the user's facial expressions and voice in real time to identify the user's emotions (joy, sadness, anger, etc.).

[0307] 6. How to change characteristics:

[0308] The server receives emotional data from the emotion engine and changes the monster's appearance and status according to the emotion. For example, if the user is happy, the monster will be set to have bright colors and high attack power.

[0309] 7. Status Determination Method:

[0310] It contains algorithms and programs for determining monster status and abilities based on the characteristics of an object.

[0311] 8. Gameplay methods:

[0312] It provides interactive features for users to adventure and battle with monsters they create.

[0313] Program processing explanation

[0314] First, the user takes a picture of an object using the camera function of their smartphone. At this time, the smartphone camera (image capture means) acquires image data using the OpenCV library etc. and temporarily stores it. After that, the captured image data is sent to the server (image analysis means) via the requests library.

[0315] The server analyzes the received image data using computer vision technology to identify the object. This analysis uses machine learning models and libraries such as TensorFlow. The characteristic information of the identified object (shape, color, texture) is input into a monster generation algorithm (generation means), and the appearance and characteristics of the monster are automatically generated.

[0316] The device receives data about the generated monster from the server and displays it on the smartphone screen, allowing the user to check detailed information about the generated monster (display means).

[0317] At the same time, the emotion engine analyzes the user's facial expressions and voice to identify their emotions in real time. The analysis results are sent to the server, and the monster's characteristics are changed based on the recognized emotion (characteristic change means). For example, if the user is happy, the monster's color becomes brighter and its attack power increases.

[0318] Users can select adventure mode or battle mode within the app and enjoy the game using monsters. The gameplay means retrieves monster data and opponent information according to the selected mode from the server, and battle scenes and adventure scenes are displayed in real time.

[0319] Examples and prompts

[0320] For example, a user can take a photo of a coffee cup on their desk, and the app will analyze the image to generate a "drink character." If the user then smiles, the character will gain an "active" status and be used in games and AR experiences.

[0321] Prompt for the generative AI model:

[0322] Let the user take a photo of a coffee cup, analyze the image, and generate a "drink character." Then, if the user smiles, make the character more vibrant and animated.

[0323] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0324] Step 1:

[0325] A user takes a photo of an object using the camera function of a smartphone. The input is the image of the object that the user photographs, and the output is the captured image data. Specifically, the user activates the smartphone camera, frames the object, and presses the shutter button.

[0326] Step 2:

[0327] The device acquires the captured image data and temporarily stores it. At this time, the input is the captured image data, and the output is an image file saved in local storage. Specifically, the OpenCV library is used to acquire the image data and save it to a file.

[0328] Step 3:

[0329] The device sends stored image data to the server. The input is the image file stored in local storage, and the output is the image data sent to the server. Specifically, the requests library is used to upload the image file to the server as an HTTP request.

[0330] Step 4:

[0331] The server analyzes the image data it receives. The input is the image data sent to the server, and the output is the analyzed object's identification results and feature information. Specifically, it uses computer vision technology and machine learning models (e.g., TensorFlow) to analyze the image and extract features such as the object's shape, color, and texture.

[0332] Step 5:

[0333] Based on the object identification results, the server inputs feature information into a monster generation algorithm to generate a monster. The input is the feature information of the identified object, and the output is the data of the generated monster. Specifically, the monster's appearance, status, and special abilities are generated based on the feature information.

[0334] Step 6:

[0335] The device receives the generated monster data from the server and displays it visually to the user. The input is the monster data sent from the server, and the output is the monster information displayed on the smartphone screen. Specifically, the received data is rendered into a UI component for visual display.

[0336] Step 7:

[0337] The emotion engine installed on the device analyzes the user's facial expressions and voice in real time to identify the user's emotions. The input is the user's facial expressions and voice data, and the output is the identified emotion data. Specifically, emotions are analyzed using the EmotionRecognizer library.

[0338] Step 8:

[0339] The server receives emotion data from the emotion engine and changes the monster's appearance and status according to the emotion. The input is emotion data, and the output is updated monster characteristic data. Specifically, an algorithm is applied to dynamically change the monster's color and status based on the emotion data.

[0340] Step 9:

[0341] The device displays updated monster data to the user based on the emotion. The input is the updated monster characteristic data, and the output is the updated monster information visually displayed on the smartphone screen. Specifically, the updated data is reflected in the UI components and re-rendered.

[0342] Step 10:

[0343] The user selects a game mode within the app and begins gameplay. The input is the user's selected game mode, and the output is an interactive game scene corresponding to the selected mode. Specifically, depending on the game mode selected, the app loads and presents an adventure mode or battle mode scene to the user.

[0344] Step 11:

[0345] The device requests data corresponding to the selected game mode from the server and obtains the necessary information. The input is the selected game mode, and the output is information about monsters and opponents obtained from the server. Specifically, a data request is sent to the server using an HTTP request, and the necessary data is received as a response.

[0346] Step 12:

[0347] The server calculates the battle results and sends them to the device. The input is the log data generated during the battle, and the output is the battle results, such as victory or defeat and experience points gained. Specifically, each event during the battle is saved as a log, and an algorithm is applied to calculate the results based on that data.

[0348] Step 13:

[0349] The terminal displays the battle results received from the server to the user. The input is the battle result data, and the output is the result information that is visually displayed on the smartphone screen. Specifically, the received result data is passed to the UI component and displayed visually.

[0350] This processing step allows the user to generate a monster based on a photographed object and use that monster to enjoy a highly personalized gaming experience.

[0351] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0352] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0353] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0354] [Second embodiment]

[0355] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0356] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0357] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0358] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0359] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0360] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0361] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0362] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0363] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0364] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0365] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0366] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0367] The system for implementing the present invention allows a user to take a photograph of an object through a smartphone application and generate an original monster based on the photograph. The system uses the following means.

[0368] System Configuration

[0369] The system includes (1) an image capturing means, (2) an image analyzing means, (3) a generating means, (4) a display means, (5) a status determining means, and (6) a game playing means.

[0370] Program processing

[0371] 1. Image capture method

[0372] The user launches the smartphone app and uses the app's camera to take a photo of any object, such as stationery, furniture, or an illustration.

[0373] 2. Image analysis methods

[0374] The device sends the captured image to a server, which uses computer vision technology to analyze the received image and identify the object in the image. For example, if the captured object is a pen, it extracts its shape, color, texture, and other characteristics.

[0375] 3. Generation means

[0376] The server runs a monster generation algorithm based on the characteristic information of the identified object. Based on the characteristics of the object, the monster's appearance, status, and special abilities are automatically generated. For example, if a pen is photographed and identified, a monster with a long, slender body is generated and configured with the ability to write.

[0377] 4. Status Determination Method

[0378] The server determines the stats (HP, attack power, defense power, etc.) of the monsters that are generated. This reflects the size, color, material, and other characteristics of the object. For example, a red pen might be set to have a high attack power.

[0379] 5. Display means

[0380] The terminal receives the monster data sent from the server and visually displays it to the user, allowing the user to check the monster's appearance and status information.

[0381] 6. Gameplay Methods

[0382] Users can use the monsters they create to participate in in-game adventures and battle modes. In single-player mode, users control monsters to progress through stages in the game. In multiplayer mode, users can battle against other users.

[0383] Specific examples

[0384] For example, suppose a user takes a photo of a pen on their desk with their smartphone. The device sends the image to a server, which analyzes the image and identifies the object as a "pen." The server then generates a monster with a long, thin shape based on the pen, and determines that this monster has a special ability called "ink attack."

[0385] The server sets the monster's status (e.g., HP 100, attack power 50, defense power 30) and sends the data to the device. The device displays the monster to the user, allowing the user to enjoy adventures and battles using the monster.

[0386] This system allows users to use everyday objects as monsters in the game, providing a constantly new experience.

[0387] The processing flow will be explained below.

[0388] Step 1:

[0389] The user starts the smartphone app and uses the camera function to take a picture of any object, such as a pen, stuffed animal, or piece of furniture.

[0390] Step 2:

[0391] The device acquires the captured image data and sends the image data and the user's identification information (such as the user ID) to the server.

[0392] Step 3:

[0393] The server analyzes the image data received from the device. Specifically, it uses computer vision technology to identify objects in the image and extract their features (shape, color, texture, etc.). For example, if a pen is photographed, the shape and color of the pen are identified.

[0394] Step 4:

[0395] The server inputs characteristic information into a monster generation algorithm based on the object identification results. The server then automatically generates the monster's appearance, status, and special abilities based on the input characteristic information. For example, based on the characteristics of a pen, it generates a long, thin monster with the ability to write.

[0396] Step 5:

[0397] The server determines the stats (HP, attack power, defense power, etc.) of the generated monster. The stats are reflected in the characteristics of the object, such as its color and shape. For example, a red pen might have a high attack power.

[0398] Step 6:

[0399] The server sends the generated monster data (image of appearance, status, special abilities, etc.) to the terminal.

[0400] Step 7:

[0401] The device analyzes the monster data received from the server and visually displays it to the user, who can then check detailed information about the monster (appearance, status, special abilities, etc.) on their smartphone screen.

[0402] Step 8:

[0403] The user selects a game mode within the app, for example, adventure mode or battle mode.

[0404] Step 9:

[0405] The terminal requests the server for the necessary data depending on the selected game mode. For example, if battle mode is selected, the terminal requests information about the opponent's monsters.

[0406] Step 10:

[0407] The device displays the battle scene based on the battle data received from the server, and the user controls their own monster to fight.

[0408] Step 11:

[0409] The server calculates the results of the battle and sends the results (win / loss, experience points and items gained, etc.) to the device.

[0410] Step 12:

[0411] The terminal displays the battle results received from the server to the user, allowing the user to check, for example, whether they won or lost, the items they acquired, and the experience points they gained.

[0412] Through this series of steps, users can create original monsters based on everyday objects they photograph, and then enjoy adventures and battles using those monsters.

[0413] Example 1

[0414] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0415] In conventional monster generation games, the monsters provided in the game are fixed, which means that the game lacks freshness and originality for users. Furthermore, there is a limitation in that users cannot customize the game experience using everyday objects. This makes it difficult to maintain interest in the game and limits the user experience.

[0416] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0417] In this invention, the server includes an image analysis means, a generation means, and a status determination means. This allows users to generate unique monsters using images they have taken, and to adjust the monster's status and special abilities. This allows users to enjoy a diverse and new gaming experience based on familiar objects.

[0418] The "image capturing means" is a means for a user to capture an image of an object using the camera function within a smartphone application.

[0419] A "terminal" is a device that transmits captured image data to a server via a communication line.

[0420] The "image analysis means" is a means for analyzing the image received by the server using computer vision technology and extracting feature information such as the shape, color, and texture of objects in the image.

[0421] The "generation means" is a means for generating a monster using a generation AI model based on the extracted feature information.

[0422] The "display means" is a means for transmitting the generated monster to the terminal and visually displaying the monster on the terminal.

[0423] The "status determining means" is a means for determining the status and special abilities of a monster based on the characteristics of the identified object.

[0424] "Gameplay means" refers to the means by which a user can engage in adventures and battles using the monsters they have created.

[0425] A system for implementing this invention allows a user to photograph an object using a smartphone application and then generate a unique monster based on that object. The system includes an image capturing unit, a terminal, an image analysis unit, a generation unit, a status determination unit, a display unit, and a gameplay unit. The specific operation and technologies used are described below.

[0426] The user launches the smartphone application "Monster Creator" and uses the app's camera function to take a photo of any object. This object can be anything, such as stationery, furniture, or an illustration. The device then sends the captured image data to a cloud server. This communication uses an internet connection.

[0427] The server uses computer vision technology to analyze the received images. This analysis utilizes image processing libraries and models, specifically the open source TensorFlow. The server identifies objects in the image and extracts feature information such as their shape, color, and texture. For example, if the captured image shows a pen, the pen's shape, color, and texture are analyzed.

[0428] The server then uses a generative AI model to generate a unique monster based on the extracted feature information. This algorithm is based on OpenAI's GPT model and determines the monster's appearance, status, and special abilities based on the object's characteristics. For example, if a pen is photographed, a long, thin monster will be generated with a special ability called ink attack.

[0429] The status of the generated monsters (HP, attack power, defense power, etc.) is also determined on the server. This process is affected by the color and material of the object, so for example, a red pen has high attack power, a blue pen has high defense power, and so on.

[0430] The server then sends data about the generated monster to the device. The device uses the received data to visually display the monster within the app. This display uses real-time 3D display technology such as the Unity engine. Here, users can check the appearance and status of the generated monster.

[0431] Finally, users can use the monsters they create to take part in in-game adventures and battle modes. In single-player mode, they can strengthen their monsters by progressing through stages, and in multiplayer mode, they can enjoy battles with other users.

[0432] Specific examples

[0433] For example, a user takes a picture of a blue pen on their desk. In this case, the device sends the image data to the server, which analyzes the image and identifies the object as a pen. Based on the characteristics of the pen, the server generates a monster with a long, thin shape and the ability to perform ink attacks. The monster's stats are set to HP 100, attack power 50, and defense power 30. The device displays the monster in 3D based on the received data, and the user can use that monster to progress through the game.

[0434] Prompt Sentence Examples

[0435] "Please take a photo of the blue pen on your desk. The server will analyze the image and create a special monster based on the pen's characteristics. This monster will have ink attack abilities and its stats will be HP 100, Attack Power 50, and Defense Power 30."

[0436] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0437] Step 1:

[0438] The user starts the smartphone application "Monster Creator" and takes a picture of the object using the camera function in the application. The input is the image data taken by the user. For example, let's say the user takes a picture of a blue pen on a desk.

[0439] Step 2:

[0440] The device sends the captured image data to the cloud server via a communication line. The input is the image data obtained in step 1, which is sent to the cloud server via an Internet connection. The output is the image data received by the cloud server.

[0441] Step 3:

[0442] The server analyzes the received image data using computer vision technology. For the analysis, it uses image processing libraries such as TensorFlow to identify and extract the shape, color, and texture of objects from the input image. The input is the image data sent from the device, and the output is the extracted object feature information. Specifically, the shape, color, and texture of the pen are analyzed.

[0443] Step 4:

[0444] The server generates monsters using a generative AI model based on the extracted feature information. This generation procedure uses an algorithm based on OpenAI's GPT model. The input is the feature information extracted in step 3, and the output is the data of the generated monster. For example, based on the pen, a long, thin monster is generated and given a special ability called an ink attack.

[0445] Step 5:

[0446] The server determines the status of the generated monster (HP, attack power, defense power, etc.). The input is the basic data of the generated monster, and the shape and color of the object are reflected in the status determination procedure. For example, a blue pen will be set as having high defense power. The output is the final status of the monster.

[0447] Step 6:

[0448] The server sends the data of the generated monster to the terminal. The input is the monster data determined in step 5, and this data is sent back to the terminal via the communication line. The output is the monster data received by the terminal.

[0449] Step 7:

[0450] The device visually displays the monster within the app based on the received monster data. Here, 3D display is performed in real time using the Unity engine. The input is the monster data received from the server, and the output is the monster visually displayed for the user. The user can check the monster's appearance and status within the app.

[0451] Step 8:

[0452] Users control the generated monsters to participate in the game's adventures and battle modes. The input is the user's actions and the monster's status data, and the output is the user's progress in the game. In single-player mode, users can strengthen their monsters, and in multiplayer mode, they can battle other users.

[0453] (Application example 1)

[0454] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0455] In modern society, an increasing number of people are seeking healthy meals and new cooking ideas. However, there are limited ways to easily access recipes based on everyday ingredients and information about the health benefits of each ingredient. This makes it difficult to easily plan meals that suit one's health condition and preferences.

[0456] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0457] In this invention, the server includes an image capturing means, an image analysis means, a generating means for generating a recipe and a healthy report based on the identified object, and a display means for displaying the generated recipe and healthy report, thereby enabling a user to easily capture an image of a food using a smartphone and instantly obtain healthy recipes and health information based on that food.

[0458] The "image capturing means" is a means for a user to capture an image of an object using a smartphone application.

[0459] The "image analysis means" is a means for analyzing a captured image and identifying an object in the image.

[0460] The "generation means" is a means for generating related recipes and healthy reports based on the identified object information.

[0461] The "display means" is a means for visually presenting the generated recipe and healthy report to the user.

[0462] The "status determining means" is a means for determining detailed recipe and health information based on characteristic information of the identified object.

[0463] The "service means" is a means for executing a service to provide recipes and healthy reports in response to a user's selection.

[0464] This invention allows a user to take a picture of a food using a smartphone application, and generates an original recipe and a healthy report based on the food. The system includes the following means.

[0465] 1. Use of Hardware and Software

[0466] Camera (hardware): Used to capture images.

[0467] OpenCV (software): A library for image processing.

[0468] Image recognition model (software): A model for analyzing objects in an image.

[0469] Python runtime environment (software): The infrastructure for running programs.

[0470] 2. Contents of data processing and data calculation

[0471] The user launches the smartphone application and takes a picture of the food with the camera.

[0472] The device processes the captured image using OpenCV and identifies the object using an image recognition model. The feature information of the identified object is then sent to the server.

[0473] The server operates algorithms to generate recipes and healthy reports based on the identified object information.

[0474] The terminal receives the recipe and healthy report sent from the server and displays them to the user.

[0475] 3. Specific Examples

[0476] User input: The user takes a picture of a tomato with their smartphone.

[0477] Output: The following information will be generated:

[0478] Recommended recipe: Tomato and basil pasta

[0479] Health Report: Rich in Vitamin C and has antioxidant properties.

[0480] 4. Examples of prompts

[0481] Prompt: "Extract features from the captured image and generate appropriate cooking recipes and health reports based on those features."

[0482] This system allows users to quickly and easily obtain health information about everyday ingredients and new recipe suggestions, helping them improve the quality of their diet.

[0483] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0484] Step 1:

[0485] A user starts the smartphone application and takes a picture of food with the camera. The user's input is the image of the food. This image data is input.

[0486] Step 2:

[0487] The device processes the captured image using OpenCV and identifies the object using an image recognition model. The input is the image captured by the user. Specifically, the image is preprocessed (resized, noise removed, etc.), and then the image recognition model is applied to identify the food. The output is feature information of the identified food.

[0488] Step 3:

[0489] The device sends the characteristic information of the identified food to the server. The input is the characteristic information of the identified food, and the output is a data packet containing that information. Specifically, the device's data transmission API is called to send the data to the server.

[0490] Step 4:

[0491] The server uses a means for generating recipes and health reports based on the received food characteristic information. The input is the food characteristic information sent from the terminal. Specifically, the server retrieves the corresponding recipe information from the database, analyzes health-related statistical data, and generates a health report. The output is the generated recipe and health report data.

[0492] Step 5:

[0493] The server sends the generated recipe and healthy report data to the terminal. The input is the generated recipe and healthy report data, and the output is a data packet. Specifically, the server calls the data transmission API to send the data to the terminal.

[0494] Step 6:

[0495] The terminal displays the recipe and healthy report data received from the server. The input is the recipe and healthy report data sent from the server. This data is visualized and displayed on the display in a format that is easy for the user to view. The output is visual information that the user can confirm.

[0496] This process flow allows users to easily take a picture of a food item using their smartphone and instantly obtain healthy recipes and health information based on that food.

[0497] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0498] A system embodying the present invention allows a user to photograph an object through a smartphone application and generate an original monster based on that object. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to generate monsters and provide a gaming experience that reflects the user's emotions. This system uses the following means:

[0499] System Configuration

[0500] The system includes (1) an image capturing means, (2) an image analyzing means, (3) a generating means, (4) a displaying means, (5) a status determining means, (6) a game playing means, and (7) an emotion engine.

[0501] Program processing

[0502] 1. Image capture method

[0503] The user starts the smartphone app and uses the camera function to take a picture of any object, such as a pen, stuffed animal, or piece of furniture.

[0504] 2. Image analysis methods

[0505] The device acquires the captured image data and sends the image data and the user's identification information (such as the user ID) to the server.

[0506] 3. Image Classification and Feature Extraction

[0507] The server analyzes the image data received from the device. Specifically, it uses computer vision technology to identify objects in the image and extract their features (shape, color, texture, etc.). For example, if a pen is photographed, the shape and color of the pen are identified.

[0508] 4. Monster Generation

[0509] The server inputs characteristic information into a monster generation algorithm based on the object identification results. The server then automatically generates the monster's appearance, status, and special abilities based on the input characteristic information. For example, based on the characteristics of a pen, it generates a long, thin monster with the ability to write.

[0510] 5. Emotion Recognition and Affect

[0511] The device's built-in emotion engine analyzes the user's facial expressions and voice in real time to identify the user's emotions (joy, sadness, anger, etc.).

[0512] 6. Reflecting monster emotions

[0513] The server receives emotional data from the emotion engine and changes the monster's appearance and status according to the emotion. For example, if the user is happy, the monster will be set to have bright colors and high attack power.

[0514] 7. Status check and display

[0515] The device receives the monster data sent from the server and visually displays it to the user, who can then check detailed information about the monster (appearance, status, special abilities, etc.) on their smartphone screen.

[0516] 8. Start playing the game

[0517] The user selects a game mode within the app, for example, adventure mode or battle mode.

[0518] 9. Data Acquisition Depending on the Mode

[0519] The terminal requests the server for the necessary data depending on the selected game mode. For example, if battle mode is selected, the terminal requests information about the opponent's monsters.

[0520] 10. Displaying battle scenes

[0521] The device displays the battle scene based on the battle data received from the server, and the user controls their own monster to fight.

[0522] 11. Calculation and notification of battle results

[0523] The server calculates the results of the battle and sends the results (win / loss, experience points and items gained, etc.) to the device.

[0524] 12. Display of battle results

[0525] The terminal displays the battle results received from the server to the user, allowing the user to check, for example, whether they won or lost, the items they acquired, and the experience points they gained.

[0526] Specific examples

[0527] For example, a user takes a photo of a pen on their desk with their smartphone. The device sends the image to a server, which analyzes the image and identifies the object as a "pen." The server then generates a monster with a long, thin shape based on the pen, and determines that the monster has special abilities such as "ink attack."

[0528] At the same time, the emotion engine recognizes that the user is smiling and sends that data to the server, which uses this information to make the monsters it generates more vibrant in color and increase their attack power.

[0529] The server sets the monster's status (for example, HP 100, attack power 70, defense power 30) and sends the data to the device. The device displays the monster to the user, and the user can enjoy adventures and battles using the monster.

[0530] This system allows users to use everyday objects as monsters in the game, and their own emotional state will affect their gaming experience, providing a more personalized experience.

[0531] The processing flow will be explained below.

[0532] Step 1:

[0533] The user starts the smartphone app and uses the camera function to take a picture of any object, such as a pen, stuffed animal, or piece of furniture.

[0534] Step 2:

[0535] The device acquires the captured image data and sends the image data and the user's identification information (such as the user ID) to the server.

[0536] Step 3:

[0537] The server analyzes the image data received from the device. Specifically, it uses computer vision technology to identify objects in the image and extract their features (shape, color, texture, etc.). For example, if a pen is photographed, the shape and color of the pen are identified.

[0538] Step 4:

[0539] The server inputs characteristic information into a monster generation algorithm based on the object identification results. The server then automatically generates the monster's appearance, status, and special abilities based on the input characteristic information. For example, based on the characteristics of a pen, it generates a long, thin monster with the ability to write.

[0540] Step 5:

[0541] The device's built-in emotion engine analyzes the user's facial expressions and voice in real time to identify the user's emotions (joy, sadness, anger, etc.).

[0542] Step 6:

[0543] The server receives emotional data from the emotion engine and changes the monster's appearance and status according to the emotion. For example, if the user is happy, the monster will be set to have bright colors and high attack power.

[0544] Step 7:

[0545] The server determines the stats (HP, attack power, defense power, etc.) of the generated monster. The stats are reflected in the characteristics of the object, such as its color and shape. For example, a red pen might have a high attack power.

[0546] Step 8:

[0547] The server sends the generated monster data (image of appearance, status, special abilities, etc.) to the terminal.

[0548] Step 9:

[0549] The device analyzes the monster data received from the server and visually displays it to the user, who can then check detailed information about the monster (appearance, status, special abilities, etc.) on their smartphone screen.

[0550] Step 10:

[0551] The user selects a game mode within the app, for example, adventure mode or battle mode.

[0552] Step 11:

[0553] The terminal requests the server for the necessary data depending on the selected game mode. For example, if battle mode is selected, the terminal requests information about the opponent's monsters.

[0554] Step 12:

[0555] The device displays the battle scene based on the battle data received from the server, and the user controls their own monster to fight.

[0556] Step 13:

[0557] The server calculates the results of the battle and sends the results (win / loss, experience points and items gained, etc.) to the device.

[0558] Step 14:

[0559] The terminal displays the battle results received from the server to the user, allowing the user to check, for example, whether they won or lost, the items they acquired, and the experience points they gained.

[0560] Through this series of steps, users can create original monsters based on everyday objects they photograph, and then enjoy adventures and battles with those monsters. Furthermore, the emotion engine reflects the user's emotions in the game experience, providing a more personalized experience.

[0561] Example 2

[0562] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0563] Modern gaming systems lack personalized elements that allow users to have individualized experiences. In particular, few systems allow users to create unique characters using everyday objects and reflect the user's emotional state in the game. Furthermore, there is a lack of technology to analyze users' emotions in real time and directly link the results to the game experience.

[0564] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0565] In this invention, the server includes an image analysis means for transmitting image data to the server, a feature extraction means for the server to analyze the image data and extract features, and a generation means for the server to generate a monster based on the extracted features. This allows a user to take pictures of everyday objects and generate a personalized monster, providing a unique game experience that reflects the user's emotions.

[0566] "Image capturing means" refers to a device or function that a user uses to capture an image of an object, and typically refers to the camera function of a smartphone, tablet, or the like.

[0567] The "image analysis means" is a device or function for analyzing captured image data and transmitting it to a server.

[0568] The "feature extraction means" refers to a technology or algorithm for analyzing and identifying features such as the shape, color, and texture of an object from the image data received by the server.

[0569] The "generation means" is a device or algorithm for generating a monster based on the extracted characteristic information.

[0570] "Display means" refers to a device or function for visually displaying the generated monsters and their status to the user, and typically refers to the display of a smartphone or tablet.

[0571] A "status determining means" is a device or algorithm for determining the status and capabilities of a monster based on the characteristics of an identified object.

[0572] The "emotion reflection means" is a device or function that uses an emotion engine to analyze the user's emotional state and change the monster's appearance or status based on the analysis results.

[0573] "Game play means" refers to a device or function for providing game play using monsters in accordance with the game mode (adventure mode, battle mode, etc.) selected by the user.

[0574] The "battle result calculation means" is a device or algorithm for calculating the battle result and transmitting the calculation result to the terminal.

[0575] An "emotion engine" is a technology or system that recognizes a user's emotions (joy, sadness, anger, etc.) in real time by analyzing the user's facial expressions and voice.

[0576] The present invention is a system that allows a user to take a photograph of an object using a smartphone app and generate an original character based on that object. This system uses the following means.

[0577] First, the user launches the smartphone app and uses the device's camera to take a photo of an object of their choice. This photo can be of an everyday object (e.g., a pen, a stuffed toy, or a piece of furniture). The device then acquires the captured image data and sends it to a server. This transmission uses a secure communication protocol (e.g., HTTPS).

[0578] The server then analyzes the image data received from the device. This analysis uses computer vision algorithms (e.g., OpenCV) and other image analysis techniques. As a result of analyzing the image data, features such as the shape, color, and texture of the object are extracted. For example, if a user takes a photo of a pen on a desk, the pen's long, slender shape and unique color are analyzed.

[0579] The server then uses the extracted features to generate a monster using a generative AI model (e.g., GAN, Transformer). This determines the monster's appearance, status, and special abilities. For example, the pen's features generate a long, thin, blue monster with an "ink attack" ability.

[0580] Furthermore, the device is equipped with an emotion engine (e.g., Microsoft Azure Emotion API) that analyzes the user's facial expressions and voice in real time to identify emotions (joy, sadness, anger, etc.). The identified emotion data is sent to a server, which then dynamically changes the monster's appearance and status based on that data. For example, if the user is smiling, the monster's colors become more vibrant and its attack power increases.

[0581] Finally, the device visually displays the monster information received from the server to the user. This display allows the user to check the appearance, status, and special abilities of the created monster. The user then selects a game mode available within the app (e.g., adventure mode or battle mode) and enjoys gameplay using the monster. In battle mode, the server calculates the battle results and sends them to the device. The device then displays the results to the user. This allows the user to check the outcome of the battle, the experience points and items acquired, etc.

[0582] As a concrete example, a user takes a photo of a pen on their desk and sends the data to a server. The server analyzes the pen's characteristics (long, thin shape, blue color, etc.) and generates a monster based on them. At the same time, the server analyzes the user's emotions (e.g., smiling) and changes the color and status of the monster. The user can then enjoy the game using the generated monster.

[0583] Prompt Sentence Examples

[0584] "Please tell me the steps to take a picture of a pen on a desk to generate a monster, and how to reflect the user's emotions on that monster."

[0585] In this way, the present invention allows users to create their own unique monsters using everyday objects, providing a personalized gaming experience that reflects their own emotional state.

[0586] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0587] Step 1:

[0588] A user launches a smartphone app and uses the camera function to take a photo of an object. The input here is the object selected by the user, and the output is the captured image data. The specific action is for the user to tap the camera button in the app to take a photo. For example, the user takes a photo of a pen on a desk with the camera.

[0589] Step 2:

[0590] The device acquires the captured image data and sends the image data and the user's identification information (such as the user ID) to the server. The input is the captured image data and the user ID, and the output is the data to be sent to the server. Specifically, the device sends the image data and the user ID to the server using the HTTPS protocol.

[0591] Step 3:

[0592] The server analyzes the image data it receives. The input is the image data sent from the device, and the output is the feature information (shape, color, texture, etc.) of the objects in the image. Specifically, it uses computer vision technology (e.g., OpenCV) to identify the objects in the image and extract their features. For example, it identifies that the pen is long and thin and blue in color.

[0593] Step 4:

[0594] The server generates monsters based on the feature information of the object. The input is the extracted feature information, and the output is the data of the generated monster (appearance, status, special abilities). Specifically, the feature information is input into a generative AI model (e.g., GAN, Transformer), and a monster is automatically generated. For example, a long, thin blue monster is generated and set to have the special ability of "ink attack."

[0595] Step 5:

[0596] The emotion engine installed on the device analyzes the user's facial expressions and voice in real time to identify the user's emotions. The input is the user's real-time facial expressions and voice, and the output is the identified emotion data. The specific operation is to use the emotion engine (e.g., Microsoft Azure Emotion API) to analyze emotions from the user's facial expressions. For example, if the user is smiling, the emotion is identified as "joy."

[0597] Step 6:

[0598] The server receives emotion data from the emotion engine and changes the monster's appearance and status according to that emotion. The input is emotion data, and the output is the modified monster data. Specifically, the server makes the monster's color more vivid and increases its attack power.

[0599] Step 7:

[0600] The device receives the monster data sent from the server and visually displays it to the user. The input is the monster data sent from the server, and the output is the monster information displayed to the user. Specifically, the device displays the monster's image and status information on the app screen.

[0601] Step 8:

[0602] The user selects a game mode within the app. The input is the mode selection made by the user, and the output is the selected mode information. The specific operation is that the user selects adventure mode or battle mode from the menu within the app.

[0603] Step 9:

[0604] The terminal requests the necessary data from the server according to the selected game mode. The input is the selected mode information, and the output is a request to the server. Specifically, when battle mode is selected, the terminal requests opponent monster information from the server.

[0605] Step 10:

[0606] The server sends the requested data to the terminal. The input is the request from the terminal, and the output is the data to be sent to the terminal (such as information about the opponent's monster). The specific operation is that the server collects the requested data and sends it to the terminal.

[0607] Step 11:

[0608] The device displays a battle scene based on the game data it receives. The input is the game data sent from the server, and the output is the displayed battle scene. Specifically, the device displays the battle scene on the app screen, and the user can control their own monster to fight.

[0609] Step 12:

[0610] The server calculates the battle results and sends the results to the device. The input is detailed battle data, and the output is the calculated results (win / lose, experience points, acquired items, etc.). Specifically, the server analyzes the progress of the battle and calculates the results.

[0611] Step 13:

[0612] The device receives the battle results from the server and displays them to the user. The input is the calculated battle result, and the output is the result information displayed to the user. Specifically, the device displays the battle results, experience points gained, items, etc. on the app screen.

[0613] Prompt Sentence Examples

[0614] "Please tell me the steps to take a picture of a pen on a desk to generate a monster, and how to reflect the user's emotions on that monster."

[0615] (Application example 2)

[0616] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0617] Conventional game systems cannot generate monsters based on objects photographed by the user, nor can they change the game experience based on the user's emotions. This limits the game experience for users, making it difficult to provide a personalized experience.

[0618] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an image capturing means, an image analysis means, a generation means, a display means, an emotion recognition means for recognizing the user's emotion, and an attribute change means for changing the characteristics of a monster based on the recognized emotion. This makes it possible to generate an original monster based on an object photographed by the user, and further to dynamically change the monster's characteristics and game experience according to the user's emotion.

[0619] The "image capturing means" is a function that allows a user to capture an image of an object using a smartphone, digital camera, or the like.

[0620] "Image analysis means" refers to technology for processing captured image data and identifying objects in the image.

[0621] "Generation means" refers to an algorithm or program for automatically generating monsters based on the results of image analysis.

[0622] "Display means" refers to a screen or display for visually presenting the generated monsters and their characteristics to the user.

[0623] "Emotion recognition means" refers to a function for analyzing and identifying emotions from the user's facial expressions, voice, etc.

[0624] The "characteristics changing means" refers to a mechanism for dynamically changing the characteristics (color, ability, status, etc.) of a monster based on the recognized user's emotions.

[0625] "Status determination means" refers to an algorithm or program for determining the status and abilities of a monster according to the characteristics of the identified object.

[0626] "Gameplay means" refers to the interactive features that allow users to enjoy adventures and battles using monsters they create.

[0627] The system embodying this invention allows users to take a photograph of an object through a smartphone application and generate an original monster based on that object. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system generates monsters and provides a gaming experience that responds to the user's emotions.

[0628] System Configuration

[0629] The system includes the following elements:

[0630] 1. Image capture method:

[0631] The user uses the camera function of their smartphone to take a picture of any object, such as a pen on a desk, a stuffed animal, or a piece of furniture.

[0632] 2. Image analysis methods:

[0633] The device captures the captured image data and sends it to a server, which uses computer vision technology to identify the object and extract its features (shape, color, texture, etc.).

[0634] 3. Generation means:

[0635] Based on the object identification results, the server inputs characteristic information into a monster generation algorithm to determine the appearance, status, and special abilities of the generated monster.

[0636] 4. Display means:

[0637] The device receives the data of the generated monster and displays it visually to the user, who can then check detailed information about the monster (appearance, status, special abilities, etc.) on their smartphone screen.

[0638] 5. Emotion recognition means:

[0639] The device's built-in emotion engine analyzes the user's facial expressions and voice in real time to identify the user's emotions (joy, sadness, anger, etc.).

[0640] 6. How to change characteristics:

[0641] The server receives emotional data from the emotion engine and changes the monster's appearance and status according to the emotion. For example, if the user is happy, the monster will be set to have bright colors and high attack power.

[0642] 7. Status Determination Method:

[0643] It contains algorithms and programs for determining monster status and abilities based on the characteristics of an object.

[0644] 8. Gameplay methods:

[0645] It provides interactive features for users to adventure and battle with monsters they create.

[0646] Program processing explanation

[0647] First, the user takes a picture of an object using the camera function of their smartphone. At this time, the smartphone camera (image capture means) acquires image data using the OpenCV library etc. and temporarily stores it. After that, the captured image data is sent to the server (image analysis means) via the requests library.

[0648] The server analyzes the received image data using computer vision technology to identify the object. This analysis uses machine learning models and libraries such as TensorFlow. The characteristic information of the identified object (shape, color, texture) is input into a monster generation algorithm (generation means), and the appearance and characteristics of the monster are automatically generated.

[0649] The device receives data about the generated monster from the server and displays it on the smartphone screen, allowing the user to check detailed information about the generated monster (display means).

[0650] At the same time, the emotion engine analyzes the user's facial expressions and voice to identify their emotions in real time. The analysis results are sent to the server, and the monster's characteristics are changed based on the recognized emotion (characteristic change means). For example, if the user is happy, the monster's color becomes brighter and its attack power increases.

[0651] Users can select adventure mode or battle mode within the app and enjoy the game using monsters. The gameplay means retrieves monster data and opponent information according to the selected mode from the server, and battle scenes and adventure scenes are displayed in real time.

[0652] Examples and prompts

[0653] For example, a user can take a photo of a coffee cup on their desk, and the app will analyze the image to generate a "drink character." If the user then smiles, the character will gain an "active" status and be used in games and AR experiences.

[0654] Prompt for the generative AI model:

[0655] Let the user take a photo of a coffee cup, analyze the image, and generate a "drink character." Then, if the user smiles, make the character more vibrant and animated.

[0656] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0657] Step 1:

[0658] A user takes a photo of an object using the camera function of a smartphone. The input is the image of the object that the user photographs, and the output is the captured image data. Specifically, the user activates the smartphone camera, frames the object, and presses the shutter button.

[0659] Step 2:

[0660] The device acquires the captured image data and temporarily stores it. At this time, the input is the captured image data, and the output is an image file saved in local storage. Specifically, the OpenCV library is used to acquire the image data and save it to a file.

[0661] Step 3:

[0662] The device sends stored image data to the server. The input is the image file stored in local storage, and the output is the image data sent to the server. Specifically, the requests library is used to upload the image file to the server as an HTTP request.

[0663] Step 4:

[0664] The server analyzes the image data it receives. The input is the image data sent to the server, and the output is the analyzed object's identification results and feature information. Specifically, it uses computer vision technology and machine learning models (e.g., TensorFlow) to analyze the image and extract features such as the object's shape, color, and texture.

[0665] Step 5:

[0666] Based on the object identification results, the server inputs feature information into a monster generation algorithm to generate a monster. The input is the feature information of the identified object, and the output is the data of the generated monster. Specifically, the monster's appearance, status, and special abilities are generated based on the feature information.

[0667] Step 6:

[0668] The device receives the generated monster data from the server and displays it visually to the user. The input is the monster data sent from the server, and the output is the monster information displayed on the smartphone screen. Specifically, the received data is rendered into a UI component for visual display.

[0669] Step 7:

[0670] The emotion engine installed on the device analyzes the user's facial expressions and voice in real time to identify the user's emotions. The input is the user's facial expressions and voice data, and the output is the identified emotion data. Specifically, emotions are analyzed using the EmotionRecognizer library.

[0671] Step 8:

[0672] The server receives emotion data from the emotion engine and changes the monster's appearance and status according to the emotion. The input is emotion data, and the output is updated monster characteristic data. Specifically, an algorithm is applied to dynamically change the monster's color and status based on the emotion data.

[0673] Step 9:

[0674] The device displays updated monster data to the user based on the emotion. The input is the updated monster characteristic data, and the output is the updated monster information visually displayed on the smartphone screen. Specifically, the updated data is reflected in the UI components and re-rendered.

[0675] Step 10:

[0676] The user selects a game mode within the app and begins gameplay. The input is the user's selected game mode, and the output is an interactive game scene corresponding to the selected mode. Specifically, depending on the game mode selected, the app loads and presents an adventure mode or battle mode scene to the user.

[0677] Step 11:

[0678] The device requests data corresponding to the selected game mode from the server and obtains the necessary information. The input is the selected game mode, and the output is information about monsters and opponents obtained from the server. Specifically, a data request is sent to the server using an HTTP request, and the necessary data is received as a response.

[0679] Step 12:

[0680] The server calculates the battle results and sends them to the device. The input is the log data generated during the battle, and the output is the battle results, such as victory or defeat and experience points gained. Specifically, each event during the battle is saved as a log, and an algorithm is applied to calculate the results based on that data.

[0681] Step 13:

[0682] The terminal displays the battle results received from the server to the user. The input is the battle result data, and the output is the result information that is visually displayed on the smartphone screen. Specifically, the received result data is passed to the UI component and displayed visually.

[0683] This processing step allows the user to generate a monster based on a photographed object and use that monster to enjoy a highly personalized gaming experience.

[0684] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0685] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0686] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0687] [Third embodiment]

[0688] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0689] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0690] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0691] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0692] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0693] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0694] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0695] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0696] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0697] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0698] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0699] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0700] The system for implementing the present invention allows a user to take a photograph of an object through a smartphone application and generate an original monster based on the photograph. The system uses the following means.

[0701] System Configuration

[0702] The system includes (1) an image capturing means, (2) an image analyzing means, (3) a generating means, (4) a display means, (5) a status determining means, and (6) a game playing means.

[0703] Program processing

[0704] 1. Image capture method

[0705] The user launches the smartphone app and uses the app's camera to take a photo of any object, such as stationery, furniture, or an illustration.

[0706] 2. Image analysis methods

[0707] The device sends the captured image to a server, which uses computer vision technology to analyze the received image and identify the object in the image. For example, if the captured object is a pen, it extracts its shape, color, texture, and other characteristics.

[0708] 3. Generation means

[0709] The server runs a monster generation algorithm based on the characteristic information of the identified object. Based on the characteristics of the object, the monster's appearance, status, and special abilities are automatically generated. For example, if a pen is photographed and identified, a monster with a long, slender body is generated and configured with the ability to write.

[0710] 4. Status Determination Method

[0711] The server determines the stats (HP, attack power, defense power, etc.) of the monsters that are generated. This reflects the size, color, material, and other characteristics of the object. For example, a red pen might be set to have a high attack power.

[0712] 5. Display means

[0713] The terminal receives the monster data sent from the server and visually displays it to the user, allowing the user to check the monster's appearance and status information.

[0714] 6. Gameplay Methods

[0715] Users can use the monsters they create to participate in in-game adventures and battle modes. In single-player mode, users control monsters to progress through stages in the game. In multiplayer mode, users can battle against other users.

[0716] Specific examples

[0717] For example, suppose a user takes a photo of a pen on their desk with their smartphone. The device sends the image to a server, which analyzes the image and identifies the object as a "pen." The server then generates a monster with a long, thin shape based on the pen, and determines that this monster has a special ability called "ink attack."

[0718] The server sets the monster's status (e.g., HP 100, attack power 50, defense power 30) and sends the data to the device. The device displays the monster to the user, allowing the user to enjoy adventures and battles using the monster.

[0719] This system allows users to use everyday objects as monsters in the game, providing a constantly new experience.

[0720] The processing flow will be explained below.

[0721] Step 1:

[0722] The user starts the smartphone app and uses the camera function to take a picture of any object, such as a pen, stuffed animal, or piece of furniture.

[0723] Step 2:

[0724] The device acquires the captured image data and sends the image data and the user's identification information (such as the user ID) to the server.

[0725] Step 3:

[0726] The server analyzes the image data received from the device. Specifically, it uses computer vision technology to identify objects in the image and extract their features (shape, color, texture, etc.). For example, if a pen is photographed, the shape and color of the pen are identified.

[0727] Step 4:

[0728] The server inputs characteristic information into a monster generation algorithm based on the object identification results. The server then automatically generates the monster's appearance, status, and special abilities based on the input characteristic information. For example, based on the characteristics of a pen, it generates a long, thin monster with the ability to write.

[0729] Step 5:

[0730] The server determines the stats (HP, attack power, defense power, etc.) of the generated monster. The stats are reflected in the characteristics of the object, such as its color and shape. For example, a red pen might have a high attack power.

[0731] Step 6:

[0732] The server sends the generated monster data (image of appearance, status, special abilities, etc.) to the terminal.

[0733] Step 7:

[0734] The device analyzes the monster data received from the server and visually displays it to the user, who can then check detailed information about the monster (appearance, status, special abilities, etc.) on their smartphone screen.

[0735] Step 8:

[0736] The user selects a game mode within the app, for example, adventure mode or battle mode.

[0737] Step 9:

[0738] The terminal requests the server for the necessary data depending on the selected game mode. For example, if battle mode is selected, the terminal requests information about the opponent's monsters.

[0739] Step 10:

[0740] The device displays the battle scene based on the battle data received from the server, and the user controls their own monster to fight.

[0741] Step 11:

[0742] The server calculates the results of the battle and sends the results (win / loss, experience points and items gained, etc.) to the device.

[0743] Step 12:

[0744] The terminal displays the battle results received from the server to the user, allowing the user to check, for example, whether they won or lost, the items they acquired, and the experience points they gained.

[0745] Through this series of steps, users can create original monsters based on everyday objects they photograph, and then enjoy adventures and battles using those monsters.

[0746] Example 1

[0747] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0748] In conventional monster generation games, the monsters provided in the game are fixed, which means that the game lacks freshness and originality for users. Furthermore, there is a limitation in that users cannot customize the game experience using everyday objects. This makes it difficult to maintain interest in the game and limits the user experience.

[0749] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0750] In this invention, the server includes an image analysis means, a generation means, and a status determination means. This allows users to generate unique monsters using images they have taken, and to adjust the monster's status and special abilities. This allows users to enjoy a diverse and new gaming experience based on familiar objects.

[0751] The "image capturing means" is a means for a user to capture an image of an object using the camera function within a smartphone application.

[0752] A "terminal" is a device that transmits captured image data to a server via a communication line.

[0753] The "image analysis means" is a means for analyzing the image received by the server using computer vision technology and extracting feature information such as the shape, color, and texture of objects in the image.

[0754] The "generation means" is a means for generating a monster using a generation AI model based on the extracted feature information.

[0755] The "display means" is a means for transmitting the generated monster to the terminal and visually displaying the monster on the terminal.

[0756] The "status determining means" is a means for determining the status and special abilities of a monster based on the characteristics of the identified object.

[0757] "Gameplay means" refers to the means by which a user can engage in adventures and battles using the monsters they have created.

[0758] A system for implementing this invention allows a user to photograph an object using a smartphone application and then generate a unique monster based on that object. The system includes an image capturing unit, a terminal, an image analysis unit, a generation unit, a status determination unit, a display unit, and a gameplay unit. The specific operation and technologies used are described below.

[0759] The user launches the smartphone application "Monster Creator" and uses the app's camera function to take a photo of any object. This object can be anything, such as stationery, furniture, or an illustration. The device then sends the captured image data to a cloud server. This communication uses an internet connection.

[0760] The server uses computer vision technology to analyze the received images. This analysis utilizes image processing libraries and models, specifically the open source TensorFlow. The server identifies objects in the image and extracts feature information such as their shape, color, and texture. For example, if the captured image shows a pen, the pen's shape, color, and texture are analyzed.

[0761] The server then uses a generative AI model to generate a unique monster based on the extracted feature information. This algorithm is based on OpenAI's GPT model and determines the monster's appearance, status, and special abilities based on the object's characteristics. For example, if a pen is photographed, a long, thin monster will be generated with a special ability called ink attack.

[0762] The status of the generated monsters (HP, attack power, defense power, etc.) is also determined on the server. This process is affected by the color and material of the object, so for example, a red pen has high attack power, a blue pen has high defense power, and so on.

[0763] The server then sends data about the generated monster to the device. The device uses the received data to visually display the monster within the app. This display uses real-time 3D display technology such as the Unity engine. Here, users can check the appearance and status of the generated monster.

[0764] Finally, users can use the monsters they create to take part in in-game adventures and battle modes. In single-player mode, they can strengthen their monsters by progressing through stages, and in multiplayer mode, they can enjoy battles with other users.

[0765] Specific examples

[0766] For example, a user takes a picture of a blue pen on their desk. In this case, the device sends the image data to the server, which analyzes the image and identifies the object as a pen. Based on the characteristics of the pen, the server generates a monster with a long, thin shape and the ability to perform ink attacks. The monster's stats are set to HP 100, attack power 50, and defense power 30. The device displays the monster in 3D based on the received data, and the user can use that monster to progress through the game.

[0767] Prompt Sentence Examples

[0768] "Please take a photo of the blue pen on your desk. The server will analyze the image and create a special monster based on the pen's characteristics. This monster will have ink attack abilities and its stats will be HP 100, Attack Power 50, and Defense Power 30."

[0769] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0770] Step 1:

[0771] The user starts the smartphone application "Monster Creator" and takes a picture of the object using the camera function in the application. The input is the image data taken by the user. For example, let's say the user takes a picture of a blue pen on a desk.

[0772] Step 2:

[0773] The device sends the captured image data to the cloud server via a communication line. The input is the image data obtained in step 1, which is sent to the cloud server via an Internet connection. The output is the image data received by the cloud server.

[0774] Step 3:

[0775] The server analyzes the received image data using computer vision technology. For the analysis, it uses image processing libraries such as TensorFlow to identify and extract the shape, color, and texture of objects from the input image. The input is the image data sent from the device, and the output is the extracted object feature information. Specifically, the shape, color, and texture of the pen are analyzed.

[0776] Step 4:

[0777] The server generates monsters using a generative AI model based on the extracted feature information. This generation procedure uses an algorithm based on OpenAI's GPT model. The input is the feature information extracted in step 3, and the output is the data of the generated monster. For example, based on the pen, a long, thin monster is generated and given a special ability called an ink attack.

[0778] Step 5:

[0779] The server determines the status of the generated monster (HP, attack power, defense power, etc.). The input is the basic data of the generated monster, and the shape and color of the object are reflected in the status determination procedure. For example, a blue pen will be set as having high defense power. The output is the final status of the monster.

[0780] Step 6:

[0781] The server sends the data of the generated monster to the terminal. The input is the monster data determined in step 5, and this data is sent back to the terminal via the communication line. The output is the monster data received by the terminal.

[0782] Step 7:

[0783] The device visually displays the monster within the app based on the received monster data. Here, 3D display is performed in real time using the Unity engine. The input is the monster data received from the server, and the output is the monster visually displayed for the user. The user can check the monster's appearance and status within the app.

[0784] Step 8:

[0785] Users control the generated monsters to participate in the game's adventures and battle modes. The input is the user's actions and the monster's status data, and the output is the user's progress in the game. In single-player mode, users can strengthen their monsters, and in multiplayer mode, they can battle other users.

[0786] (Application example 1)

[0787] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0788] In modern society, an increasing number of people are seeking healthy meals and new cooking ideas. However, there are limited ways to easily access recipes based on everyday ingredients and information about the health benefits of each ingredient. This makes it difficult to easily plan meals that suit one's health condition and preferences.

[0789] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0790] In this invention, the server includes an image capturing means, an image analysis means, a generating means for generating a recipe and a healthy report based on the identified object, and a display means for displaying the generated recipe and healthy report, thereby enabling a user to easily capture an image of a food using a smartphone and instantly obtain healthy recipes and health information based on that food.

[0791] The "image capturing means" is a means for a user to capture an image of an object using a smartphone application.

[0792] The "image analysis means" is a means for analyzing a captured image and identifying an object in the image.

[0793] The "generation means" is a means for generating related recipes and healthy reports based on the identified object information.

[0794] The "display means" is a means for visually presenting the generated recipe and healthy report to the user.

[0795] The "status determining means" is a means for determining detailed recipe and health information based on characteristic information of the identified object.

[0796] The "service means" is a means for executing a service to provide recipes and healthy reports in response to a user's selection.

[0797] This invention allows a user to take a picture of a food using a smartphone application, and generates an original recipe and a healthy report based on the food. The system includes the following means.

[0798] 1. Use of Hardware and Software

[0799] Camera (hardware): Used to capture images.

[0800] OpenCV (software): A library for image processing.

[0801] Image recognition model (software): A model for analyzing objects in an image.

[0802] Python runtime environment (software): The infrastructure for running programs.

[0803] 2. Contents of data processing and data calculation

[0804] The user launches the smartphone application and takes a picture of the food with the camera.

[0805] The device processes the captured image using OpenCV and identifies the object using an image recognition model. The feature information of the identified object is then sent to the server.

[0806] The server operates algorithms to generate recipes and healthy reports based on the identified object information.

[0807] The terminal receives the recipe and healthy report sent from the server and displays them to the user.

[0808] 3. Specific Examples

[0809] User input: The user takes a picture of a tomato with their smartphone.

[0810] Output: The following information will be generated:

[0811] Recommended recipe: Tomato and basil pasta

[0812] Health Report: Rich in Vitamin C and has antioxidant properties.

[0813] 4. Examples of prompts

[0814] Prompt: "Extract features from the captured image and generate appropriate cooking recipes and health reports based on those features."

[0815] This system allows users to quickly and easily obtain health information about everyday ingredients and new recipe suggestions, helping them improve the quality of their diet.

[0816] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0817] Step 1:

[0818] A user starts the smartphone application and takes a picture of food with the camera. The user's input is the image of the food. This image data is input.

[0819] Step 2:

[0820] The device processes the captured image using OpenCV and identifies the object using an image recognition model. The input is the image captured by the user. Specifically, the image is preprocessed (resized, noise removed, etc.), and then the image recognition model is applied to identify the food. The output is feature information of the identified food.

[0821] Step 3:

[0822] The device sends the characteristic information of the identified food to the server. The input is the characteristic information of the identified food, and the output is a data packet containing that information. Specifically, the device's data transmission API is called to send the data to the server.

[0823] Step 4:

[0824] The server uses a means for generating recipes and health reports based on the received food characteristic information. The input is the food characteristic information sent from the terminal. Specifically, the server retrieves the corresponding recipe information from the database, analyzes health-related statistical data, and generates a health report. The output is the generated recipe and health report data.

[0825] Step 5:

[0826] The server sends the generated recipe and healthy report data to the terminal. The input is the generated recipe and healthy report data, and the output is a data packet. Specifically, the server calls the data transmission API to send the data to the terminal.

[0827] Step 6:

[0828] The terminal displays the recipe and healthy report data received from the server. The input is the recipe and healthy report data sent from the server. This data is visualized and displayed on the display in a format that is easy for the user to view. The output is visual information that the user can confirm.

[0829] This process flow allows users to easily take a picture of a food item using their smartphone and instantly obtain healthy recipes and health information based on that food.

[0830] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0831] A system embodying the present invention allows a user to photograph an object through a smartphone application and generate an original monster based on that object. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to generate monsters and provide a gaming experience that reflects the user's emotions. This system uses the following means:

[0832] System Configuration

[0833] The system includes (1) an image capturing means, (2) an image analyzing means, (3) a generating means, (4) a displaying means, (5) a status determining means, (6) a game playing means, and (7) an emotion engine.

[0834] Program processing

[0835] 1. Image capture method

[0836] The user starts the smartphone app and uses the camera function to take a picture of any object, such as a pen, stuffed animal, or piece of furniture.

[0837] 2. Image analysis methods

[0838] The device acquires the captured image data and sends the image data and the user's identification information (such as the user ID) to the server.

[0839] 3. Image Classification and Feature Extraction

[0840] The server analyzes the image data received from the device. Specifically, it uses computer vision technology to identify objects in the image and extract their features (shape, color, texture, etc.). For example, if a pen is photographed, the shape and color of the pen are identified.

[0841] 4. Monster Generation

[0842] The server inputs characteristic information into a monster generation algorithm based on the object identification results. The server then automatically generates the monster's appearance, status, and special abilities based on the input characteristic information. For example, based on the characteristics of a pen, it generates a long, thin monster with the ability to write.

[0843] 5. Emotion Recognition and Affect

[0844] The device's built-in emotion engine analyzes the user's facial expressions and voice in real time to identify the user's emotions (joy, sadness, anger, etc.).

[0845] 6. Reflecting monster emotions

[0846] The server receives emotional data from the emotion engine and changes the monster's appearance and status according to the emotion. For example, if the user is happy, the monster will be set to have bright colors and high attack power.

[0847] 7. Status check and display

[0848] The device receives the monster data sent from the server and visually displays it to the user, who can then check detailed information about the monster (appearance, status, special abilities, etc.) on their smartphone screen.

[0849] 8. Start playing the game

[0850] The user selects a game mode within the app, for example, adventure mode or battle mode.

[0851] 9. Data Acquisition Depending on the Mode

[0852] The terminal requests the server for the necessary data depending on the selected game mode. For example, if battle mode is selected, the terminal requests information about the opponent's monsters.

[0853] 10. Displaying battle scenes

[0854] The device displays the battle scene based on the battle data received from the server, and the user controls their own monster to fight.

[0855] 11. Calculation and notification of battle results

[0856] The server calculates the results of the battle and sends the results (win / loss, experience points and items gained, etc.) to the device.

[0857] 12. Display of battle results

[0858] The terminal displays the battle results received from the server to the user, allowing the user to check, for example, whether they won or lost, the items they acquired, and the experience points they gained.

[0859] Specific examples

[0860] For example, a user takes a photo of a pen on their desk with their smartphone. The device sends the image to a server, which analyzes the image and identifies the object as a "pen." The server then generates a monster with a long, thin shape based on the pen, and determines that the monster has special abilities such as "ink attack."

[0861] At the same time, the emotion engine recognizes that the user is smiling and sends that data to the server, which uses this information to make the monsters it generates more vibrant in color and increase their attack power.

[0862] The server sets the monster's status (for example, HP 100, attack power 70, defense power 30) and sends the data to the device. The device displays the monster to the user, and the user can enjoy adventures and battles using the monster.

[0863] This system allows users to use everyday objects as monsters in the game, and their own emotional state will affect their gaming experience, providing a more personalized experience.

[0864] The processing flow will be explained below.

[0865] Step 1:

[0866] The user starts the smartphone app and uses the camera function to take a picture of any object, such as a pen, stuffed animal, or piece of furniture.

[0867] Step 2:

[0868] The device acquires the captured image data and sends the image data and the user's identification information (such as the user ID) to the server.

[0869] Step 3:

[0870] The server analyzes the image data received from the device. Specifically, it uses computer vision technology to identify objects in the image and extract their features (shape, color, texture, etc.). For example, if a pen is photographed, the shape and color of the pen are identified.

[0871] Step 4:

[0872] The server inputs characteristic information into a monster generation algorithm based on the object identification results. The server then automatically generates the monster's appearance, status, and special abilities based on the input characteristic information. For example, based on the characteristics of a pen, it generates a long, thin monster with the ability to write.

[0873] Step 5:

[0874] The device's built-in emotion engine analyzes the user's facial expressions and voice in real time to identify the user's emotions (joy, sadness, anger, etc.).

[0875] Step 6:

[0876] The server receives emotional data from the emotion engine and changes the monster's appearance and status according to the emotion. For example, if the user is happy, the monster will be set to have bright colors and high attack power.

[0877] Step 7:

[0878] The server determines the stats (HP, attack power, defense power, etc.) of the generated monster. The stats are reflected in the characteristics of the object, such as its color and shape. For example, a red pen might have a high attack power.

[0879] Step 8:

[0880] The server sends the generated monster data (image of appearance, status, special abilities, etc.) to the terminal.

[0881] Step 9:

[0882] The device analyzes the monster data received from the server and visually displays it to the user, who can then check detailed information about the monster (appearance, status, special abilities, etc.) on their smartphone screen.

[0883] Step 10:

[0884] The user selects a game mode within the app, for example, adventure mode or battle mode.

[0885] Step 11:

[0886] The terminal requests the server for the necessary data depending on the selected game mode. For example, if battle mode is selected, the terminal requests information about the opponent's monsters.

[0887] Step 12:

[0888] The device displays the battle scene based on the battle data received from the server, and the user controls their own monster to fight.

[0889] Step 13:

[0890] The server calculates the results of the battle and sends the results (win / loss, experience points and items gained, etc.) to the device.

[0891] Step 14:

[0892] The terminal displays the battle results received from the server to the user, allowing the user to check, for example, whether they won or lost, the items they acquired, and the experience points they gained.

[0893] Through this series of steps, users can create original monsters based on everyday objects they photograph, and then enjoy adventures and battles with those monsters. Furthermore, the emotion engine reflects the user's emotions in the game experience, providing a more personalized experience.

[0894] Example 2

[0895] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0896] Modern gaming systems lack personalized elements that allow users to have individualized experiences. In particular, few systems allow users to create unique characters using everyday objects and reflect the user's emotional state in the game. Furthermore, there is a lack of technology to analyze users' emotions in real time and directly link the results to the game experience.

[0897] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0898] In this invention, the server includes an image analysis means for transmitting image data to the server, a feature extraction means for the server to analyze the image data and extract features, and a generation means for the server to generate a monster based on the extracted features. This allows a user to take pictures of everyday objects and generate a personalized monster, providing a unique game experience that reflects the user's emotions.

[0899] "Image capturing means" refers to a device or function that a user uses to capture an image of an object, and typically refers to the camera function of a smartphone, tablet, or the like.

[0900] The "image analysis means" is a device or function for analyzing captured image data and transmitting it to a server.

[0901] The "feature extraction means" refers to a technology or algorithm for analyzing and identifying features such as the shape, color, and texture of an object from the image data received by the server.

[0902] The "generation means" is a device or algorithm for generating a monster based on the extracted characteristic information.

[0903] "Display means" refers to a device or function for visually displaying the generated monsters and their status to the user, and typically refers to the display of a smartphone or tablet.

[0904] A "status determining means" is a device or algorithm for determining the status and capabilities of a monster based on the characteristics of an identified object.

[0905] The "emotion reflection means" is a device or function that uses an emotion engine to analyze the user's emotional state and change the monster's appearance or status based on the analysis results.

[0906] "Game play means" refers to a device or function for providing game play using monsters in accordance with the game mode (adventure mode, battle mode, etc.) selected by the user.

[0907] The "battle result calculation means" is a device or algorithm for calculating the battle result and transmitting the calculation result to the terminal.

[0908] An "emotion engine" is a technology or system that recognizes a user's emotions (joy, sadness, anger, etc.) in real time by analyzing the user's facial expressions and voice.

[0909] The present invention is a system that allows a user to take a photograph of an object using a smartphone app and generate an original character based on that object. This system uses the following means.

[0910] First, the user launches the smartphone app and uses the device's camera to take a photo of an object of their choice. This photo can be of an everyday object (e.g., a pen, a stuffed toy, or a piece of furniture). The device then acquires the captured image data and sends it to a server. This transmission uses a secure communication protocol (e.g., HTTPS).

[0911] The server then analyzes the image data received from the device. This analysis uses computer vision algorithms (e.g., OpenCV) and other image analysis techniques. As a result of analyzing the image data, features such as the shape, color, and texture of the object are extracted. For example, if a user takes a photo of a pen on a desk, the pen's long, slender shape and unique color are analyzed.

[0912] The server then uses the extracted features to generate a monster using a generative AI model (e.g., GAN, Transformer). This determines the monster's appearance, status, and special abilities. For example, the pen's features generate a long, thin, blue monster with an "ink attack" ability.

[0913] Furthermore, the device is equipped with an emotion engine (e.g., Microsoft Azure Emotion API) that analyzes the user's facial expressions and voice in real time to identify emotions (joy, sadness, anger, etc.). The identified emotion data is sent to a server, which then dynamically changes the monster's appearance and status based on that data. For example, if the user is smiling, the monster's colors become more vibrant and its attack power increases.

[0914] Finally, the device visually displays the monster information received from the server to the user. This display allows the user to check the appearance, status, and special abilities of the created monster. The user then selects a game mode available within the app (e.g., adventure mode or battle mode) and enjoys gameplay using the monster. In battle mode, the server calculates the battle results and sends them to the device. The device then displays the results to the user. This allows the user to check the outcome of the battle, the experience points and items acquired, etc.

[0915] As a concrete example, a user takes a photo of a pen on their desk and sends the data to a server. The server analyzes the pen's characteristics (long, thin shape, blue color, etc.) and generates a monster based on them. At the same time, the server analyzes the user's emotions (e.g., smiling) and changes the color and status of the monster. The user can then enjoy the game using the generated monster.

[0916] Prompt Sentence Examples

[0917] "Please tell me the steps to take a picture of a pen on a desk to generate a monster, and how to reflect the user's emotions on that monster."

[0918] In this way, the present invention allows users to create their own unique monsters using everyday objects, providing a personalized gaming experience that reflects their own emotional state.

[0919] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0920] Step 1:

[0921] A user launches a smartphone app and uses the camera function to take a photo of an object. The input here is the object selected by the user, and the output is the captured image data. The specific action is for the user to tap the camera button in the app to take a photo. For example, the user takes a photo of a pen on a desk with the camera.

[0922] Step 2:

[0923] The device acquires the captured image data and sends the image data and the user's identification information (such as the user ID) to the server. The input is the captured image data and the user ID, and the output is the data to be sent to the server. Specifically, the device sends the image data and the user ID to the server using the HTTPS protocol.

[0924] Step 3:

[0925] The server analyzes the image data it receives. The input is the image data sent from the device, and the output is the feature information (shape, color, texture, etc.) of the objects in the image. Specifically, it uses computer vision technology (e.g., OpenCV) to identify the objects in the image and extract their features. For example, it identifies that the pen is long and thin and blue in color.

[0926] Step 4:

[0927] The server generates monsters based on the feature information of the object. The input is the extracted feature information, and the output is the data of the generated monster (appearance, status, special abilities). Specifically, the feature information is input into a generative AI model (e.g., GAN, Transformer), and a monster is automatically generated. For example, a long, thin blue monster is generated and set to have the special ability of "ink attack."

[0928] Step 5:

[0929] The emotion engine installed on the device analyzes the user's facial expressions and voice in real time to identify the user's emotions. The input is the user's real-time facial expressions and voice, and the output is the identified emotion data. The specific operation is to use the emotion engine (e.g., Microsoft Azure Emotion API) to analyze emotions from the user's facial expressions. For example, if the user is smiling, the emotion is identified as "joy."

[0930] Step 6:

[0931] The server receives emotion data from the emotion engine and changes the monster's appearance and status according to that emotion. The input is emotion data, and the output is the modified monster data. Specifically, the server makes the monster's color more vivid and increases its attack power.

[0932] Step 7:

[0933] The device receives the monster data sent from the server and visually displays it to the user. The input is the monster data sent from the server, and the output is the monster information displayed to the user. Specifically, the device displays the monster's image and status information on the app screen.

[0934] Step 8:

[0935] The user selects a game mode within the app. The input is the mode selection made by the user, and the output is the selected mode information. The specific operation is that the user selects adventure mode or battle mode from the menu within the app.

[0936] Step 9:

[0937] The terminal requests the necessary data from the server according to the selected game mode. The input is the selected mode information, and the output is a request to the server. Specifically, when battle mode is selected, the terminal requests opponent monster information from the server.

[0938] Step 10:

[0939] The server sends the requested data to the terminal. The input is the request from the terminal, and the output is the data to be sent to the terminal (such as information about the opponent's monster). The specific operation is that the server collects the requested data and sends it to the terminal.

[0940] Step 11:

[0941] The device displays a battle scene based on the game data it receives. The input is the game data sent from the server, and the output is the displayed battle scene. Specifically, the device displays the battle scene on the app screen, and the user can control their own monster to fight.

[0942] Step 12:

[0943] The server calculates the battle results and sends the results to the device. The input is detailed battle data, and the output is the calculated results (win / lose, experience points, acquired items, etc.). Specifically, the server analyzes the progress of the battle and calculates the results.

[0944] Step 13:

[0945] The device receives the battle results from the server and displays them to the user. The input is the calculated battle result, and the output is the result information displayed to the user. Specifically, the device displays the battle results, experience points gained, items, etc. on the app screen.

[0946] Prompt Sentence Examples

[0947] "Please tell me the steps to take a picture of a pen on a desk to generate a monster, and how to reflect the user's emotions on that monster."

[0948] (Application example 2)

[0949] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0950] Conventional game systems cannot generate monsters based on objects photographed by the user, nor can they change the game experience based on the user's emotions. This limits the game experience for users, making it difficult to provide a personalized experience.

[0951] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an image capturing means, an image analysis means, a generation means, a display means, an emotion recognition means for recognizing the user's emotion, and an attribute change means for changing the characteristics of a monster based on the recognized emotion. This makes it possible to generate an original monster based on an object photographed by the user, and further to dynamically change the monster's characteristics and game experience according to the user's emotion.

[0952] The "image capturing means" is a function that allows a user to capture an image of an object using a smartphone, digital camera, or the like.

[0953] "Image analysis means" refers to technology for processing captured image data and identifying objects in the image.

[0954] "Generation means" refers to an algorithm or program for automatically generating monsters based on the results of image analysis.

[0955] "Display means" refers to a screen or display for visually presenting the generated monsters and their characteristics to the user.

[0956] "Emotion recognition means" refers to a function for analyzing and identifying emotions from the user's facial expressions, voice, etc.

[0957] The "characteristics changing means" refers to a mechanism for dynamically changing the characteristics (color, ability, status, etc.) of a monster based on the recognized user's emotions.

[0958] "Status determination means" refers to an algorithm or program for determining the status and abilities of a monster according to the characteristics of the identified object.

[0959] "Gameplay means" refers to the interactive features that allow users to enjoy adventures and battles using monsters they create.

[0960] The system embodying this invention allows users to take a photograph of an object through a smartphone application and generate an original monster based on that object. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system generates monsters and provides a gaming experience that responds to the user's emotions.

[0961] System Configuration

[0962] The system includes the following elements:

[0963] 1. Image capture method:

[0964] The user uses the camera function of their smartphone to take a picture of any object, such as a pen on a desk, a stuffed animal, or a piece of furniture.

[0965] 2. Image analysis methods:

[0966] The device captures the captured image data and sends it to a server, which uses computer vision technology to identify the object and extract its features (shape, color, texture, etc.).

[0967] 3. Generation means:

[0968] Based on the object identification results, the server inputs characteristic information into a monster generation algorithm to determine the appearance, status, and special abilities of the generated monster.

[0969] 4. Display means:

[0970] The device receives the data of the generated monster and displays it visually to the user, who can then check detailed information about the monster (appearance, status, special abilities, etc.) on their smartphone screen.

[0971] 5. Emotion recognition means:

[0972] The device's built-in emotion engine analyzes the user's facial expressions and voice in real time to identify the user's emotions (joy, sadness, anger, etc.).

[0973] 6. How to change characteristics:

[0974] The server receives emotional data from the emotion engine and changes the monster's appearance and status according to the emotion. For example, if the user is happy, the monster will be set to have bright colors and high attack power.

[0975] 7. Status Determination Method:

[0976] It contains algorithms and programs for determining monster status and abilities based on the characteristics of an object.

[0977] 8. Gameplay methods:

[0978] It provides interactive features for users to adventure and battle with monsters they create.

[0979] Program processing explanation

[0980] First, the user takes a picture of an object using the camera function of their smartphone. At this time, the smartphone camera (image capture means) acquires image data using the OpenCV library etc. and temporarily stores it. After that, the captured image data is sent to the server (image analysis means) via the requests library.

[0981] The server analyzes the received image data using computer vision technology to identify the object. This analysis uses machine learning models and libraries such as TensorFlow. The characteristic information of the identified object (shape, color, texture) is input into a monster generation algorithm (generation means), and the appearance and characteristics of the monster are automatically generated.

[0982] The device receives data about the generated monster from the server and displays it on the smartphone screen, allowing the user to check detailed information about the generated monster (display means).

[0983] At the same time, the emotion engine analyzes the user's facial expressions and voice to identify their emotions in real time. The analysis results are sent to the server, and the monster's characteristics are changed based on the recognized emotion (characteristic change means). For example, if the user is happy, the monster's color becomes brighter and its attack power increases.

[0984] Users can select adventure mode or battle mode within the app and enjoy the game using monsters. The gameplay means retrieves monster data and opponent information according to the selected mode from the server, and battle scenes and adventure scenes are displayed in real time.

[0985] Examples and prompts

[0986] For example, a user can take a photo of a coffee cup on their desk, and the app will analyze the image to generate a "drink character." If the user then smiles, the character will gain an "active" status and be used in games and AR experiences.

[0987] Prompt for the generative AI model:

[0988] Let the user take a photo of a coffee cup, analyze the image, and generate a "drink character." Then, if the user smiles, make the character more vibrant and animated.

[0989] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0990] Step 1:

[0991] A user takes a photo of an object using the camera function of a smartphone. The input is the image of the object that the user photographs, and the output is the captured image data. Specifically, the user activates the smartphone camera, frames the object, and presses the shutter button.

[0992] Step 2:

[0993] The device acquires the captured image data and temporarily stores it. At this time, the input is the captured image data, and the output is an image file saved in local storage. Specifically, the OpenCV library is used to acquire the image data and save it to a file.

[0994] Step 3:

[0995] The device sends stored image data to the server. The input is the image file stored in local storage, and the output is the image data sent to the server. Specifically, the requests library is used to upload the image file to the server as an HTTP request.

[0996] Step 4:

[0997] The server analyzes the image data it receives. The input is the image data sent to the server, and the output is the analyzed object's identification results and feature information. Specifically, it uses computer vision technology and machine learning models (e.g., TensorFlow) to analyze the image and extract features such as the object's shape, color, and texture.

[0998] Step 5:

[0999] Based on the object identification results, the server inputs feature information into a monster generation algorithm to generate a monster. The input is the feature information of the identified object, and the output is the data of the generated monster. Specifically, the monster's appearance, status, and special abilities are generated based on the feature information.

[1000] Step 6:

[1001] The device receives the generated monster data from the server and displays it visually to the user. The input is the monster data sent from the server, and the output is the monster information displayed on the smartphone screen. Specifically, the received data is rendered into a UI component for visual display.

[1002] Step 7:

[1003] The emotion engine installed on the device analyzes the user's facial expressions and voice in real time to identify the user's emotions. The input is the user's facial expressions and voice data, and the output is the identified emotion data. Specifically, emotions are analyzed using the EmotionRecognizer library.

[1004] Step 8:

[1005] The server receives emotion data from the emotion engine and changes the monster's appearance and status according to the emotion. The input is emotion data, and the output is updated monster characteristic data. Specifically, an algorithm is applied to dynamically change the monster's color and status based on the emotion data.

[1006] Step 9:

[1007] The device displays updated monster data to the user based on the emotion. The input is the updated monster characteristic data, and the output is the updated monster information visually displayed on the smartphone screen. Specifically, the updated data is reflected in the UI components and re-rendered.

[1008] Step 10:

[1009] The user selects a game mode within the app and begins gameplay. The input is the user's selected game mode, and the output is an interactive game scene corresponding to the selected mode. Specifically, depending on the game mode selected, the app loads and presents an adventure mode or battle mode scene to the user.

[1010] Step 11:

[1011] The device requests data corresponding to the selected game mode from the server and obtains the necessary information. The input is the selected game mode, and the output is information about monsters and opponents obtained from the server. Specifically, a data request is sent to the server using an HTTP request, and the necessary data is received as a response.

[1012] Step 12:

[1013] The server calculates the battle results and sends them to the device. The input is the log data generated during the battle, and the output is the battle results, such as victory or defeat and experience points gained. Specifically, each event during the battle is saved as a log, and an algorithm is applied to calculate the results based on that data.

[1014] Step 13:

[1015] The terminal displays the battle results received from the server to the user. The input is the battle result data, and the output is the result information that is visually displayed on the smartphone screen. Specifically, the received result data is passed to the UI component and displayed visually.

[1016] This processing step allows the user to generate a monster based on a photographed object and use that monster to enjoy a highly personalized gaming experience.

[1017] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1018] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1019] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1020] [Fourth embodiment]

[1021] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1022] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1023] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1024] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1025] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1026] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1027] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1028] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1029] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1030] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1032] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1034] The system for implementing the present invention allows a user to take a photograph of an object through a smartphone application and generate an original monster based on the photograph. The system uses the following means.

[1035] System Configuration

[1036] The system includes (1) an image capturing means, (2) an image analyzing means, (3) a generating means, (4) a display means, (5) a status determining means, and (6) a game playing means.

[1037] Program processing

[1038] 1. Image capture method

[1039] The user launches the smartphone app and uses the app's camera to take a photo of any object, such as stationery, furniture, or an illustration.

[1040] 2. Image analysis methods

[1041] The device sends the captured image to a server, which uses computer vision technology to analyze the received image and identify the object in the image. For example, if the captured object is a pen, it extracts its shape, color, texture, and other characteristics.

[1042] 3. Generation means

[1043] The server runs a monster generation algorithm based on the characteristic information of the identified object. Based on the characteristics of the object, the monster's appearance, status, and special abilities are automatically generated. For example, if a pen is photographed and identified, a monster with a long, slender body is generated and configured with the ability to write.

[1044] 4. Status Determination Method

[1045] The server determines the stats (HP, attack power, defense power, etc.) of the monsters that are generated. This reflects the size, color, material, and other characteristics of the object. For example, a red pen might be set to have a high attack power.

[1046] 5. Display means

[1047] The terminal receives the monster data sent from the server and visually displays it to the user, allowing the user to check the monster's appearance and status information.

[1048] 6. Gameplay Methods

[1049] Users can use the monsters they create to participate in in-game adventures and battle modes. In single-player mode, users control monsters to progress through stages in the game. In multiplayer mode, users can battle against other users.

[1050] Specific examples

[1051] For example, suppose a user takes a photo of a pen on their desk with their smartphone. The device sends the image to a server, which analyzes the image and identifies the object as a "pen." The server then generates a monster with a long, thin shape based on the pen, and determines that this monster has a special ability called "ink attack."

[1052] The server sets the monster's status (e.g., HP 100, attack power 50, defense power 30) and sends the data to the device. The device displays the monster to the user, allowing the user to enjoy adventures and battles using the monster.

[1053] This system allows users to use everyday objects as monsters in the game, providing a constantly new experience.

[1054] The processing flow will be explained below.

[1055] Step 1:

[1056] The user starts the smartphone app and uses the camera function to take a picture of any object, such as a pen, stuffed animal, or piece of furniture.

[1057] Step 2:

[1058] The device acquires the captured image data and sends the image data and the user's identification information (such as the user ID) to the server.

[1059] Step 3:

[1060] The server analyzes the image data received from the device. Specifically, it uses computer vision technology to identify objects in the image and extract their features (shape, color, texture, etc.). For example, if a pen is photographed, the shape and color of the pen are identified.

[1061] Step 4:

[1062] The server inputs characteristic information into a monster generation algorithm based on the object identification results. The server then automatically generates the monster's appearance, status, and special abilities based on the input characteristic information. For example, based on the characteristics of a pen, it generates a long, thin monster with the ability to write.

[1063] Step 5:

[1064] The server determines the stats (HP, attack power, defense power, etc.) of the generated monster. The stats are reflected in the characteristics of the object, such as its color and shape. For example, a red pen might have a high attack power.

[1065] Step 6:

[1066] The server sends the generated monster data (image of appearance, status, special abilities, etc.) to the terminal.

[1067] Step 7:

[1068] The device analyzes the monster data received from the server and visually displays it to the user, who can then check detailed information about the monster (appearance, status, special abilities, etc.) on their smartphone screen.

[1069] Step 8:

[1070] The user selects a game mode within the app, for example, adventure mode or battle mode.

[1071] Step 9:

[1072] The terminal requests the server for the necessary data depending on the selected game mode. For example, if battle mode is selected, the terminal requests information about the opponent's monsters.

[1073] Step 10:

[1074] The device displays the battle scene based on the battle data received from the server, and the user controls their own monster to fight.

[1075] Step 11:

[1076] The server calculates the results of the battle and sends the results (win / loss, experience points and items gained, etc.) to the device.

[1077] Step 12:

[1078] The terminal displays the battle results received from the server to the user, allowing the user to check, for example, whether they won or lost, the items they acquired, and the experience points they gained.

[1079] Through this series of steps, users can create original monsters based on everyday objects they photograph, and then enjoy adventures and battles using those monsters.

[1080] Example 1

[1081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1082] In conventional monster generation games, the monsters provided in the game are fixed, which means that the game lacks freshness and originality for users. Furthermore, there is a limitation in that users cannot customize the game experience using everyday objects. This makes it difficult to maintain interest in the game and limits the user experience.

[1083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1084] In this invention, the server includes an image analysis means, a generation means, and a status determination means. This allows users to generate unique monsters using images they have taken, and to adjust the monster's status and special abilities. This allows users to enjoy a diverse and new gaming experience based on familiar objects.

[1085] The "image capturing means" is a means for a user to capture an image of an object using the camera function within a smartphone application.

[1086] A "terminal" is a device that transmits captured image data to a server via a communication line.

[1087] The "image analysis means" is a means for analyzing the image received by the server using computer vision technology and extracting feature information such as the shape, color, and texture of objects in the image.

[1088] The "generation means" is a means for generating a monster using a generation AI model based on the extracted feature information.

[1089] The "display means" is a means for transmitting the generated monster to the terminal and visually displaying the monster on the terminal.

[1090] The "status determining means" is a means for determining the status and special abilities of a monster based on the characteristics of the identified object.

[1091] "Gameplay means" refers to the means by which a user can engage in adventures and battles using the monsters they have created.

[1092] A system for implementing this invention allows a user to photograph an object using a smartphone application and then generate a unique monster based on that object. The system includes an image capturing unit, a terminal, an image analysis unit, a generation unit, a status determination unit, a display unit, and a gameplay unit. The specific operation and technologies used are described below.

[1093] The user launches the smartphone application "Monster Creator" and uses the app's camera function to take a photo of any object. This object can be anything, such as stationery, furniture, or an illustration. The device then sends the captured image data to a cloud server. This communication uses an internet connection.

[1094] The server uses computer vision technology to analyze the received images. This analysis utilizes image processing libraries and models, specifically the open source TensorFlow. The server identifies objects in the image and extracts feature information such as their shape, color, and texture. For example, if the captured image shows a pen, the pen's shape, color, and texture are analyzed.

[1095] The server then uses a generative AI model to generate a unique monster based on the extracted feature information. This algorithm is based on OpenAI's GPT model and determines the monster's appearance, status, and special abilities based on the object's characteristics. For example, if a pen is photographed, a long, thin monster will be generated with a special ability called ink attack.

[1096] The status of the generated monsters (HP, attack power, defense power, etc.) is also determined on the server. This process is affected by the color and material of the object, so for example, a red pen has high attack power, a blue pen has high defense power, and so on.

[1097] The server then sends data about the generated monster to the device. The device uses the received data to visually display the monster within the app. This display uses real-time 3D display technology such as the Unity engine. Here, users can check the appearance and status of the generated monster.

[1098] Finally, users can use the monsters they create to take part in in-game adventures and battle modes. In single-player mode, they can strengthen their monsters by progressing through stages, and in multiplayer mode, they can enjoy battles with other users.

[1099] Specific examples

[1100] For example, a user takes a picture of a blue pen on their desk. In this case, the device sends the image data to the server, which analyzes the image and identifies the object as a pen. Based on the characteristics of the pen, the server generates a monster with a long, thin shape and the ability to perform ink attacks. The monster's stats are set to HP 100, attack power 50, and defense power 30. The device displays the monster in 3D based on the received data, and the user can use that monster to progress through the game.

[1101] Prompt Sentence Examples

[1102] "Please take a photo of the blue pen on your desk. The server will analyze the image and create a special monster based on the pen's characteristics. This monster will have ink attack abilities and its stats will be HP 100, Attack Power 50, and Defense Power 30."

[1103] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1104] Step 1:

[1105] The user starts the smartphone application "Monster Creator" and takes a picture of the object using the camera function in the application. The input is the image data taken by the user. For example, let's say the user takes a picture of a blue pen on a desk.

[1106] Step 2:

[1107] The device sends the captured image data to the cloud server via a communication line. The input is the image data obtained in step 1, which is sent to the cloud server via an Internet connection. The output is the image data received by the cloud server.

[1108] Step 3:

[1109] The server analyzes the received image data using computer vision technology. For the analysis, it uses image processing libraries such as TensorFlow to identify and extract the shape, color, and texture of objects from the input image. The input is the image data sent from the device, and the output is the extracted object feature information. Specifically, the shape, color, and texture of the pen are analyzed.

[1110] Step 4:

[1111] The server generates monsters using a generative AI model based on the extracted feature information. This generation procedure uses an algorithm based on OpenAI's GPT model. The input is the feature information extracted in step 3, and the output is the data of the generated monster. For example, based on the pen, a long, thin monster is generated and given a special ability called an ink attack.

[1112] Step 5:

[1113] The server determines the status of the generated monster (HP, attack power, defense power, etc.). The input is the basic data of the generated monster, and the shape and color of the object are reflected in the status determination procedure. For example, a blue pen will be set as having high defense power. The output is the final status of the monster.

[1114] Step 6:

[1115] The server sends the data of the generated monster to the terminal. The input is the monster data determined in step 5, and this data is sent back to the terminal via the communication line. The output is the monster data received by the terminal.

[1116] Step 7:

[1117] The device visually displays the monster within the app based on the received monster data. Here, 3D display is performed in real time using the Unity engine. The input is the monster data received from the server, and the output is the monster visually displayed for the user. The user can check the monster's appearance and status within the app.

[1118] Step 8:

[1119] Users control the generated monsters to participate in the game's adventures and battle modes. The input is the user's actions and the monster's status data, and the output is the user's progress in the game. In single-player mode, users can strengthen their monsters, and in multiplayer mode, they can battle other users.

[1120] (Application example 1)

[1121] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1122] In modern society, an increasing number of people are seeking healthy meals and new cooking ideas. However, there are limited ways to easily access recipes based on everyday ingredients and information about the health benefits of each ingredient. This makes it difficult to easily plan meals that suit one's health condition and preferences.

[1123] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1124] In this invention, the server includes an image capturing means, an image analysis means, a generating means for generating a recipe and a healthy report based on the identified object, and a display means for displaying the generated recipe and healthy report, thereby enabling a user to easily capture an image of a food using a smartphone and instantly obtain healthy recipes and health information based on that food.

[1125] The "image capturing means" is a means for a user to capture an image of an object using a smartphone application.

[1126] The "image analysis means" is a means for analyzing a captured image and identifying an object in the image.

[1127] The "generation means" is a means for generating related recipes and healthy reports based on the identified object information.

[1128] The "display means" is a means for visually presenting the generated recipe and healthy report to the user.

[1129] The "status determining means" is a means for determining detailed recipe and health information based on characteristic information of the identified object.

[1130] The "service means" is a means for executing a service to provide recipes and healthy reports in response to a user's selection.

[1131] This invention allows a user to take a picture of a food using a smartphone application, and generates an original recipe and a healthy report based on the food. The system includes the following means.

[1132] 1. Use of Hardware and Software

[1133] Camera (hardware): Used to capture images.

[1134] OpenCV (software): A library for image processing.

[1135] Image recognition model (software): A model for analyzing objects in an image.

[1136] Python runtime environment (software): The infrastructure for running programs.

[1137] 2. Contents of data processing and data calculation

[1138] The user launches the smartphone application and takes a picture of the food with the camera.

[1139] The device processes the captured image using OpenCV and identifies the object using an image recognition model. The feature information of the identified object is then sent to the server.

[1140] The server operates algorithms to generate recipes and healthy reports based on the identified object information.

[1141] The terminal receives the recipe and healthy report sent from the server and displays them to the user.

[1142] 3. Specific Examples

[1143] User input: The user takes a picture of a tomato with their smartphone.

[1144] Output: The following information will be generated:

[1145] Recommended recipe: Tomato and basil pasta

[1146] Health Report: Rich in Vitamin C and has antioxidant properties.

[1147] 4. Examples of prompts

[1148] Prompt: "Extract features from the captured image and generate appropriate cooking recipes and health reports based on those features."

[1149] This system allows users to quickly and easily obtain health information about everyday ingredients and new recipe suggestions, helping them improve the quality of their diet.

[1150] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1151] Step 1:

[1152] A user starts the smartphone application and takes a picture of food with the camera. The user's input is the image of the food. This image data is input.

[1153] Step 2:

[1154] The device processes the captured image using OpenCV and identifies the object using an image recognition model. The input is the image captured by the user. Specifically, the image is preprocessed (resized, noise removed, etc.), and then the image recognition model is applied to identify the food. The output is feature information of the identified food.

[1155] Step 3:

[1156] The device sends the characteristic information of the identified food to the server. The input is the characteristic information of the identified food, and the output is a data packet containing that information. Specifically, the device's data transmission API is called to send the data to the server.

[1157] Step 4:

[1158] The server uses a means for generating recipes and health reports based on the received food characteristic information. The input is the food characteristic information sent from the terminal. Specifically, the server retrieves the corresponding recipe information from the database, analyzes health-related statistical data, and generates a health report. The output is the generated recipe and health report data.

[1159] Step 5:

[1160] The server sends the generated recipe and healthy report data to the terminal. The input is the generated recipe and healthy report data, and the output is a data packet. Specifically, the server calls the data transmission API to send the data to the terminal.

[1161] Step 6:

[1162] The terminal displays the recipe and healthy report data received from the server. The input is the recipe and healthy report data sent from the server. This data is visualized and displayed on the display in a format that is easy for the user to view. The output is visual information that the user can confirm.

[1163] This process flow allows users to easily take a picture of a food item using their smartphone and instantly obtain healthy recipes and health information based on that food.

[1164] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1165] A system embodying the present invention allows a user to photograph an object through a smartphone application and generate an original monster based on that object. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to generate monsters and provide a gaming experience that reflects the user's emotions. This system uses the following means:

[1166] System Configuration

[1167] The system includes (1) an image capturing means, (2) an image analyzing means, (3) a generating means, (4) a displaying means, (5) a status determining means, (6) a game playing means, and (7) an emotion engine.

[1168] Program processing

[1169] 1. Image capture method

[1170] The user starts the smartphone app and uses the camera function to take a picture of any object, such as a pen, stuffed animal, or piece of furniture.

[1171] 2. Image analysis methods

[1172] The device acquires the captured image data and sends the image data and the user's identification information (such as the user ID) to the server.

[1173] 3. Image Classification and Feature Extraction

[1174] The server analyzes the image data received from the device. Specifically, it uses computer vision technology to identify objects in the image and extract their features (shape, color, texture, etc.). For example, if a pen is photographed, the shape and color of the pen are identified.

[1175] 4. Monster Generation

[1176] The server inputs characteristic information into a monster generation algorithm based on the object identification results. The server then automatically generates the monster's appearance, status, and special abilities based on the input characteristic information. For example, based on the characteristics of a pen, it generates a long, thin monster with the ability to write.

[1177] 5. Emotion Recognition and Affect

[1178] The device's built-in emotion engine analyzes the user's facial expressions and voice in real time to identify the user's emotions (joy, sadness, anger, etc.).

[1179] 6. Reflecting monster emotions

[1180] The server receives emotional data from the emotion engine and changes the monster's appearance and status according to the emotion. For example, if the user is happy, the monster will be set to have bright colors and high attack power.

[1181] 7. Status check and display

[1182] The device receives the monster data sent from the server and visually displays it to the user, who can then check detailed information about the monster (appearance, status, special abilities, etc.) on their smartphone screen.

[1183] 8. Start playing the game

[1184] The user selects a game mode within the app, for example, adventure mode or battle mode.

[1185] 9. Data Acquisition Depending on the Mode

[1186] The terminal requests the server for the necessary data depending on the selected game mode. For example, if battle mode is selected, the terminal requests information about the opponent's monsters.

[1187] 10. Displaying battle scenes

[1188] The device displays the battle scene based on the battle data received from the server, and the user controls their own monster to fight.

[1189] 11. Calculation and notification of battle results

[1190] The server calculates the results of the battle and sends the results (win / loss, experience points and items gained, etc.) to the device.

[1191] 12. Display of battle results

[1192] The terminal displays the battle results received from the server to the user, allowing the user to check, for example, whether they won or lost, the items they acquired, and the experience points they gained.

[1193] Specific examples

[1194] For example, a user takes a photo of a pen on their desk with their smartphone. The device sends the image to a server, which analyzes the image and identifies the object as a "pen." The server then generates a monster with a long, thin shape based on the pen, and determines that the monster has special abilities such as "ink attack."

[1195] At the same time, the emotion engine recognizes that the user is smiling and sends that data to the server, which uses this information to make the monsters it generates more vibrant in color and increase their attack power.

[1196] The server sets the monster's status (for example, HP 100, attack power 70, defense power 30) and sends the data to the device. The device displays the monster to the user, and the user can enjoy adventures and battles using the monster.

[1197] This system allows users to use everyday objects as monsters in the game, and their own emotional state will affect their gaming experience, providing a more personalized experience.

[1198] The processing flow will be explained below.

[1199] Step 1:

[1200] The user starts the smartphone app and uses the camera function to take a picture of any object, such as a pen, stuffed animal, or piece of furniture.

[1201] Step 2:

[1202] The device acquires the captured image data and sends the image data and the user's identification information (such as the user ID) to the server.

[1203] Step 3:

[1204] The server analyzes the image data received from the device. Specifically, it uses computer vision technology to identify objects in the image and extract their features (shape, color, texture, etc.). For example, if a pen is photographed, the shape and color of the pen are identified.

[1205] Step 4:

[1206] The server inputs characteristic information into a monster generation algorithm based on the object identification results. The server then automatically generates the monster's appearance, status, and special abilities based on the input characteristic information. For example, based on the characteristics of a pen, it generates a long, thin monster with the ability to write.

[1207] Step 5:

[1208] The device's built-in emotion engine analyzes the user's facial expressions and voice in real time to identify the user's emotions (joy, sadness, anger, etc.).

[1209] Step 6:

[1210] The server receives emotional data from the emotion engine and changes the monster's appearance and status according to the emotion. For example, if the user is happy, the monster will be set to have bright colors and high attack power.

[1211] Step 7:

[1212] The server determines the stats (HP, attack power, defense power, etc.) of the generated monster. The stats are reflected in the characteristics of the object, such as its color and shape. For example, a red pen might have a high attack power.

[1213] Step 8:

[1214] The server sends the generated monster data (image of appearance, status, special abilities, etc.) to the terminal.

[1215] Step 9:

[1216] The device analyzes the monster data received from the server and visually displays it to the user, who can then check detailed information about the monster (appearance, status, special abilities, etc.) on their smartphone screen.

[1217] Step 10:

[1218] The user selects a game mode within the app, for example, adventure mode or battle mode.

[1219] Step 11:

[1220] The terminal requests the server for the necessary data depending on the selected game mode. For example, if battle mode is selected, the terminal requests information about the opponent's monsters.

[1221] Step 12:

[1222] The device displays the battle scene based on the battle data received from the server, and the user controls their own monster to fight.

[1223] Step 13:

[1224] The server calculates the results of the battle and sends the results (win / loss, experience points and items gained, etc.) to the device.

[1225] Step 14:

[1226] The terminal displays the battle results received from the server to the user, allowing the user to check, for example, whether they won or lost, the items they acquired, and the experience points they gained.

[1227] Through this series of steps, users can create original monsters based on everyday objects they photograph, and then enjoy adventures and battles with those monsters. Furthermore, the emotion engine reflects the user's emotions in the game experience, providing a more personalized experience.

[1228] Example 2

[1229] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1230] Modern gaming systems lack personalized elements that allow users to have individualized experiences. In particular, few systems allow users to create unique characters using everyday objects and reflect the user's emotional state in the game. Furthermore, there is a lack of technology to analyze users' emotions in real time and directly link the results to the game experience.

[1231] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1232] In this invention, the server includes an image analysis means for transmitting image data to the server, a feature extraction means for the server to analyze the image data and extract features, and a generation means for the server to generate a monster based on the extracted features. This allows a user to take pictures of everyday objects and generate a personalized monster, providing a unique game experience that reflects the user's emotions.

[1233] "Image capturing means" refers to a device or function that a user uses to capture an image of an object, and typically refers to the camera function of a smartphone, tablet, or the like.

[1234] The "image analysis means" is a device or function for analyzing captured image data and transmitting it to a server.

[1235] The "feature extraction means" refers to a technology or algorithm for analyzing and identifying features such as the shape, color, and texture of an object from the image data received by the server.

[1236] The "generation means" is a device or algorithm for generating a monster based on the extracted characteristic information.

[1237] "Display means" refers to a device or function for visually displaying the generated monsters and their status to the user, and typically refers to the display of a smartphone or tablet.

[1238] A "status determining means" is a device or algorithm for determining the status and capabilities of a monster based on the characteristics of an identified object.

[1239] The "emotion reflection means" is a device or function that uses an emotion engine to analyze the user's emotional state and change the monster's appearance or status based on the analysis results.

[1240] "Game play means" refers to a device or function for providing game play using monsters in accordance with the game mode (adventure mode, battle mode, etc.) selected by the user.

[1241] The "battle result calculation means" is a device or algorithm for calculating the battle result and transmitting the calculation result to the terminal.

[1242] An "emotion engine" is a technology or system that recognizes a user's emotions (joy, sadness, anger, etc.) in real time by analyzing the user's facial expressions and voice.

[1243] The present invention is a system that allows a user to take a photograph of an object using a smartphone app and generate an original character based on that object. This system uses the following means.

[1244] First, the user launches the smartphone app and uses the device's camera to take a photo of an object of their choice. This photo can be of an everyday object (e.g., a pen, a stuffed toy, or a piece of furniture). The device then acquires the captured image data and sends it to a server. This transmission uses a secure communication protocol (e.g., HTTPS).

[1245] The server then analyzes the image data received from the device. This analysis uses computer vision algorithms (e.g., OpenCV) and other image analysis techniques. As a result of analyzing the image data, features such as the shape, color, and texture of the object are extracted. For example, if a user takes a photo of a pen on a desk, the pen's long, slender shape and unique color are analyzed.

[1246] The server then uses the extracted features to generate a monster using a generative AI model (e.g., GAN, Transformer). This determines the monster's appearance, status, and special abilities. For example, the pen's features generate a long, thin, blue monster with an "ink attack" ability.

[1247] Furthermore, the device is equipped with an emotion engine (e.g., Microsoft Azure Emotion API) that analyzes the user's facial expressions and voice in real time to identify emotions (joy, sadness, anger, etc.). The identified emotion data is sent to a server, which then dynamically changes the monster's appearance and status based on that data. For example, if the user is smiling, the monster's colors become more vibrant and its attack power increases.

[1248] Finally, the device visually displays the monster information received from the server to the user. This display allows the user to check the appearance, status, and special abilities of the created monster. The user then selects a game mode available within the app (e.g., adventure mode or battle mode) and enjoys gameplay using the monster. In battle mode, the server calculates the battle results and sends them to the device. The device then displays the results to the user. This allows the user to check the outcome of the battle, the experience points and items acquired, etc.

[1249] As a concrete example, a user takes a photo of a pen on their desk and sends the data to a server. The server analyzes the pen's characteristics (long, thin shape, blue color, etc.) and generates a monster based on them. At the same time, the server analyzes the user's emotions (e.g., smiling) and changes the color and status of the monster. The user can then enjoy the game using the generated monster.

[1250] Prompt Sentence Examples

[1251] "Please tell me the steps to take a picture of a pen on a desk to generate a monster, and how to reflect the user's emotions on that monster."

[1252] In this way, the present invention allows users to create their own unique monsters using everyday objects, providing a personalized gaming experience that reflects their own emotional state.

[1253] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1254] Step 1:

[1255] A user launches a smartphone app and uses the camera function to take a photo of an object. The input here is the object selected by the user, and the output is the captured image data. The specific action is for the user to tap the camera button in the app to take a photo. For example, the user takes a photo of a pen on a desk with the camera.

[1256] Step 2:

[1257] The device acquires the captured image data and sends the image data and the user's identification information (such as the user ID) to the server. The input is the captured image data and the user ID, and the output is the data to be sent to the server. Specifically, the device sends the image data and the user ID to the server using the HTTPS protocol.

[1258] Step 3:

[1259] The server analyzes the image data it receives. The input is the image data sent from the device, and the output is the feature information (shape, color, texture, etc.) of the objects in the image. Specifically, it uses computer vision technology (e.g., OpenCV) to identify the objects in the image and extract their features. For example, it identifies that the pen is long and thin and blue in color.

[1260] Step 4:

[1261] The server generates monsters based on the feature information of the object. The input is the extracted feature information, and the output is the data of the generated monster (appearance, status, special abilities). Specifically, the feature information is input into a generative AI model (e.g., GAN, Transformer), and a monster is automatically generated. For example, a long, thin blue monster is generated and set to have the special ability of "ink attack."

[1262] Step 5:

[1263] The emotion engine installed on the device analyzes the user's facial expressions and voice in real time to identify the user's emotions. The input is the user's real-time facial expressions and voice, and the output is the identified emotion data. The specific operation is to use the emotion engine (e.g., Microsoft Azure Emotion API) to analyze emotions from the user's facial expressions. For example, if the user is smiling, the emotion is identified as "joy."

[1264] Step 6:

[1265] The server receives emotion data from the emotion engine and changes the monster's appearance and status according to that emotion. The input is emotion data, and the output is the modified monster data. Specifically, the server makes the monster's color more vivid and increases its attack power.

[1266] Step 7:

[1267] The device receives the monster data sent from the server and visually displays it to the user. The input is the monster data sent from the server, and the output is the monster information displayed to the user. Specifically, the device displays the monster's image and status information on the app screen.

[1268] Step 8:

[1269] The user selects a game mode within the app. The input is the mode selection made by the user, and the output is the selected mode information. The specific operation is that the user selects adventure mode or battle mode from the menu within the app.

[1270] Step 9:

[1271] The terminal requests the necessary data from the server according to the selected game mode. The input is the selected mode information, and the output is a request to the server. Specifically, when battle mode is selected, the terminal requests opponent monster information from the server.

[1272] Step 10:

[1273] The server sends the requested data to the terminal. The input is the request from the terminal, and the output is the data to be sent to the terminal (such as information about the opponent's monster). The specific operation is that the server collects the requested data and sends it to the terminal.

[1274] Step 11:

[1275] The device displays a battle scene based on the game data it receives. The input is the game data sent from the server, and the output is the displayed battle scene. Specifically, the device displays the battle scene on the app screen, and the user can control their own monster to fight.

[1276] Step 12:

[1277] The server calculates the battle results and sends the results to the device. The input is detailed battle data, and the output is the calculated results (win / lose, experience points, acquired items, etc.). Specifically, the server analyzes the progress of the battle and calculates the results.

[1278] Step 13:

[1279] The device receives the battle results from the server and displays them to the user. The input is the calculated battle result, and the output is the result information displayed to the user. Specifically, the device displays the battle results, experience points gained, items, etc. on the app screen.

[1280] Prompt Sentence Examples

[1281] "Please tell me the steps to take a picture of a pen on a desk to generate a monster, and how to reflect the user's emotions on that monster."

[1282] (Application example 2)

[1283] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1284] Conventional game systems cannot generate monsters based on objects photographed by the user, nor can they change the game experience based on the user's emotions. This limits the game experience for users, making it difficult to provide a personalized experience.

[1285] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an image capturing means, an image analysis means, a generation means, a display means, an emotion recognition means for recognizing the user's emotion, and an attribute change means for changing the characteristics of a monster based on the recognized emotion. This makes it possible to generate an original monster based on an object photographed by the user, and further to dynamically change the monster's characteristics and game experience according to the user's emotion.

[1286] The "image capturing means" is a function that allows a user to capture an image of an object using a smartphone, digital camera, or the like.

[1287] "Image analysis means" refers to technology for processing captured image data and identifying objects in the image.

[1288] "Generation means" refers to an algorithm or program for automatically generating monsters based on the results of image analysis.

[1289] "Display means" refers to a screen or display for visually presenting the generated monsters and their characteristics to the user.

[1290] "Emotion recognition means" refers to a function for analyzing and identifying emotions from the user's facial expressions, voice, etc.

[1291] The "characteristics changing means" refers to a mechanism for dynamically changing the characteristics (color, ability, status, etc.) of a monster based on the recognized user's emotions.

[1292] "Status determination means" refers to an algorithm or program for determining the status and abilities of a monster according to the characteristics of the identified object.

[1293] "Gameplay means" refers to the interactive features that allow users to enjoy adventures and battles using monsters they create.

[1294] The system embodying this invention allows users to take a photograph of an object through a smartphone application and generate an original monster based on that object. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, the system generates monsters and provides a gaming experience that responds to the user's emotions.

[1295] System Configuration

[1296] The system includes the following elements:

[1297] 1. Image capture method:

[1298] The user uses the camera function of their smartphone to take a picture of any object, such as a pen on a desk, a stuffed animal, or a piece of furniture.

[1299] 2. Image analysis methods:

[1300] The device captures the captured image data and sends it to a server, which uses computer vision technology to identify the object and extract its features (shape, color, texture, etc.).

[1301] 3. Generation means:

[1302] Based on the object identification results, the server inputs characteristic information into a monster generation algorithm to determine the appearance, status, and special abilities of the generated monster.

[1303] 4. Display means:

[1304] The device receives the data of the generated monster and displays it visually to the user, who can then check detailed information about the monster (appearance, status, special abilities, etc.) on their smartphone screen.

[1305] 5. Emotion recognition means:

[1306] The device's built-in emotion engine analyzes the user's facial expressions and voice in real time to identify the user's emotions (joy, sadness, anger, etc.).

[1307] 6. How to change characteristics:

[1308] The server receives emotional data from the emotion engine and changes the monster's appearance and status according to the emotion. For example, if the user is happy, the monster will be set to have bright colors and high attack power.

[1309] 7. Status Determination Method:

[1310] It contains algorithms and programs for determining monster status and abilities based on the characteristics of an object.

[1311] 8. Gameplay methods:

[1312] It provides interactive features for users to adventure and battle with monsters they create.

[1313] Program processing explanation

[1314] First, the user takes a picture of an object using the camera function of their smartphone. At this time, the smartphone camera (image capture means) acquires image data using the OpenCV library etc. and temporarily stores it. After that, the captured image data is sent to the server (image analysis means) via the requests library.

[1315] The server analyzes the received image data using computer vision technology to identify the object. This analysis uses machine learning models and libraries such as TensorFlow. The characteristic information of the identified object (shape, color, texture) is input into a monster generation algorithm (generation means), and the appearance and characteristics of the monster are automatically generated.

[1316] The device receives data about the generated monster from the server and displays it on the smartphone screen, allowing the user to check detailed information about the generated monster (display means).

[1317] At the same time, the emotion engine analyzes the user's facial expressions and voice to identify their emotions in real time. The analysis results are sent to the server, and the monster's characteristics are changed based on the recognized emotion (characteristic change means). For example, if the user is happy, the monster's color becomes brighter and its attack power increases.

[1318] Users can select adventure mode or battle mode within the app and enjoy the game using monsters. The gameplay means retrieves monster data and opponent information according to the selected mode from the server, and battle scenes and adventure scenes are displayed in real time.

[1319] Examples and prompts

[1320] For example, a user can take a photo of a coffee cup on their desk, and the app will analyze the image to generate a "drink character." If the user then smiles, the character will gain an "active" status and be used in games and AR experiences.

[1321] Prompt for the generative AI model:

[1322] Let the user take a photo of a coffee cup, analyze the image, and generate a "drink character." Then, if the user smiles, make the character more vibrant and animated.

[1323] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1324] Step 1:

[1325] A user takes a photo of an object using the camera function of a smartphone. The input is the image of the object that the user photographs, and the output is the captured image data. Specifically, the user activates the smartphone camera, frames the object, and presses the shutter button.

[1326] Step 2:

[1327] The device acquires the captured image data and temporarily stores it. At this time, the input is the captured image data, and the output is an image file saved in local storage. Specifically, the OpenCV library is used to acquire the image data and save it to a file.

[1328] Step 3:

[1329] The device sends stored image data to the server. The input is the image file stored in local storage, and the output is the image data sent to the server. Specifically, the requests library is used to upload the image file to the server as an HTTP request.

[1330] Step 4:

[1331] The server analyzes the image data it receives. The input is the image data sent to the server, and the output is the analyzed object's identification results and feature information. Specifically, it uses computer vision technology and machine learning models (e.g., TensorFlow) to analyze the image and extract features such as the object's shape, color, and texture.

[1332] Step 5:

[1333] Based on the object identification results, the server inputs feature information into a monster generation algorithm to generate a monster. The input is the feature information of the identified object, and the output is the data of the generated monster. Specifically, the monster's appearance, status, and special abilities are generated based on the feature information.

[1334] Step 6:

[1335] The device receives the generated monster data from the server and displays it visually to the user. The input is the monster data sent from the server, and the output is the monster information displayed on the smartphone screen. Specifically, the received data is rendered into a UI component for visual display.

[1336] Step 7:

[1337] The emotion engine installed on the device analyzes the user's facial expressions and voice in real time to identify the user's emotions. The input is the user's facial expressions and voice data, and the output is the identified emotion data. Specifically, emotions are analyzed using the EmotionRecognizer library.

[1338] Step 8:

[1339] The server receives emotion data from the emotion engine and changes the monster's appearance and status according to the emotion. The input is emotion data, and the output is updated monster characteristic data. Specifically, an algorithm is applied to dynamically change the monster's color and status based on the emotion data.

[1340] Step 9:

[1341] The device displays updated monster data to the user based on the emotion. The input is the updated monster characteristic data, and the output is the updated monster information visually displayed on the smartphone screen. Specifically, the updated data is reflected in the UI components and re-rendered.

[1342] Step 10:

[1343] The user selects a game mode within the app and begins gameplay. The input is the user's selected game mode, and the output is an interactive game scene corresponding to the selected mode. Specifically, depending on the game mode selected, the app loads and presents an adventure mode or battle mode scene to the user.

[1344] Step 11:

[1345] The device requests data corresponding to the selected game mode from the server and obtains the necessary information. The input is the selected game mode, and the output is information about monsters and opponents obtained from the server. Specifically, a data request is sent to the server using an HTTP request, and the necessary data is received as a response.

[1346] Step 12:

[1347] The server calculates the battle results and sends them to the device. The input is the log data generated during the battle, and the output is the battle results, such as victory or defeat and experience points gained. Specifically, each event during the battle is saved as a log, and an algorithm is applied to calculate the results based on that data.

[1348] Step 13:

[1349] The terminal displays the battle results received from the server to the user. The input is the battle result data, and the output is the result information that is visually displayed on the smartphone screen. Specifically, the received result data is passed to the UI component and displayed visually.

[1350] This processing step allows the user to generate a monster based on a photographed object and use that monster to enjoy a highly personalized gaming experience.

[1351] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1352] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1353] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1354] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1355] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1356] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1357] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1358] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1359] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1360] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1361] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1362] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1363] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1364] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1365] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1366] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1367] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1368] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1369] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1370] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1371] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1372] The following is further disclosed regarding the above embodiment.

[1373] (Claim 1)

[1374] Image capturing means;

[1375] image analysis means for identifying captured images;

[1376] A generating means for generating a monster based on the identified object;

[1377] a display means for displaying the generated monster;

[1378] A system including:

[1379] (Claim 2)

[1380] 10. The system of claim 1, further comprising a status determining means for determining the status and capabilities of the monster based on the characteristics of the identified object.

[1381] (Claim 3)

[1382] 10. The system of claim 1, further comprising gameplay means for providing adventure or battle using monsters depending on a mode selected by a user.

[1383] "Example 1"

[1384] (Claim 1)

[1385] Image capturing means;

[1386] a terminal that transmits the captured image to a server via a communication line;

[1387] image analysis means for analyzing the received image using computer vision technology and extracting feature information of the identified object;

[1388] A generation means for generating a monster using a generation AI model based on the feature information;

[1389] a display means for transmitting the generated monster to a terminal and visually displaying the monster;

[1390] A system including:

[1391] (Claim 2)

[1392] 10. The system of claim 1, further comprising a status determining means for determining a status and special ability of the monster based on the characteristics of the identified object.

[1393] (Claim 3)

[1394] 10. The system according to claim 1, further comprising a gameplay means for providing an adventure or battle using the generated monsters depending on a mode selected by the user.

[1395] "Application Example 1"

[1396] (Claim 1)

[1397] Image capturing means;

[1398] image analysis means for identifying captured images;

[1399] generating means for generating recipes and healthy reports based on the identified objects;

[1400] a display means for displaying the generated recipe and healthy report;

[1401] A system including:

[1402] (Claim 2)

[1403] 10. The system of claim 1, further comprising a status determining means for determining recipe and health information based on the characteristics of the identified object.

[1404] (Claim 3)

[1405] 10. The system of claim 1, further comprising a service means for providing recipes and healthy reports according to a mode selected by a user.

[1406] "Example 2: Combining Emotion Engines"

[1407] (Claim 1)

[1408] image capturing means for a user to capture an image;

[1409] An image analysis means for transmitting image data from the terminal to a server;

[1410] A feature extraction means for the server to analyze image data and extract features;

[1411] A generation means for generating a monster based on the extracted features by the server;

[1412] a display means for displaying the generated monster to the user;

[1413] A system including:

[1414] (Claim 2)

[1415] a status determination means for determining the status and capabilities of the monster based on the characteristics of the identified object;

[1416] An emotion reflection means for recognizing the user's emotions using an emotion engine and changing the monster's appearance and status according to the emotions;

[1417] The system of claim 1 further comprising:

[1418] (Claim 3)

[1419] gameplay means for providing adventure or battle using monsters depending on a mode selected by a user;

[1420] a battle result calculation means for the server to calculate the battle result and transmit the calculation result to the terminal;

[1421] The system of claim 1 further comprising:

[1422] "Application example 2 when combining emotion engines"

[1423] (Claim 1)

[1424] Image capturing means;

[1425] image analysis means for identifying captured images;

[1426] A generating means for generating a monster based on the identified object;

[1427] a display means for displaying the generated monster;

[1428] emotion recognition means for recognizing an emotion of a user;

[1429] a characteristic changing means for changing the characteristics of the monster based on the recognized emotion;

[1430] A system including:

[1431] (Claim 2)

[1432] 10. The system of claim 1, further comprising a status determining means for determining the status and capabilities of the monster based on the characteristics of the identified object.

[1433] (Claim 3)

[1434] 10. The system of claim 1, further comprising gameplay means for providing adventure or battle using monsters depending on a mode selected by a user. [Explanation of symbols]

[1435] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. Image capturing means; image analysis means for identifying captured images; A generating means for generating a monster based on the identified object; a display means for displaying the generated monster; A system including:

2. 2. The system of claim 1, further comprising a status determining means for determining the status and capabilities of the monster based on the characteristics of the identified object.

3. 10. The system according to claim 1, further comprising gameplay means for providing adventures or battles using monsters depending on a mode selected by a user.

Images