System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving user input data; means for generating a design prompt based on the received input data; means for transmitting the generated design prompt to an image generation means to generate a character image; and means for transmitting the generated character image back to a user device for display.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditionally, character design work has required significant costs and time, placing a heavy financial burden on small businesses and sole proprietors. Furthermore, when outsourcing to an external designer, quality and consistency of the design can become an issue, limiting the marketing activities of the company's own brand. [Means for solving the problem]

[0005] To solve this problem, the present invention provides a system including a means for receiving user-input data, a generating means for generating design prompts based on the received data, a means for sending the generated prompts to an image generating means to generate a character image, and a means for returning the generated image to a user device for display. This system creates design prompts by utilizing a natural language generation model based on the character type, features, color, and theme input by the user, and automatically generates character images based on the design prompts. This reduces expensive design costs and enables the creation of high-quality characters in a short period of time.

[0006] "User-input data" refers to data entered by the user regarding design requirements such as the character type, characteristics, color, and theme.

[0007] A "means for receiving" is a system component that allows the server to receive user input data.

[0008] A "generator" is a system component for creating design prompts based on received user input data.

[0009] A "design prompt" is text data that describes the specific characteristics and design direction of a character.

[0010] The "image generation means" is a system component that automatically generates a character image based on the generated design prompt.

[0011] A "character image" is an image file of a visual character generated by the image generation means based on the design prompt.

[0012] The "means for returning and displaying" is a system component for sending the generated character image from the server to the user device and displaying it.

[0013] A "natural language generation model" is an artificial intelligence model that automatically generates design prompts in a format that humans can understand based on user-input data. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The system of the present invention receives user input data, generates character design prompts based on the data, generates an image based on the prompts, and provides the generated character image to the user. This system is mainly composed of the following means:

[0036] Receiving User Input

[0037] Using a terminal, users can input detailed requirements for the character design, including the character's type, characteristics, color, theme, etc. For example, a pet supply company might input requirements such as "Shiba Inu character, lively personality, predominant color orange."

[0038] Generate Design Prompts

[0039] Requirements entered by the user are sent from the terminal to the server. The server receives this input data and generates a design prompt using a natural language generation model, which serves as a generation means. For example, a design prompt such as "Shiba Inu, energetic, orange character" may be generated based on the entered requirements.

[0040] Executing image generation

[0041] The server sends the generated design prompt to the image generation means, which reads the prompt and automatically generates a character image. Through this process, multiple character image ideas are generated in a short period of time.

[0042] Return and display of generated images

[0043] The generated character image is sent back to the terminal from the server. The terminal displays this character image to the user. The user can check the image and re-enter correction requirements if necessary. If no corrections are required, the user can save or download the final character image.

[0044] Specific examples

[0045] Scenario: A pet supply company (the user) wants to design a new dog character.

[0046] 1. "The user uses the terminal interface to input detailed requirements for a dog character (type: dog, breed: Shiba Inu, personality: energetic, primary color: orange)."

[0047] 2. "The terminal sends these input data to the server."

[0048] 3. "The server receives the user's input data and uses a natural language generation model to generate detailed design prompts. For example, 'A Shiba Inu, energetic, orange character.'"

[0049] 4. "The generated design prompt is sent by the server to the image generation means, and a character image is automatically generated."

[0050] 5. The generated character image is sent back to the device from the server and displayed to the user.

[0051] 6. "The user reviews the image and re-enters any corrections required, if necessary. Once the final image is confirmed, it is saved or downloaded."

[0052] In this way, by using the system of this invention, users can efficiently generate high-quality characters in a short period of time, reducing expensive design costs. This system can also be applied to the automatic creation of social media stamps and the design of avatars and items for the Metaverse market.

[0053] The processing flow will be explained below.

[0054] Step 1:

[0055] The user uses a dedicated interface on the device to input the character design requirements (type, characteristics, color, theme, etc.). For example, requirements such as "Shiba Inu character, lively personality, main color orange" can be input.

[0056] Step 2:

[0057] The device prepares to send the user's input data to the server. The input data is converted into a format such as JSON or XML and sent to the server.

[0058] Step 3:

[0059] The server receives the user-entered data, which is temporarily stored in a database or memory.

[0060] Step 4:

[0061] Based on the input data received by the server, a natural language generation model is invoked. The model generates detailed design prompts based on the user's requirements. For example, a prompt such as "Shiba Inu, energetic, orange character" may be generated.

[0062] Step 5:

[0063] The server sends the generated design prompts to the image generation means, which interprets the prompts using the image generation model and uses them to generate the character image.

[0064] Step 6:

[0065] The image generation means generates a character image based on the design prompt. The image generation model uses multiple neural networks to generate a character image that meets the user's requirements.

[0066] Step 7:

[0067] The generated character image is sent back from the image generating means to the server, which receives and temporarily stores this image.

[0068] Step 8:

[0069] The server sends the generated character image to the terminal, where it is converted into a format that the user can view (e.g., JPEG or PNG) and sent to the terminal.

[0070] Step 9:

[0071] The terminal displays the generated character image to the user, who can check the generated image and re-enter correction requirements as necessary.

[0072] Step 10:

[0073] If the user is satisfied with the image, they can save or download the final character image. If any corrections are needed, they will repeat the process from step 1.

[0074] Example 1

[0075] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0076] In conventional character design generation systems, even when users input specific design requirements, the generated character images often did not fully reflect the user's requests. Furthermore, the process for users to reflect revision requirements in the generated character images was inefficient, resulting in time-consuming and costly processes. This created the problem of being unable to generate, revise, and finalize character images quickly and efficiently.

[0077] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0078] In this invention, the server includes means for receiving user input data, means for generating design prompts based on the received input data, means for transmitting the generated design prompts to the image generating means and generating a plurality of character images, means for returning the generated character images to the user device and displaying them, and means for the user to input requirements for modifying the generated character images. This makes it possible to efficiently generate character images that reflect the specific requirements of the user and to efficiently perform the process of modifying the generated character images.

[0079] "User input data" refers to information such as the type, characteristics, color, and theme of a character that is input by a user through a terminal.

[0080] "Generation means" refers to a function within the system that generates design prompts based on user input data using a natural language generation model.

[0081] "Design prompt" refers to specific character design instructions that the generating means creates based on user input data.

[0082] "Image generation means" refers to a function within the system that analyzes design prompts and automatically generates character images based on them.

[0083] "User device" refers to a terminal operated by a user, and is an electronic device for displaying and checking the generated character image.

[0084] "Modification requirements" refer to specific design requirements that the user adds or changes as necessary after looking at the generated character image.

[0085] A "natural language generation model" is a type of artificial intelligence that has the ability to understand and generate language that humans naturally use, and refers to the technology used to generate prompts based on text data.

[0086] The system of the present invention receives user input data, generates character design prompts based on the data, generates an image based on the prompts, and provides the generated character image to the user. An embodiment of the system includes the following means.

[0087] Receiving User Input

[0088] Using a terminal, users can input detailed requirements for the character design. These requirements include the character's type, characteristics, color, theme, etc. For example, a pet supply company might input requirements such as "a Shiba Inu character with a lively personality and a primary color of orange." The terminal then sends this input data to a server. Terminals can be personal computers, smartphones, tablets, etc.

[0089] Generate Design Prompts

[0090] The server receives the user's input data sent from the device. The server uses a generative AI model (e.g., GPT-3 (registered trademark) or BERT) to generate a design prompt based on the input data. The prompt is in the form of a "Shiba Inu, energetic, orange character." A natural language generation model is used as the generation method, and an appropriate design prompt is generated based on the input data.

[0091] Executing image generation

[0092] The server sends the generated design prompt to an image generation tool (e.g., DALL-E or DeepArt). The image generation model analyzes the prompt and automatically generates a character image. Through this process, multiple character image ideas are generated in a short time. The server temporarily stores the generated image data.

[0093] Return and display of generated images

[0094] The generated character image is sent back to the terminal from the server. The terminal displays this character image to the user. The user can check the displayed image and re-enter correction requirements as necessary.

[0095] Specific examples

[0096] Take the example of a pet supply company (the user) wanting to design a new dog character.

[0097] 1. The user uses the terminal interface to input detailed requirements for the dog character (type: dog, breed: Shiba Inu, personality: energetic, main color: orange).

[0098] 2. The terminal sends the input data to the server.

[0099] 3. The server receives the user's input data and uses a generative AI model (e.g., GPT-3) to generate a detailed design prompt, such as "a Shiba Inu, energetic, orange character."

[0100] 4. The generated design prompt is sent by the server to an image generation means (for example, DALL-E), and a character image is automatically generated.

[0101] 5. The generated character image is sent back from the server to the device and displayed to the user.

[0102] 6. The user checks the image and re-enters any corrections if necessary. Once the final image is confirmed, it is saved or downloaded.

[0103] By using this system, users can efficiently generate high-quality character images in a short period of time, reducing design costs. This system can also be applied to the automatic creation of social media stamps and the design of avatars and items for the Metaverse market.

[0104] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0105] Step 1:

[0106] The user opens the terminal interface and inputs detailed requirements for the character design, including the character's type, characteristics, color, and theme. For example, the user might input "Type: Dog, Breed: Shiba Inu, Personality: Energetic, Main Color: Orange."

[0107] Input: Character type, traits, color, theme

[0108] Output: User input data

[0109] Step 2:

[0110] The terminal sends the data entered by the user to the server using the HTTPS protocol, and the data is encrypted during transmission.

[0111] Input: User-entered data

[0112] Output: The encrypted input data

[0113] Step 3:

[0114] The server receives and decrypts the encrypted data. Next, it uses a generative AI model (e.g., GPT-3) to generate design prompts based on the received data. For example, from the data "Type: Dog, Breed: Shiba Inu, Personality: Energetic, Primary Color: Orange," it generates the prompt "Shiba Inu, Energetic, Orange Character."

[0115] Input: Encrypted input data

[0116] Output: Design prompt

[0117] Step 4:

[0118] The server sends the generated design prompts to an image generation model (e.g., DALL-E), which analyzes the prompts and generates multiple character images based on them. The generated images are stored in the server's temporary storage.

[0119] Input: Design prompt

[0120] Output: Generated character image

[0121] Step 5:

[0122] The server converts the generated character image into an appropriate format (JPEG or PNG), compresses it if necessary, and then sends it to the terminal.

[0123] Input: Generated character image

[0124] Output: Converted and compressed character image

[0125] Step 6:

[0126] The terminal receives the character image sent from the server and displays it to the user. The user can check the displayed image and re-enter correction requirements if necessary. If no corrections are required, the user can save or download the image.

[0127] Input: Converted and compressed character image

[0128] Output: Character image that can be viewed by the user

[0129] Step 7 (if necessary):

[0130] If the user inputs modified requirements, the terminal retransmits the new design requirements to the server, and the processes from step 2 to step 6 are performed again.

[0131] Input: Correction Requirement

[0132] Output: New Design Requirements

[0133] In this way, each processing step works in conjunction with the others, allowing the user to generate a high-quality character image in a short period of time and even make corrections as needed.

[0134] (Application example 1)

[0135] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0136] Conventional character design systems have the problem that even if a user inputs detailed character requirements, they lack a means to immediately check the results in a virtual environment, limiting the user experience. Furthermore, re-entering revised requirements and determining the final character image requires multiple steps, resulting in low efficiency. The present invention aims to solve these problems by providing a system that quickly and efficiently generates high-quality characters and allows them to be checked in a virtual environment in real time.

[0137] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0138] In this invention, the server includes means for receiving user input data, means for generating design prompts based on the received input data, means for transmitting the generated design prompts to the image generation means to generate a character image, interface means for allowing the user to check the generated character image in real time within the virtual environment, means for allowing the user to re-input correction requirements, and means for saving or downloading the final character image, thereby enabling the user to quickly generate high-quality characters and check and correct them in real time within the virtual environment.

[0139] "User-input data" refers to data that the user inputs specific requirements such as the type, characteristics, color, and theme of the character.

[0140] A "generator" is a device used to generate design prompts based on user input data.

[0141] A "design prompt" is a command statement generated by the generating means for realizing a character design.

[0142] The "image generation means" is a means for generating a character image based on the design prompt.

[0143] A "user device" is a terminal device used by a user, on which a generated character image is displayed.

[0144] The "interface means" is a means for the user to check the character image generated in the virtual environment in real time.

[0145] The "correction requirements" are requirements that the user inputs in order to make corrections to the generated character image.

[0146] "Storage or download means" refers to a means for saving or downloading the final character image.

[0147] A "virtual environment" refers to a virtual space or interface that allows you to check and modify character images in real time.

[0148] The embodiment of the present invention is to build a system according to the following procedure, which specifically includes the entire process from receiving user input data to checking it in the virtual environment, and generating and correcting character images.

[0149] Program generation and natural language explanation

[0150] Receiving user-entered data

[0151] Users use a smartphone app to input detailed requirements for their character, including the character's type, characteristics, color, and theme. The smartphone app is developed using Flutter (registered trademark).

[0152] Generate Design Prompts

[0153] The received user input data is sent from the smartphone app to a server, which uses Flask (Python) to generate design prompts using a natural language generation model (TENSORFLOW (registered trademark) or PyTorch).

[0154] Executing image generation

[0155] The generated prompts are sent to an image generation engine such as Stable Diffusion, which generates character images based on the prompts. This generation process takes place on the server.

[0156] Check in a virtual environment

[0157] The generated character image is displayed in a virtual environment through an HMD (e.g., Oculus Quest 2) so that it can be viewed in real time. The user can view the character in this virtual environment and re-enter correction requirements as needed.

[0158] Revision and Finalization

[0159] When the user re-enters the correction requirements using the smartphone app, the server generates a new design prompt and sends it to the image generation engine. The final generated character image is sent to the user device (smartphone app).

[0160] Specific examples

[0161] A user types in, "I want a character who is a blue-haired wizard in a fantasy world and has a kind expression."

[0162] Example of a generated design prompt: "Fantasy wizard, blue hair, kind expression, wearing robes"

[0163] The final generated avatar will be designed based on this prompt.

[0164] Hardware and software used

[0165] Smartphone app: Flutter

[0166] Server side: Flask (Python)

[0167] Natural language generation model: TensorFlow, PyTorch

[0168] Image generation engine: Stable Diffusion

[0169] Virtual Environment Interface: Oculus Quest 2

[0170] This system allows users to quickly and efficiently generate high-quality characters and then review and modify them in real time in a virtual environment. This system is expected to have a wide range of applications, not just in the entertainment industry.

[0171] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0172] Step 1:

[0173] A user opens a smartphone app and inputs requirements for the character type, features, color, and theme. The input requirements are saved as user-input data.

[0174] Input: User requirements for character type, traits, color, and theme

[0175] Output: User input data

[0176] Step 2:

[0177] The smartphone app sends user-entered data to the server, which receives the data in JSON format and stores it for processing.

[0178] Input: User input data (JSON format)

[0179] Output: User-entered data stored on the server

[0180] Step 3:

[0181] The server generates design prompts based on the received user input data, utilizing natural language generation models (TensorFlow or PyTorch) to translate user requirements into text prompts.

[0182] Input: User-entered data stored on the server

[0183] Output: Design prompt

[0184] Step 4:

[0185] The server sends the generated design prompt to an image generator (such as Stable Diffusion), which reads the prompt and automatically generates a character image.

[0186] Input: Design prompt

[0187] Output: Generated character image

[0188] Step 5:

[0189] The server sends the generated character image to a smartphone app and an HMD (such as Oculus Quest 2), where it is displayed in real time in the virtual environment. The user can view the character within this environment.

[0190] Input: Generated character image

[0191] Output: Image of the character displayed in the virtual environment

[0192] Step 6:

[0193] The user checks the character image in the virtual environment and re-enters any necessary corrections through the smartphone app, which then sends the re-entered data to the server.

[0194] Input: Correction Requirement

[0195] Output: Re-entered modified requirements data

[0196] Step 7:

[0197] The server again generates design prompts and sends them to the image generator to generate a modified character image, and this process is repeated until the user is satisfied.

[0198] Input: Re-entered modified requirement data

[0199] Output: Modified character image

[0200] Step 8:

[0201] Once the final character image has been finalized, the server sends this image data to a smartphone app, where the user can save or download it.

[0202] Input: Final generated character image

[0203] Output: Saved or downloaded character image

[0204] By following these steps, users can efficiently generate high-quality characters and then check and modify them in real time in a virtual environment.

[0205] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0206] The system of the present invention generates more optimal character design proposals by combining not only user input data but also an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described below.

[0207] Receiving User Input

[0208] Using a terminal, users can input the design requirements for the character (type, characteristics, color, theme, etc.) For example, a pet supply company might input requirements such as "a Shiba Inu character with a lively personality and orange as the main color."

[0209] Use of emotion engine

[0210] When a user inputs design requirements, the emotion engine analyzes the user's emotions. This can be done using voice input and facial expression recognition technology. For example, when a user inputs a "cheerful character," the engine analyzes their intention from their tone of voice and facial expression.

[0211] Generate Design Prompts

[0212] The server receives the user's input data and the analysis results of the emotion engine. Based on this data, a natural language generation model is invoked to generate detailed design prompts. For example, a specific prompt such as "Shiba Inu, energetic, orange character" is generated.

[0213] Executing image generation

[0214] The generated design prompt is sent from the server to the image generation means, which reads the prompt and automatically generates a character image. Through this process, multiple character image ideas are generated in a short period of time.

[0215] Return and display of generated images

[0216] The generated character image is sent from the server to the terminal and displayed to the user. The user can check the generated image and re-enter correction requirements if necessary. If no corrections are required, the user can save or download the final character image.

[0217] Specific examples

[0218] Scenario: A pet supply company (the user) wants to design a new dog character.

[0219] 1. The user uses the terminal interface to input detailed requirements for a dog character (e.g., type: dog, breed: Shiba Inu, personality: energetic, main color: orange).

[0220] 2. The emotion engine analyzes the user's voice and facial expressions when inputting design requirements, such as a smile or bright tone to express a lively personality.

[0221] 3. The device sends the input data and the emotion engine's analysis results to the server.

[0222] 4. The server receives this data and uses a natural language generation model to generate detailed design prompts, such as "Shiba Inu, energetic, orange character."

[0223] 5. The generated design prompt is sent to an image generating means, and a character image is automatically generated.

[0224] 6. The generated character image is sent from the server to the device and displayed to the user.

[0225] 7. The user checks the generated image and re-enters any necessary corrections. Once the final image is confirmed, the user saves or downloads it.

[0226] This system enables more intuitive and effective character design that reflects the user's emotions, reducing expensive design costs and enabling high-quality character generation in a short period of time. The use of an emotion engine makes it possible to generate design prompts that more accurately reflect the user's intentions, making it particularly effective when character design that appeals to emotions is required.

[0227] The processing flow will be explained below.

[0228] Step 1:

[0229] The user inputs the character's design requirements (type, characteristics, color, theme) using a dedicated interface on the device. For example, the user might input "Shiba Inu character, lively personality, main color orange."

[0230] Step 2:

[0231] The device sends the user's input data to the emotion engine, which then analyzes the user's emotions using their voice input and facial expression recognition. For example, when the user says "I'm fine," the emotion is read from the tone of voice and facial expression.

[0232] Step 3:

[0233] The emotion engine returns the analysis results to the device. The analysis results include data indicating the user's emotions. For example, it may return information such as "The user is expressing a cheerful emotion."

[0234] Step 4:

[0235] The device sends the user input data and the analysis results of the emotion engine to the server, which then receives both the user's design requirements and emotions.

[0236] Step 5:

[0237] The server invokes a generation method (natural language generation model) based on the received data to generate a detailed design prompt. For example, a prompt such as "Shiba Inu, energetic, orange character" is generated.

[0238] Step 6:

[0239] The server transmits the generated design prompt to the image generating means, which reads the prompt and automatically generates a character image.

[0240] Step 7:

[0241] The image generating means returns the generated character image to the server, which receives and temporarily stores the image.

[0242] Step 8:

[0243] The server sends the generated character image to the terminal, where it is converted into a format that can be viewed by the user and sent to the terminal.

[0244] Step 9:

[0245] The device displays the generated character image to the user. The user can check the image and input corrections as needed. For example, they can input corrections such as "I want the expression to be a little brighter."

[0246] Step 10:

[0247] If the user is satisfied with the image, they can save or download the final character image. If any corrections are needed, they can repeat the process from step 1 again.

[0248] Step 11:

[0249] Once the user has finalized the character image, the system will save the image and make it available to the user for marketing and commercial use.

[0250] In this way, the system efficiently and effectively generates character designs that reflect the user's emotions. The introduction of an emotion engine makes it possible to generate design prompts that accurately reflect the user's intentions and emotions, resulting in results that are more satisfying for the user.

[0251] Example 2

[0252] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0253] In modern character design, there are only a limited number of systems that can accurately reflect the user's emotions and intentions. Conventional design systems rely solely on user input data, which often results in inaccurate reflection of the user's emotions and intentions. Furthermore, the design process is complex and time-consuming, making it difficult to generate high-quality characters quickly. This invention was developed to address these issues.

[0254] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0255] In this invention, the server includes means for receiving user input data, means for utilizing an emotion engine that analyzes the user's emotions, and means for generating design prompts based on the received input data and the analyzed emotion data, thereby enabling the rapid generation of high-quality character images that reflect the user's emotions and intentions.

[0256] "User input data" refers to design requirement information input by a user through a terminal for character creation, including the character's type, characteristics, color, and theme.

[0257] The "emotion engine" is a technology that analyzes the user's tone of voice and facial expressions when inputting information and recognizes the user's emotions in real time.

[0258] The "generator" is a system component for generating detailed design prompts based on received user input data and analyzed emotion data, primarily utilizing natural language generation models.

[0259] A "design prompt" is a set of instructions containing specific character design details that are generated based on user input data and emotion data.

[0260] The "image generation means" is a system component that automatically generates character images based on the generated design prompts. It mainly uses an image generation model.

[0261] "User device" refers to an electronic device used by a user to input data for character creation and to view the generated character image. Examples include personal computers and smartphones.

[0262] The system of the present invention generates intuitive and effective character design proposals by combining user input data with an emotion engine. A specific embodiment of this system will be described below.

[0263] Hardware and Software Configuration

[0264] Hardware used

[0265] User Device: Electronic devices such as computers, tablets, and smartphones.

[0266] Server: A cloud-based server with a powerful processor and lots of memory.

[0267] Software used

[0268] Emotion Engine: Speech and facial recognition software to analyze the user's tone of voice and facial expressions.

[0269] Generator: Software that generates design prompts using a natural language generation model (e.g., GPT-3).

[0270] Image generation means: Software that generates character images using an image generation model (e.g., DALL-E).

[0271] Details of the main process

[0272] Receiving User Input

[0273] The user inputs the character design requirements using the user device. For example, the user can specify details such as "Shiba Inu character, lively personality, main color is orange." The device receives this input data and temporarily stores it.

[0274] Use of emotion engine

[0275] When a user inputs design requirements, the emotion engine analyzes the user's voice and facial expressions in real time. For example, when a user inputs "a lively character," the engine analyzes their intention from the tone of their voice and facial expression. The device acquires this emotion data and saves the analysis results.

[0276] Generate Design Prompts

[0277] The device sends the user's input data and the emotion engine's analysis results to the server, which receives this data and uses a natural language generation model to generate detailed design prompts. For example, a specific prompt such as "Shiba Inu, energetic, orange character" is generated.

[0278] Executing image generation

[0279] The generated design prompt is sent from the server to the image generation means, which reads the prompt and automatically generates a character image. Through this process, multiple character image ideas are generated in a short period of time.

[0280] Return and display of generated images

[0281] The generated character images are sent from the server to the user device. The user device receives these images and displays them to the user. The user can review the generated images and re-enter correction requirements as needed. If no corrections are required, the user saves or downloads the final character image.

[0282] Specific examples

[0283] For example, if a pet supply company (user) wants to design a new dog character, the process would go something like this:

[0284] 1. The user uses the terminal interface to input detailed requirements for a dog character (e.g., type: dog, breed: Shiba Inu, personality: energetic, main color: orange).

[0285] 2. The emotion engine analyzes the voice and facial expressions used when users input design requirements, such as smiles and bright tones that express a lively personality.

[0286] 3. The device sends the input data and the emotion engine's analysis results to the server.

[0287] 4. The server receives this data and uses a natural language generation model to generate detailed design prompts, such as "Shiba Inu, energetic, orange character."

[0288] 5. The generated design prompt is sent to an image generating means, and a character image is automatically generated.

[0289] 6. The generated character image is sent from the server to the user device and displayed to the user.

[0290] 7. The user checks the generated image and re-enters any necessary corrections. Once the final image is confirmed, the user saves or downloads it.

[0291] Example prompt sentence:

[0292] "Shiba Inu, a lively, orange character"

[0293] This system enables more intuitive and effective character design that reflects the user's emotions, reducing expensive design costs and enabling high-quality character generation in a short period of time. Furthermore, the use of an emotion engine makes it possible to generate design prompts that more accurately reflect the user's intentions, making it particularly effective when character design is required to appeal to emotions.

[0294] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0295] Step 1: Receiving User Input

[0296] A user inputs design requirements (e.g., type, characteristics, color, theme) of a character using a user device. For example, the user inputs requirements such as "a Shiba Inu character with a lively personality and orange as the main color."

[0297] Input: Character type, traits, color, theme

[0298] Data processing: The device temporarily stores the entered data.

[0299] Output: Input design requirements data

[0300] Specific operation: The user enters "Dog breed: Shiba Inu," "Personality: Energetic," and "Main color: Orange" into the device's input form, and the device receives and temporarily stores this information.

[0301] Step 2: Use the Emotion Engine

[0302] As users input their design requirements, the emotion engine analyzes their tone of voice and facial expressions in real time.

[0303] Input: User's voice data, facial expression data

[0304] Data processing: The device sends the voice data and facial expression images to the analysis engine to obtain emotional data.

[0305] Output: User sentiment analysis results

[0306] Specific operation: When a user types "cheerful personality," the camera and microphone capture the tone of voice and facial expression, and the emotion engine analyzes that "the user has a cheerful emotion."

[0307] Step 3: Generate design prompts

[0308] The device sends the user input data and the emotion engine's analysis results to the server.

[0309] The server receives this data and uses natural language generation models to generate detailed design prompts.

[0310] Input: User input data and sentiment analysis results

[0311] Data processing: The server organizes this data and generates design prompts using a natural language generation model.

[0312] Output: Generated design prompts

[0313] Specific operation: Based on the data "Shiba Inu, lively personality, orange color, lively emotion," the server uses a natural language generation model to generate the prompt "Shiba Inu, lively, orange character."

[0314] Step 4: Running Image Generation

[0315] The server sends the generated design prompt to the image generating means.

[0316] An image generating means automatically generates a character image based on the design prompt.

[0317] Input: Generated design prompt

[0318] Data processing: The image generation means reads the design prompt and generates a character image using the image generation model.

[0319] Output: Multiple character image ideas

[0320] Specific operation: The server passes the prompt "Shiba Inu, energetic, orange character" to the image generation model, and multiple character images are automatically generated.

[0321] Step 5: Return and display the generated image

[0322] The server transmits the generated character image to the user device.

[0323] The terminal receives these images and displays them to the user.

[0324] Input: Generated character image

[0325] Data processing: The device receives the image data and prepares it for display to the user.

[0326] Output: Generated character image that can be viewed by the user

[0327] Specific operation: The server sends a character image to the terminal, and the terminal displays the image to the user.

[0328] Step 6: Re-enter the corrections and save the final image

[0329] The user can review the generated image and re-enter any necessary corrections. If no corrections are required, the user can save or download the final character image.

[0330] Input: User correction requirements or save / download requests

[0331] Data processing: The terminal sends the modified requirements back to the server, which generates a new design prompt.

[0332] Output: Revised character image or final character image

[0333] Specific operation: The user checks the character image and inputs any correction requirements, such as "I would like the color to be a little brighter." The device sends this information to the server, which then generates a new design prompt and generates a new character image. The user can then save or download the final image they are satisfied with.

[0334] (Application example 2)

[0335] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0336] Conventional character design systems generate character images based solely on user input data, which means they are unable to fully reflect the user's emotions and intentions. Furthermore, the quality and intuitive design of the generated characters often fall short of user satisfaction. This results in increased costs and time for character design, making it difficult to respond to situations where emotionally appealing characters are required.

[0337] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0338] In this invention, the server includes means for receiving user input data, means for generating design prompts based on the received input data and analyzed emotion data, means for transmitting the generated design prompts to an image generation means to generate a character image, and means for returning the generated character image to a user device for display. This enables the automatic generation of high-quality, intuitive character images that reflect the user's emotions and intentions.

[0339] "User-input data" refers to information containing specific requirements a user has for character generation, such as the character's category, attributes, primary color, and motif.

[0340] "Emotion data" is data obtained by analyzing the emotions and intentions expressed by users when designing characters. It is acquired using voice input and facial expression recognition technology.

[0341] "Design prompts" are instructions for character design that are generated based on user input data and emotion data. Images are generated based on these.

[0342] "Image generation means" refers to a system or software that reads design prompts and automatically generates character images, for example, using a generative AI model.

[0343] A "user device" is a terminal used by a user to display the generated character image, and includes, for example, mobile devices such as smartphones and tablets.

[0344] A "natural language generation model" is an AI technology that analyzes user-input data and generates design prompts in a format that is easy for humans to understand. A representative example is GPT-3.

[0345] An "emotion analysis engine" is a system that analyzes emotions from the user's voice, facial expressions, etc., and generates emotional data, which allows the user's intentions to be reflected more accurately.

[0346] The present invention relates to a system for generating a character image based on user input data and emotion data. Specific embodiments are described below.

[0347] Receiving User Input

[0348] A user uses a user device such as a smartphone or tablet to input character design requirements, including specific requirements such as the character's category (e.g., animal or human), attributes (e.g., friendliness or liveliness), primary color, and motif.

[0349] Emotional Data Analysis

[0350] When a user inputs design requirements, the emotion analysis engine uses voice input and facial expression recognition technology to analyze the user's emotions. This involves analyzing the tone of voice and facial expressions when the user inputs a "friendly character" and recording the emotional data.

[0351] Generate Design Prompts

[0352] The input data and emotion data sent from the user device are received by a server. The server uses a natural language generation model (e.g., GPT-3) to generate a design prompt based on the received data. The generated design prompt becomes a specific instruction sentence, such as "blue cat, friendly, smiling, simple design."

[0353] Executing image generation

[0354] The generated design prompts are sent from the server to an image generation means (e.g., a generative AI model such as DALL-E or Stable Diffusion). This image generation means reads the design prompts and automatically generates corresponding character images. Multiple image proposals are generated in a short period of time.

[0355] Return and display of generated images

[0356] The generated character image is sent back to the user's device via the server and displayed to the user. The user can check this image and re-enter correction requirements as necessary. When the user is finally satisfied with the character image, they can save or download it.

[0357] Specific example explanation

[0358] For example, if a user wants to design a blue cat character for a pet supplies online store, the following prompt sentence is generated:

[0359] Example prompt sentence:

[0360] "Blue cat, friendly, smiling, simple design"

[0361] This allows users to quickly obtain high-quality character images that reflect their intentions, reducing design costs and improving the efficiency of character generation.

[0362] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0363] Step 1:

[0364] The user inputs the character design requirements using the terminal.

[0365] Specific actions: Enter data such as the character's category, attributes, primary color, and motif in the terminal's application interface.

[0366] Input: Character design requirements (e.g. blue cat, friendly, smiling).

[0367] Output: The entered data is sent from the terminal to the server.

[0368] Step 2:

[0369] The server receives the user input data and analyzes the user's emotions using an emotion analysis engine.

[0370] Specific operation: The server receives the voice and facial expression data sent along with the input data and generates emotion data using an emotion analysis engine.

[0371] Input: User input data, voice data, facial expression data.

[0372] Output: Emotion data is generated and stored in the server.

[0373] Step 3:

[0374] A server generates design prompts based on the user input data and the emotion data.

[0375] Specific operation: Using a natural language generation model (e.g., GPT-3), analyze user input data and sentiment data to generate design prompts.

[0376] Input: User input data, emotion data.

[0377] Output: Design prompt (e.g., "Blue cat, friendly, smiling, simple design").

[0378] Step 4:

[0379] The server transmits the generated design prompt to the image generating means, which generates a character image.

[0380] Specific operations: Provide design prompts to an image generation means (e.g., DALL-E or Stable Diffusion) to generate a character image.

[0381] Input: Design prompt.

[0382] Output: Generated character image.

[0383] Step 5:

[0384] The server returns the generated character image to the terminal and displays it to the user.

[0385] Specific operation: The generated character image is transferred from the server to the terminal and displayed on the user device's display.

[0386] Input: Generated character image.

[0387] Output: The generated character image is displayed on the terminal.

[0388] Step 6:

[0389] The user checks the generated image and re-enters correction requirements if necessary.

[0390] Specific operation: The user checks the displayed character image, and if any corrections are necessary, corrects the input data and emotion data again and sends them.

[0391] Input: Modification requirements (e.g. change primary color, change attributes, etc.).

[0392] Output: The corrected input data is sent from the terminal to the server.

[0393] Step 7:

[0394] The server regenerates the design prompts based on the modified input data, updates the character image, and returns the final character image to the user for saving or download.

[0395] Specific operations: Emotion analysis and design prompt generation are performed again based on the corrected input data, the character image is updated using the image generation means, and the final character image is returned to the user.

[0396] Input: Modified input data, emotion data.

[0397] Output: The final character image is sent back to your device and can be saved or downloaded.

[0398] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0399] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0400] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0401] [Second embodiment]

[0402] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0403] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0404] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0405] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0406] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0407] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0408] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0409] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0410] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0411] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0412] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0413] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0414] The system of the present invention receives user input data, generates character design prompts based on the data, generates an image based on the prompts, and provides the generated character image to the user. This system is mainly composed of the following means:

[0415] Receiving User Input

[0416] Using a terminal, users can input detailed requirements for the character design, including the character's type, characteristics, color, theme, etc. For example, a pet supply company might input requirements such as "Shiba Inu character, lively personality, predominant color orange."

[0417] Generate Design Prompts

[0418] Requirements entered by the user are sent from the terminal to the server. The server receives this input data and generates a design prompt using a natural language generation model, which serves as a generation means. For example, a design prompt such as "Shiba Inu, energetic, orange character" may be generated based on the entered requirements.

[0419] Executing image generation

[0420] The server sends the generated design prompt to the image generation means, which reads the prompt and automatically generates a character image. Through this process, multiple character image ideas are generated in a short period of time.

[0421] Return and display of generated images

[0422] The generated character image is sent back to the terminal from the server. The terminal displays this character image to the user. The user can check the image and re-enter correction requirements if necessary. If no corrections are required, the user can save or download the final character image.

[0423] Specific examples

[0424] Scenario: A pet supply company (the user) wants to design a new dog character.

[0425] 1. "The user uses the terminal interface to input detailed requirements for a dog character (type: dog, breed: Shiba Inu, personality: energetic, primary color: orange)."

[0426] 2. "The terminal sends these input data to the server."

[0427] 3. "The server receives the user's input data and uses a natural language generation model to generate detailed design prompts. For example, 'A Shiba Inu, energetic, orange character.'"

[0428] 4. "The generated design prompt is sent by the server to the image generation means, and a character image is automatically generated."

[0429] 5. The generated character image is sent back to the device from the server and displayed to the user.

[0430] 6. "The user reviews the image and re-enters any corrections required, if necessary. Once the final image is confirmed, it is saved or downloaded."

[0431] In this way, by using the system of this invention, users can efficiently generate high-quality characters in a short period of time, reducing expensive design costs. This system can also be applied to the automatic creation of social media stamps and the design of avatars and items for the Metaverse market.

[0432] The processing flow will be explained below.

[0433] Step 1:

[0434] The user uses a dedicated interface on the device to input the character design requirements (type, characteristics, color, theme, etc.). For example, requirements such as "Shiba Inu character, lively personality, main color orange" can be input.

[0435] Step 2:

[0436] The device prepares to send the user's input data to the server. The input data is converted into a format such as JSON or XML and sent to the server.

[0437] Step 3:

[0438] The server receives the user-entered data, which is temporarily stored in a database or memory.

[0439] Step 4:

[0440] Based on the input data received by the server, a natural language generation model is invoked. The model generates detailed design prompts based on the user's requirements. For example, a prompt such as "Shiba Inu, energetic, orange character" may be generated.

[0441] Step 5:

[0442] The server sends the generated design prompts to the image generation means, which interprets the prompts using the image generation model and uses them to generate the character image.

[0443] Step 6:

[0444] The image generation means generates a character image based on the design prompt. The image generation model uses multiple neural networks to generate a character image that meets the user's requirements.

[0445] Step 7:

[0446] The generated character image is sent back from the image generating means to the server, which receives and temporarily stores this image.

[0447] Step 8:

[0448] The server sends the generated character image to the terminal, where it is converted into a format that the user can view (e.g., JPEG or PNG) and sent to the terminal.

[0449] Step 9:

[0450] The terminal displays the generated character image to the user, who can check the generated image and re-enter correction requirements as necessary.

[0451] Step 10:

[0452] If the user is satisfied with the image, they can save or download the final character image. If any corrections are needed, they will repeat the process from step 1.

[0453] Example 1

[0454] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0455] In conventional character design generation systems, even when users input specific design requirements, the generated character images often did not fully reflect the user's requests. Furthermore, the process for users to reflect revision requirements in the generated character images was inefficient, resulting in time-consuming and costly processes. This created the problem of being unable to generate, revise, and finalize character images quickly and efficiently.

[0456] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0457] In this invention, the server includes means for receiving user input data, means for generating design prompts based on the received input data, means for transmitting the generated design prompts to the image generating means and generating a plurality of character images, means for returning the generated character images to the user device and displaying them, and means for the user to input requirements for modifying the generated character images. This makes it possible to efficiently generate character images that reflect the specific requirements of the user and to efficiently perform the process of modifying the generated character images.

[0458] "User input data" refers to information such as the type, characteristics, color, and theme of a character that is input by a user through a terminal.

[0459] "Generation means" refers to a function within the system that generates design prompts based on user input data using a natural language generation model.

[0460] "Design prompt" refers to specific character design instructions that the generating means creates based on user input data.

[0461] "Image generation means" refers to a function within the system that analyzes design prompts and automatically generates character images based on them.

[0462] "User device" refers to a terminal operated by a user, and is an electronic device for displaying and checking the generated character image.

[0463] "Modification requirements" refer to specific design requirements that the user adds or changes as necessary after looking at the generated character image.

[0464] A "natural language generation model" is a type of artificial intelligence that has the ability to understand and generate language that humans naturally use, and refers to the technology used to generate prompts based on text data.

[0465] The system of the present invention receives user input data, generates character design prompts based on the data, generates an image based on the prompts, and provides the generated character image to the user. An embodiment of the system includes the following means.

[0466] Receiving User Input

[0467] Using a terminal, users can input detailed requirements for the character design. These requirements include the character's type, characteristics, color, theme, etc. For example, a pet supply company might input requirements such as "a Shiba Inu character with a lively personality and a primary color of orange." The terminal then sends this input data to a server. Terminals can be personal computers, smartphones, tablets, etc.

[0468] Generate Design Prompts

[0469] The server receives the user's input data sent from the device. The server uses a generative AI model (such as GPT-3 or BERT) to generate a design prompt based on the input data. The prompt is in the form of a "Shiba Inu, Genki, Orange character." A natural language generation model is used as the generation method, and an appropriate design prompt is generated based on the input data.

[0470] Executing image generation

[0471] The server sends the generated design prompt to an image generation tool (e.g., DALL-E or DeepArt). The image generation model analyzes the prompt and automatically generates a character image. Through this process, multiple character image ideas are generated in a short time. The server temporarily stores the generated image data.

[0472] Return and display of generated images

[0473] The generated character image is sent back to the terminal from the server. The terminal displays this character image to the user. The user can check the displayed image and re-enter correction requirements as necessary.

[0474] Specific examples

[0475] Take the example of a pet supply company (the user) wanting to design a new dog character.

[0476] 1. The user uses the terminal interface to input detailed requirements for the dog character (type: dog, breed: Shiba Inu, personality: energetic, main color: orange).

[0477] 2. The terminal sends the input data to the server.

[0478] 3. The server receives the user's input data and uses a generative AI model (e.g., GPT-3) to generate a detailed design prompt, such as "a Shiba Inu, energetic, orange character."

[0479] 4. The generated design prompt is sent by the server to an image generation means (for example, DALL-E), and a character image is automatically generated.

[0480] 5. The generated character image is sent back from the server to the device and displayed to the user.

[0481] 6. The user checks the image and re-enters any corrections if necessary. Once the final image is confirmed, it is saved or downloaded.

[0482] By using this system, users can efficiently generate high-quality character images in a short period of time, reducing design costs. This system can also be applied to the automatic creation of social media stamps and the design of avatars and items for the Metaverse market.

[0483] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0484] Step 1:

[0485] The user opens the terminal interface and inputs detailed requirements for the character design, including the character's type, characteristics, color, and theme. For example, the user might input "Type: Dog, Breed: Shiba Inu, Personality: Energetic, Main Color: Orange."

[0486] Input: Character type, traits, color, theme

[0487] Output: User input data

[0488] Step 2:

[0489] The terminal sends the data entered by the user to the server using the HTTPS protocol, and the data is encrypted during transmission.

[0490] Input: User-entered data

[0491] Output: The encrypted input data

[0492] Step 3:

[0493] The server receives and decrypts the encrypted data. Next, it uses a generative AI model (e.g., GPT-3) to generate design prompts based on the received data. For example, from the data "Type: Dog, Breed: Shiba Inu, Personality: Energetic, Primary Color: Orange," it generates the prompt "Shiba Inu, Energetic, Orange Character."

[0494] Input: Encrypted input data

[0495] Output: Design prompt

[0496] Step 4:

[0497] The server sends the generated design prompts to an image generation model (e.g., DALL-E), which analyzes the prompts and generates multiple character images based on them. The generated images are stored in the server's temporary storage.

[0498] Input: Design prompt

[0499] Output: Generated character image

[0500] Step 5:

[0501] The server converts the generated character image into an appropriate format (JPEG or PNG), compresses it if necessary, and then sends it to the terminal.

[0502] Input: Generated character image

[0503] Output: Converted and compressed character image

[0504] Step 6:

[0505] The terminal receives the character image sent from the server and displays it to the user. The user can check the displayed image and re-enter correction requirements if necessary. If no corrections are required, the user can save or download the image.

[0506] Input: Converted and compressed character image

[0507] Output: Character image that can be viewed by the user

[0508] Step 7 (if necessary):

[0509] If the user inputs modified requirements, the terminal retransmits the new design requirements to the server, and the processes from step 2 to step 6 are performed again.

[0510] Input: Correction Requirement

[0511] Output: New Design Requirements

[0512] In this way, each processing step works in conjunction with the others, allowing the user to generate a high-quality character image in a short period of time and even make corrections as needed.

[0513] (Application example 1)

[0514] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0515] Conventional character design systems have the problem that even if a user inputs detailed character requirements, they lack a means to immediately check the results in a virtual environment, limiting the user experience. Furthermore, re-entering revised requirements and determining the final character image requires multiple steps, resulting in low efficiency. The present invention aims to solve these problems by providing a system that quickly and efficiently generates high-quality characters and allows them to be checked in a virtual environment in real time.

[0516] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0517] In this invention, the server includes means for receiving user input data, means for generating design prompts based on the received input data, means for transmitting the generated design prompts to the image generation means to generate a character image, interface means for allowing the user to check the generated character image in real time within the virtual environment, means for allowing the user to re-input correction requirements, and means for saving or downloading the final character image, thereby enabling the user to quickly generate high-quality characters and check and correct them in real time within the virtual environment.

[0518] "User-input data" refers to data that the user inputs specific requirements such as the type, characteristics, color, and theme of the character.

[0519] A "generator" is a device used to generate design prompts based on user input data.

[0520] A "design prompt" is a command statement generated by the generating means for realizing a character design.

[0521] The "image generation means" is a means for generating a character image based on the design prompt.

[0522] A "user device" is a terminal device used by a user, on which a generated character image is displayed.

[0523] The "interface means" is a means for the user to check the character image generated in the virtual environment in real time.

[0524] The "correction requirements" are requirements that the user inputs in order to make corrections to the generated character image.

[0525] "Storage or download means" refers to a means for saving or downloading the final character image.

[0526] A "virtual environment" refers to a virtual space or interface that allows you to check and modify character images in real time.

[0527] The embodiment of the present invention is to build a system according to the following procedure, which specifically includes the entire process from receiving user input data to checking it in the virtual environment, and generating and correcting character images.

[0528] Program generation and natural language explanation

[0529] Receiving user-entered data

[0530] Users use a smartphone app to input detailed requirements for their character, including the character's type, characteristics, color, and theme. The smartphone app is developed using Flutter.

[0531] Generate Design Prompts

[0532] The received user input data is sent from the smartphone app to a server, which uses Flask (Python) to generate design prompts using a natural language generation model (TensorFlow or PyTorch).

[0533] Executing image generation

[0534] The generated prompts are sent to an image generation engine such as Stable Diffusion, which generates character images based on the prompts. This generation process takes place on the server.

[0535] Check in a virtual environment

[0536] The generated character image is displayed in a virtual environment through an HMD (e.g., Oculus Quest 2) so that it can be viewed in real time. The user can view the character in this virtual environment and re-enter correction requirements as needed.

[0537] Revision and Finalization

[0538] When the user re-enters the correction requirements using the smartphone app, the server generates a new design prompt and sends it to the image generation engine. The final generated character image is sent to the user device (smartphone app).

[0539] Specific examples

[0540] A user types in, "I want a character who is a blue-haired wizard in a fantasy world and has a kind expression."

[0541] Example of a generated design prompt: "Fantasy wizard, blue hair, kind expression, wearing robes"

[0542] The final generated avatar will be designed based on this prompt.

[0543] Hardware and software used

[0544] Smartphone app: Flutter

[0545] Server side: Flask (Python)

[0546] Natural language generation model: TensorFlow, PyTorch

[0547] Image generation engine: Stable Diffusion

[0548] Virtual Environment Interface: Oculus Quest 2

[0549] This system allows users to quickly and efficiently generate high-quality characters and then review and modify them in real time in a virtual environment. This system is expected to have a wide range of applications, not just in the entertainment industry.

[0550] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0551] Step 1:

[0552] A user opens a smartphone app and inputs requirements for the character type, features, color, and theme. The input requirements are saved as user-input data.

[0553] Input: User requirements for character type, traits, color, and theme

[0554] Output: User input data

[0555] Step 2:

[0556] The smartphone app sends user-entered data to the server, which receives the data in JSON format and stores it for processing.

[0557] Input: User input data (JSON format)

[0558] Output: User-entered data stored on the server

[0559] Step 3:

[0560] The server generates design prompts based on the received user input data, utilizing natural language generation models (TensorFlow or PyTorch) to translate user requirements into text prompts.

[0561] Input: User-entered data stored on the server

[0562] Output: Design prompt

[0563] Step 4:

[0564] The server sends the generated design prompt to an image generator (such as Stable Diffusion), which reads the prompt and automatically generates a character image.

[0565] Input: Design prompt

[0566] Output: Generated character image

[0567] Step 5:

[0568] The server sends the generated character image to a smartphone app and an HMD (such as Oculus Quest 2), where it is displayed in real time in the virtual environment. The user can view the character within this environment.

[0569] Input: Generated character image

[0570] Output: Image of the character displayed in the virtual environment

[0571] Step 6:

[0572] The user checks the character image in the virtual environment and re-enters any necessary corrections through the smartphone app, which then sends the re-entered data to the server.

[0573] Input: Correction Requirement

[0574] Output: Re-entered modified requirements data

[0575] Step 7:

[0576] The server again generates design prompts and sends them to the image generator to generate a modified character image, and this process is repeated until the user is satisfied.

[0577] Input: Re-entered modified requirement data

[0578] Output: Modified character image

[0579] Step 8:

[0580] Once the final character image has been finalized, the server sends this image data to a smartphone app, where the user can save or download it.

[0581] Input: Final generated character image

[0582] Output: Saved or downloaded character image

[0583] By following these steps, users can efficiently generate high-quality characters and then check and modify them in real time in a virtual environment.

[0584] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0585] The system of the present invention generates more optimal character design proposals by combining not only user input data but also an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described below.

[0586] Receiving User Input

[0587] Using a terminal, users can input the design requirements for the character (type, characteristics, color, theme, etc.) For example, a pet supply company might input requirements such as "a Shiba Inu character with a lively personality and orange as the main color."

[0588] Use of emotion engine

[0589] When a user inputs design requirements, the emotion engine analyzes the user's emotions. This can be done using voice input and facial expression recognition technology. For example, when a user inputs a "cheerful character," the engine analyzes their intention from their tone of voice and facial expression.

[0590] Generate Design Prompts

[0591] The server receives the user's input data and the analysis results of the emotion engine. Based on this data, a natural language generation model is invoked to generate detailed design prompts. For example, a specific prompt such as "Shiba Inu, energetic, orange character" is generated.

[0592] Executing image generation

[0593] The generated design prompt is sent from the server to the image generation means, which reads the prompt and automatically generates a character image. Through this process, multiple character image ideas are generated in a short period of time.

[0594] Return and display of generated images

[0595] The generated character image is sent from the server to the terminal and displayed to the user. The user can check the generated image and re-enter correction requirements if necessary. If no corrections are required, the user can save or download the final character image.

[0596] Specific examples

[0597] Scenario: A pet supply company (the user) wants to design a new dog character.

[0598] 1. The user uses the terminal interface to input detailed requirements for a dog character (e.g., type: dog, breed: Shiba Inu, personality: energetic, main color: orange).

[0599] 2. The emotion engine analyzes the user's voice and facial expressions when inputting design requirements, such as a smile or bright tone to express a lively personality.

[0600] 3. The device sends the input data and the emotion engine's analysis results to the server.

[0601] 4. The server receives this data and uses a natural language generation model to generate detailed design prompts, such as "Shiba Inu, energetic, orange character."

[0602] 5. The generated design prompt is sent to an image generating means, and a character image is automatically generated.

[0603] 6. The generated character image is sent from the server to the device and displayed to the user.

[0604] 7. The user checks the generated image and re-enters any necessary corrections. Once the final image is confirmed, the user saves or downloads it.

[0605] This system enables more intuitive and effective character design that reflects the user's emotions, reducing expensive design costs and enabling high-quality character generation in a short period of time. The use of an emotion engine makes it possible to generate design prompts that more accurately reflect the user's intentions, making it particularly effective when character design that appeals to emotions is required.

[0606] The processing flow will be explained below.

[0607] Step 1:

[0608] The user inputs the character's design requirements (type, characteristics, color, theme) using a dedicated interface on the device. For example, the user might input "Shiba Inu character, lively personality, main color orange."

[0609] Step 2:

[0610] The device sends the user's input data to the emotion engine, which then analyzes the user's emotions using their voice input and facial expression recognition. For example, when the user says "I'm fine," the emotion is read from the tone of voice and facial expression.

[0611] Step 3:

[0612] The emotion engine returns the analysis results to the device. The analysis results include data indicating the user's emotions. For example, it may return information such as "The user is expressing a cheerful emotion."

[0613] Step 4:

[0614] The device sends the user input data and the analysis results of the emotion engine to the server, which then receives both the user's design requirements and emotions.

[0615] Step 5:

[0616] The server invokes a generation method (natural language generation model) based on the received data to generate a detailed design prompt. For example, a prompt such as "Shiba Inu, energetic, orange character" is generated.

[0617] Step 6:

[0618] The server transmits the generated design prompt to the image generating means, which reads the prompt and automatically generates a character image.

[0619] Step 7:

[0620] The image generating means returns the generated character image to the server, which receives and temporarily stores the image.

[0621] Step 8:

[0622] The server sends the generated character image to the terminal, where it is converted into a format that can be viewed by the user and sent to the terminal.

[0623] Step 9:

[0624] The device displays the generated character image to the user. The user can check the image and input corrections as needed. For example, they can input corrections such as "I want the expression to be a little brighter."

[0625] Step 10:

[0626] If the user is satisfied with the image, they can save or download the final character image. If any corrections are needed, they can repeat the process from step 1 again.

[0627] Step 11:

[0628] Once the user has finalized the character image, the system will save the image and make it available to the user for marketing and commercial use.

[0629] In this way, the system efficiently and effectively generates character designs that reflect the user's emotions. The introduction of an emotion engine makes it possible to generate design prompts that accurately reflect the user's intentions and emotions, resulting in results that are more satisfying for the user.

[0630] Example 2

[0631] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0632] In modern character design, there are only a limited number of systems that can accurately reflect the user's emotions and intentions. Conventional design systems rely solely on user input data, which often results in inaccurate reflection of the user's emotions and intentions. Furthermore, the design process is complex and time-consuming, making it difficult to generate high-quality characters quickly. This invention was developed to address these issues.

[0633] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0634] In this invention, the server includes means for receiving user input data, means for utilizing an emotion engine that analyzes the user's emotions, and means for generating design prompts based on the received input data and the analyzed emotion data, thereby enabling the rapid generation of high-quality character images that reflect the user's emotions and intentions.

[0635] "User input data" refers to design requirement information input by a user through a terminal for character creation, including the character's type, characteristics, color, and theme.

[0636] The "emotion engine" is a technology that analyzes the user's tone of voice and facial expressions when inputting information and recognizes the user's emotions in real time.

[0637] The "generator" is a system component for generating detailed design prompts based on received user input data and analyzed emotion data, primarily utilizing natural language generation models.

[0638] A "design prompt" is a set of instructions containing specific character design details that are generated based on user input data and emotion data.

[0639] The "image generation means" is a system component that automatically generates character images based on the generated design prompts. It mainly uses an image generation model.

[0640] "User device" refers to an electronic device used by a user to input data for character creation and to view the generated character image. Examples include personal computers and smartphones.

[0641] The system of the present invention generates intuitive and effective character design proposals by combining user input data with an emotion engine. A specific embodiment of this system will be described below.

[0642] Hardware and Software Configuration

[0643] Hardware used

[0644] User Device: Electronic devices such as computers, tablets, and smartphones.

[0645] Server: A cloud-based server with a powerful processor and lots of memory.

[0646] Software used

[0647] Emotion Engine: Speech and facial recognition software to analyze the user's tone of voice and facial expressions.

[0648] Generator: Software that generates design prompts using a natural language generation model (e.g., GPT-3).

[0649] Image generation means: Software that generates character images using an image generation model (e.g., DALL-E).

[0650] Details of the main process

[0651] Receiving User Input

[0652] The user inputs the character design requirements using the user device. For example, the user can specify details such as "Shiba Inu character, lively personality, main color is orange." The device receives this input data and temporarily stores it.

[0653] Use of emotion engine

[0654] When a user inputs design requirements, the emotion engine analyzes the user's voice and facial expressions in real time. For example, when a user inputs "a lively character," the engine analyzes their intention from the tone of their voice and facial expression. The device acquires this emotion data and saves the analysis results.

[0655] Generate Design Prompts

[0656] The device sends the user's input data and the emotion engine's analysis results to the server, which receives this data and uses a natural language generation model to generate detailed design prompts. For example, a specific prompt such as "Shiba Inu, energetic, orange character" is generated.

[0657] Executing image generation

[0658] The generated design prompt is sent from the server to the image generation means, which reads the prompt and automatically generates a character image. Through this process, multiple character image ideas are generated in a short period of time.

[0659] Return and display of generated images

[0660] The generated character images are sent from the server to the user device. The user device receives these images and displays them to the user. The user can review the generated images and re-enter correction requirements as needed. If no corrections are required, the user saves or downloads the final character image.

[0661] Specific examples

[0662] For example, if a pet supply company (user) wants to design a new dog character, the process would go something like this:

[0663] 1. The user uses the terminal interface to input detailed requirements for a dog character (e.g., type: dog, breed: Shiba Inu, personality: energetic, main color: orange).

[0664] 2. The emotion engine analyzes the voice and facial expressions used when users input design requirements, such as smiles and bright tones that express a lively personality.

[0665] 3. The device sends the input data and the emotion engine's analysis results to the server.

[0666] 4. The server receives this data and uses a natural language generation model to generate detailed design prompts, such as "Shiba Inu, energetic, orange character."

[0667] 5. The generated design prompt is sent to an image generating means, and a character image is automatically generated.

[0668] 6. The generated character image is sent from the server to the user device and displayed to the user.

[0669] 7. The user checks the generated image and re-enters any necessary corrections. Once the final image is confirmed, the user saves or downloads it.

[0670] Example prompt sentence:

[0671] "Shiba Inu, a lively, orange character"

[0672] This system enables more intuitive and effective character design that reflects the user's emotions, reducing expensive design costs and enabling high-quality character generation in a short period of time. Furthermore, the use of an emotion engine makes it possible to generate design prompts that more accurately reflect the user's intentions, making it particularly effective when character design is required to appeal to emotions.

[0673] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0674] Step 1: Receiving User Input

[0675] A user inputs design requirements (e.g., type, characteristics, color, theme) of a character using a user device. For example, the user inputs requirements such as "a Shiba Inu character with a lively personality and orange as the main color."

[0676] Input: Character type, traits, color, theme

[0677] Data processing: The device temporarily stores the entered data.

[0678] Output: Input design requirements data

[0679] Specific operation: The user enters "Dog breed: Shiba Inu," "Personality: Energetic," and "Main color: Orange" into the device's input form, and the device receives and temporarily stores this information.

[0680] Step 2: Use the Emotion Engine

[0681] As users input their design requirements, the emotion engine analyzes their tone of voice and facial expressions in real time.

[0682] Input: User's voice data, facial expression data

[0683] Data processing: The device sends the voice data and facial expression images to the analysis engine to obtain emotional data.

[0684] Output: User sentiment analysis results

[0685] Specific operation: When a user types "cheerful personality," the camera and microphone capture the tone of voice and facial expression, and the emotion engine analyzes that "the user has a cheerful emotion."

[0686] Step 3: Generate design prompts

[0687] The device sends the user input data and the emotion engine's analysis results to the server.

[0688] The server receives this data and uses natural language generation models to generate detailed design prompts.

[0689] Input: User input data and sentiment analysis results

[0690] Data processing: The server organizes this data and generates design prompts using a natural language generation model.

[0691] Output: Generated design prompts

[0692] Specific operation: Based on the data "Shiba Inu, lively personality, orange color, lively emotion," the server uses a natural language generation model to generate the prompt "Shiba Inu, lively, orange character."

[0693] Step 4: Running Image Generation

[0694] The server sends the generated design prompt to the image generating means.

[0695] An image generating means automatically generates a character image based on the design prompt.

[0696] Input: Generated design prompt

[0697] Data processing: The image generation means reads the design prompt and generates a character image using the image generation model.

[0698] Output: Multiple character image ideas

[0699] Specific operation: The server passes the prompt "Shiba Inu, energetic, orange character" to the image generation model, and multiple character images are automatically generated.

[0700] Step 5: Return and display the generated image

[0701] The server transmits the generated character image to the user device.

[0702] The terminal receives these images and displays them to the user.

[0703] Input: Generated character image

[0704] Data processing: The device receives the image data and prepares it for display to the user.

[0705] Output: Generated character image that can be viewed by the user

[0706] Specific operation: The server sends a character image to the terminal, and the terminal displays the image to the user.

[0707] Step 6: Re-enter the corrections and save the final image

[0708] The user can review the generated image and re-enter any necessary corrections. If no corrections are required, the user can save or download the final character image.

[0709] Input: User correction requirements or save / download requests

[0710] Data processing: The terminal sends the modified requirements back to the server, which generates a new design prompt.

[0711] Output: Revised character image or final character image

[0712] Specific operation: The user checks the character image and inputs any correction requirements, such as "I would like the color to be a little brighter." The device sends this information to the server, which then generates a new design prompt and generates a new character image. The user can then save or download the final image they are satisfied with.

[0713] (Application example 2)

[0714] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0715] Conventional character design systems generate character images based solely on user input data, which means they are unable to fully reflect the user's emotions and intentions. Furthermore, the quality and intuitive design of the generated characters often fall short of user satisfaction. This results in increased costs and time for character design, making it difficult to respond to situations where emotionally appealing characters are required.

[0716] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0717] In this invention, the server includes means for receiving user input data, means for generating design prompts based on the received input data and analyzed emotion data, means for transmitting the generated design prompts to an image generation means to generate a character image, and means for returning the generated character image to a user device for display. This enables the automatic generation of high-quality, intuitive character images that reflect the user's emotions and intentions.

[0718] "User-input data" refers to information containing specific requirements a user has for character generation, such as the character's category, attributes, primary color, and motif.

[0719] "Emotion data" is data obtained by analyzing the emotions and intentions expressed by users when designing characters. It is acquired using voice input and facial expression recognition technology.

[0720] "Design prompts" are instructions for character design that are generated based on user input data and emotion data. Images are generated based on these.

[0721] "Image generation means" refers to a system or software that reads design prompts and automatically generates character images, for example, using a generative AI model.

[0722] A "user device" is a terminal used by a user to display the generated character image, and includes, for example, mobile devices such as smartphones and tablets.

[0723] A "natural language generation model" is an AI technology that analyzes user-input data and generates design prompts in a format that is easy for humans to understand. A representative example is GPT-3.

[0724] An "emotion analysis engine" is a system that analyzes emotions from the user's voice, facial expressions, etc., and generates emotional data, which allows the user's intentions to be reflected more accurately.

[0725] The present invention relates to a system for generating a character image based on user input data and emotion data. Specific embodiments are described below.

[0726] Receiving User Input

[0727] A user uses a user device such as a smartphone or tablet to input character design requirements, including specific requirements such as the character's category (e.g., animal or human), attributes (e.g., friendliness or liveliness), primary color, and motif.

[0728] Emotional Data Analysis

[0729] When a user inputs design requirements, the emotion analysis engine uses voice input and facial expression recognition technology to analyze the user's emotions. This involves analyzing the tone of voice and facial expressions when the user inputs a "friendly character" and recording the emotional data.

[0730] Generate Design Prompts

[0731] The input data and emotion data sent from the user device are received by a server. The server uses a natural language generation model (e.g., GPT-3) to generate a design prompt based on the received data. The generated design prompt becomes a specific instruction sentence, such as "blue cat, friendly, smiling, simple design."

[0732] Executing image generation

[0733] The generated design prompts are sent from the server to an image generation means (e.g., a generative AI model such as DALL-E or Stable Diffusion). This image generation means reads the design prompts and automatically generates corresponding character images. Multiple image proposals are generated in a short period of time.

[0734] Return and display of generated images

[0735] The generated character image is sent back to the user's device via the server and displayed to the user. The user can check this image and re-enter correction requirements as necessary. When the user is finally satisfied with the character image, they can save or download it.

[0736] Specific example explanation

[0737] For example, if a user wants to design a blue cat character for a pet supplies online store, the following prompt sentence is generated:

[0738] Example prompt sentence:

[0739] "Blue cat, friendly, smiling, simple design"

[0740] This allows users to quickly obtain high-quality character images that reflect their intentions, reducing design costs and improving the efficiency of character generation.

[0741] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0742] Step 1:

[0743] The user inputs the character design requirements using the terminal.

[0744] Specific actions: Enter data such as the character's category, attributes, primary color, and motif in the terminal's application interface.

[0745] Input: Character design requirements (e.g. blue cat, friendly, smiling).

[0746] Output: The entered data is sent from the terminal to the server.

[0747] Step 2:

[0748] The server receives the user input data and analyzes the user's emotions using an emotion analysis engine.

[0749] Specific operation: The server receives the voice and facial expression data sent along with the input data and generates emotion data using an emotion analysis engine.

[0750] Input: User input data, voice data, facial expression data.

[0751] Output: Emotion data is generated and stored in the server.

[0752] Step 3:

[0753] A server generates design prompts based on the user input data and the emotion data.

[0754] Specific operation: Using a natural language generation model (e.g., GPT-3), analyze user input data and sentiment data to generate design prompts.

[0755] Input: User input data, emotion data.

[0756] Output: Design prompt (e.g., "Blue cat, friendly, smiling, simple design").

[0757] Step 4:

[0758] The server transmits the generated design prompt to the image generating means, which generates a character image.

[0759] Specific operations: Provide design prompts to an image generation means (e.g., DALL-E or Stable Diffusion) to generate a character image.

[0760] Input: Design prompt.

[0761] Output: Generated character image.

[0762] Step 5:

[0763] The server returns the generated character image to the terminal and displays it to the user.

[0764] Specific operation: The generated character image is transferred from the server to the terminal and displayed on the user device's display.

[0765] Input: Generated character image.

[0766] Output: The generated character image is displayed on the terminal.

[0767] Step 6:

[0768] The user checks the generated image and re-enters correction requirements if necessary.

[0769] Specific operation: The user checks the displayed character image, and if any corrections are necessary, corrects the input data and emotion data again and sends them.

[0770] Input: Modification requirements (e.g. change primary color, change attributes, etc.).

[0771] Output: The corrected input data is sent from the terminal to the server.

[0772] Step 7:

[0773] The server regenerates the design prompts based on the modified input data, updates the character image, and returns the final character image to the user for saving or download.

[0774] Specific operations: Emotion analysis and design prompt generation are performed again based on the corrected input data, the character image is updated using the image generation means, and the final character image is returned to the user.

[0775] Input: Modified input data, emotion data.

[0776] Output: The final character image is sent back to your device and can be saved or downloaded.

[0777] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0778] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0779] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0780] [Third embodiment]

[0781] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0782] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0783] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0784] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0785] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0786] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0787] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0788] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0789] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0790] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0791] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0792] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0793] The system of the present invention receives user input data, generates character design prompts based on the data, generates an image based on the prompts, and provides the generated character image to the user. This system is mainly composed of the following means:

[0794] Receiving User Input

[0795] Using a terminal, users can input detailed requirements for the character design, including the character's type, characteristics, color, theme, etc. For example, a pet supply company might input requirements such as "Shiba Inu character, lively personality, predominant color orange."

[0796] Generate Design Prompts

[0797] Requirements entered by the user are sent from the terminal to the server. The server receives this input data and generates a design prompt using a natural language generation model, which serves as a generation means. For example, a design prompt such as "Shiba Inu, energetic, orange character" may be generated based on the entered requirements.

[0798] Executing image generation

[0799] The server sends the generated design prompt to the image generation means, which reads the prompt and automatically generates a character image. Through this process, multiple character image ideas are generated in a short period of time.

[0800] Return and display of generated images

[0801] The generated character image is sent back to the terminal from the server. The terminal displays this character image to the user. The user can check the image and re-enter correction requirements if necessary. If no corrections are required, the user can save or download the final character image.

[0802] Specific examples

[0803] Scenario: A pet supply company (the user) wants to design a new dog character.

[0804] 1. "The user uses the terminal interface to input detailed requirements for a dog character (type: dog, breed: Shiba Inu, personality: energetic, primary color: orange)."

[0805] 2. "The terminal sends these input data to the server."

[0806] 3. "The server receives the user's input data and uses a natural language generation model to generate detailed design prompts. For example, 'A Shiba Inu, energetic, orange character.'"

[0807] 4. "The generated design prompt is sent by the server to the image generation means, and a character image is automatically generated."

[0808] 5. The generated character image is sent back to the device from the server and displayed to the user.

[0809] 6. "The user reviews the image and re-enters any corrections required, if necessary. Once the final image is confirmed, it is saved or downloaded."

[0810] In this way, by using the system of this invention, users can efficiently generate high-quality characters in a short period of time, reducing expensive design costs. This system can also be applied to the automatic creation of social media stamps and the design of avatars and items for the Metaverse market.

[0811] The processing flow will be explained below.

[0812] Step 1:

[0813] The user uses a dedicated interface on the device to input the character design requirements (type, characteristics, color, theme, etc.). For example, requirements such as "Shiba Inu character, lively personality, main color orange" can be input.

[0814] Step 2:

[0815] The device prepares to send the user's input data to the server. The input data is converted into a format such as JSON or XML and sent to the server.

[0816] Step 3:

[0817] The server receives the user-entered data, which is temporarily stored in a database or memory.

[0818] Step 4:

[0819] Based on the input data received by the server, a natural language generation model is invoked. The model generates detailed design prompts based on the user's requirements. For example, a prompt such as "Shiba Inu, energetic, orange character" may be generated.

[0820] Step 5:

[0821] The server sends the generated design prompts to the image generation means, which interprets the prompts using the image generation model and uses them to generate the character image.

[0822] Step 6:

[0823] The image generation means generates a character image based on the design prompt. The image generation model uses multiple neural networks to generate a character image that meets the user's requirements.

[0824] Step 7:

[0825] The generated character image is sent back from the image generating means to the server, which receives and temporarily stores this image.

[0826] Step 8:

[0827] The server sends the generated character image to the terminal, where it is converted into a format that the user can view (e.g., JPEG or PNG) and sent to the terminal.

[0828] Step 9:

[0829] The terminal displays the generated character image to the user, who can check the generated image and re-enter correction requirements as necessary.

[0830] Step 10:

[0831] If the user is satisfied with the image, they can save or download the final character image. If any corrections are needed, they will repeat the process from step 1.

[0832] Example 1

[0833] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0834] In conventional character design generation systems, even when users input specific design requirements, the generated character images often did not fully reflect the user's requests. Furthermore, the process for users to reflect revision requirements in the generated character images was inefficient, resulting in time-consuming and costly processes. This created the problem of being unable to generate, revise, and finalize character images quickly and efficiently.

[0835] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0836] In this invention, the server includes means for receiving user input data, means for generating design prompts based on the received input data, means for transmitting the generated design prompts to the image generating means and generating a plurality of character images, means for returning the generated character images to the user device and displaying them, and means for the user to input requirements for modifying the generated character images. This makes it possible to efficiently generate character images that reflect the specific requirements of the user and to efficiently perform the process of modifying the generated character images.

[0837] "User input data" refers to information such as the type, characteristics, color, and theme of a character that is input by a user through a terminal.

[0838] "Generation means" refers to a function within the system that generates design prompts based on user input data using a natural language generation model.

[0839] "Design prompt" refers to specific character design instructions that the generating means creates based on user input data.

[0840] "Image generation means" refers to a function within the system that analyzes design prompts and automatically generates character images based on them.

[0841] "User device" refers to a terminal operated by a user, and is an electronic device for displaying and checking the generated character image.

[0842] "Modification requirements" refer to specific design requirements that the user adds or changes as necessary after looking at the generated character image.

[0843] A "natural language generation model" is a type of artificial intelligence that has the ability to understand and generate language that humans naturally use, and refers to the technology used to generate prompts based on text data.

[0844] The system of the present invention receives user input data, generates character design prompts based on the data, generates an image based on the prompts, and provides the generated character image to the user. An embodiment of the system includes the following means.

[0845] Receiving User Input

[0846] Using a terminal, users can input detailed requirements for the character design. These requirements include the character's type, characteristics, color, theme, etc. For example, a pet supply company might input requirements such as "a Shiba Inu character with a lively personality and a primary color of orange." The terminal then sends this input data to a server. Terminals can be personal computers, smartphones, tablets, etc.

[0847] Generate Design Prompts

[0848] The server receives the user's input data sent from the device. The server uses a generative AI model (such as GPT-3 or BERT) to generate a design prompt based on the input data. The prompt is in the form of a "Shiba Inu, Genki, Orange character." A natural language generation model is used as the generation method, and an appropriate design prompt is generated based on the input data.

[0849] Executing image generation

[0850] The server sends the generated design prompt to an image generation tool (e.g., DALL-E or DeepArt). The image generation model analyzes the prompt and automatically generates a character image. Through this process, multiple character image ideas are generated in a short time. The server temporarily stores the generated image data.

[0851] Return and display of generated images

[0852] The generated character image is sent back to the terminal from the server. The terminal displays this character image to the user. The user can check the displayed image and re-enter correction requirements as necessary.

[0853] Specific examples

[0854] Take the example of a pet supply company (the user) wanting to design a new dog character.

[0855] 1. The user uses the terminal interface to input detailed requirements for the dog character (type: dog, breed: Shiba Inu, personality: energetic, main color: orange).

[0856] 2. The terminal sends the input data to the server.

[0857] 3. The server receives the user's input data and uses a generative AI model (e.g., GPT-3) to generate a detailed design prompt, such as "a Shiba Inu, energetic, orange character."

[0858] 4. The generated design prompt is sent by the server to an image generation means (for example, DALL-E), and a character image is automatically generated.

[0859] 5. The generated character image is sent back from the server to the device and displayed to the user.

[0860] 6. The user checks the image and re-enters any corrections if necessary. Once the final image is confirmed, it is saved or downloaded.

[0861] By using this system, users can efficiently generate high-quality character images in a short period of time, reducing design costs. This system can also be applied to the automatic creation of social media stamps and the design of avatars and items for the Metaverse market.

[0862] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0863] Step 1:

[0864] The user opens the terminal interface and inputs detailed requirements for the character design, including the character's type, characteristics, color, and theme. For example, the user might input "Type: Dog, Breed: Shiba Inu, Personality: Energetic, Main Color: Orange."

[0865] Input: Character type, traits, color, theme

[0866] Output: User input data

[0867] Step 2:

[0868] The terminal sends the data entered by the user to the server using the HTTPS protocol, and the data is encrypted during transmission.

[0869] Input: User-entered data

[0870] Output: The encrypted input data

[0871] Step 3:

[0872] The server receives and decrypts the encrypted data. Next, it uses a generative AI model (e.g., GPT-3) to generate design prompts based on the received data. For example, from the data "Type: Dog, Breed: Shiba Inu, Personality: Energetic, Primary Color: Orange," it generates the prompt "Shiba Inu, Energetic, Orange Character."

[0873] Input: Encrypted input data

[0874] Output: Design prompt

[0875] Step 4:

[0876] The server sends the generated design prompts to an image generation model (e.g., DALL-E), which analyzes the prompts and generates multiple character images based on them. The generated images are stored in the server's temporary storage.

[0877] Input: Design prompt

[0878] Output: Generated character image

[0879] Step 5:

[0880] The server converts the generated character image into an appropriate format (JPEG or PNG), compresses it if necessary, and then sends it to the terminal.

[0881] Input: Generated character image

[0882] Output: Converted and compressed character image

[0883] Step 6:

[0884] The terminal receives the character image sent from the server and displays it to the user. The user can check the displayed image and re-enter correction requirements if necessary. If no corrections are required, the user can save or download the image.

[0885] Input: Converted and compressed character image

[0886] Output: Character image that can be viewed by the user

[0887] Step 7 (if necessary):

[0888] If the user inputs modified requirements, the terminal retransmits the new design requirements to the server, and the processes from step 2 to step 6 are performed again.

[0889] Input: Correction Requirement

[0890] Output: New Design Requirements

[0891] In this way, each processing step works in conjunction with the others, allowing the user to generate a high-quality character image in a short period of time and even make corrections as needed.

[0892] (Application example 1)

[0893] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0894] Conventional character design systems have the problem that even if a user inputs detailed character requirements, they lack a means to immediately check the results in a virtual environment, limiting the user experience. Furthermore, re-entering revised requirements and determining the final character image requires multiple steps, resulting in low efficiency. The present invention aims to solve these problems by providing a system that quickly and efficiently generates high-quality characters and allows them to be checked in a virtual environment in real time.

[0895] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0896] In this invention, the server includes means for receiving user input data, means for generating design prompts based on the received input data, means for transmitting the generated design prompts to the image generation means to generate a character image, interface means for allowing the user to check the generated character image in real time within the virtual environment, means for allowing the user to re-input correction requirements, and means for saving or downloading the final character image, thereby enabling the user to quickly generate high-quality characters and check and correct them in real time within the virtual environment.

[0897] "User-input data" refers to data that the user inputs specific requirements such as the type, characteristics, color, and theme of the character.

[0898] A "generator" is a device used to generate design prompts based on user input data.

[0899] A "design prompt" is a command statement generated by the generating means for realizing a character design.

[0900] The "image generation means" is a means for generating a character image based on the design prompt.

[0901] A "user device" is a terminal device used by a user, on which a generated character image is displayed.

[0902] The "interface means" is a means for the user to check the character image generated in the virtual environment in real time.

[0903] The "correction requirements" are requirements that the user inputs in order to make corrections to the generated character image.

[0904] "Storage or download means" refers to a means for saving or downloading the final character image.

[0905] A "virtual environment" refers to a virtual space or interface that allows you to check and modify character images in real time.

[0906] The embodiment of the present invention is to build a system according to the following procedure, which specifically includes the entire process from receiving user input data to checking it in the virtual environment, and generating and correcting character images.

[0907] Program generation and natural language explanation

[0908] Receiving user-entered data

[0909] Users use a smartphone app to input detailed requirements for their character, including the character's type, characteristics, color, and theme. The smartphone app is developed using Flutter.

[0910] Generate Design Prompts

[0911] The received user input data is sent from the smartphone app to a server, which uses Flask (Python) to generate design prompts using a natural language generation model (TensorFlow or PyTorch).

[0912] Executing image generation

[0913] The generated prompts are sent to an image generation engine such as Stable Diffusion, which generates character images based on the prompts. This generation process takes place on the server.

[0914] Check in a virtual environment

[0915] The generated character image is displayed in a virtual environment through an HMD (e.g., Oculus Quest 2) so that it can be viewed in real time. The user can view the character in this virtual environment and re-enter correction requirements as needed.

[0916] Revision and Finalization

[0917] When the user re-enters the correction requirements using the smartphone app, the server generates a new design prompt and sends it to the image generation engine. The final generated character image is sent to the user device (smartphone app).

[0918] Specific examples

[0919] A user types in, "I want a character who is a blue-haired wizard in a fantasy world and has a kind expression."

[0920] Example of a generated design prompt: "Fantasy wizard, blue hair, kind expression, wearing robes"

[0921] The final generated avatar will be designed based on this prompt.

[0922] Hardware and software used

[0923] Smartphone app: Flutter

[0924] Server side: Flask (Python)

[0925] Natural language generation model: TensorFlow, PyTorch

[0926] Image generation engine: Stable Diffusion

[0927] Virtual Environment Interface: Oculus Quest 2

[0928] This system allows users to quickly and efficiently generate high-quality characters and then review and modify them in real time in a virtual environment. This system is expected to have a wide range of applications, not just in the entertainment industry.

[0929] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0930] Step 1:

[0931] A user opens a smartphone app and inputs requirements for the character type, features, color, and theme. The input requirements are saved as user-input data.

[0932] Input: User requirements for character type, traits, color, and theme

[0933] Output: User input data

[0934] Step 2:

[0935] The smartphone app sends user-entered data to the server, which receives the data in JSON format and stores it for processing.

[0936] Input: User input data (JSON format)

[0937] Output: User-entered data stored on the server

[0938] Step 3:

[0939] The server generates design prompts based on the received user input data, utilizing natural language generation models (TensorFlow or PyTorch) to translate user requirements into text prompts.

[0940] Input: User-entered data stored on the server

[0941] Output: Design prompt

[0942] Step 4:

[0943] The server sends the generated design prompt to an image generator (such as Stable Diffusion), which reads the prompt and automatically generates a character image.

[0944] Input: Design prompt

[0945] Output: Generated character image

[0946] Step 5:

[0947] The server sends the generated character image to a smartphone app and an HMD (such as Oculus Quest 2), where it is displayed in real time in the virtual environment. The user can view the character within this environment.

[0948] Input: Generated character image

[0949] Output: Image of the character displayed in the virtual environment

[0950] Step 6:

[0951] The user checks the character image in the virtual environment and re-enters any necessary corrections through the smartphone app, which then sends the re-entered data to the server.

[0952] Input: Correction Requirement

[0953] Output: Re-entered modified requirements data

[0954] Step 7:

[0955] The server again generates design prompts and sends them to the image generator to generate a modified character image, and this process is repeated until the user is satisfied.

[0956] Input: Re-entered modified requirement data

[0957] Output: Modified character image

[0958] Step 8:

[0959] Once the final character image has been finalized, the server sends this image data to a smartphone app, where the user can save or download it.

[0960] Input: Final generated character image

[0961] Output: Saved or downloaded character image

[0962] By following these steps, users can efficiently generate high-quality characters and then check and modify them in real time in a virtual environment.

[0963] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0964] The system of the present invention generates more optimal character design proposals by combining not only user input data but also an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described below.

[0965] Receiving User Input

[0966] Using a terminal, users can input the design requirements for the character (type, characteristics, color, theme, etc.) For example, a pet supply company might input requirements such as "a Shiba Inu character with a lively personality and orange as the main color."

[0967] Use of emotion engine

[0968] When a user inputs design requirements, the emotion engine analyzes the user's emotions. This can be done using voice input and facial expression recognition technology. For example, when a user inputs a "cheerful character," the engine analyzes their intention from their tone of voice and facial expression.

[0969] Generate Design Prompts

[0970] The server receives the user's input data and the analysis results of the emotion engine. Based on this data, a natural language generation model is invoked to generate detailed design prompts. For example, a specific prompt such as "Shiba Inu, energetic, orange character" is generated.

[0971] Executing image generation

[0972] The generated design prompt is sent from the server to the image generation means, which reads the prompt and automatically generates a character image. Through this process, multiple character image ideas are generated in a short period of time.

[0973] Return and display of generated images

[0974] The generated character image is sent from the server to the terminal and displayed to the user. The user can check the generated image and re-enter correction requirements if necessary. If no corrections are required, the user can save or download the final character image.

[0975] Specific examples

[0976] Scenario: A pet supply company (the user) wants to design a new dog character.

[0977] 1. The user uses the terminal interface to input detailed requirements for a dog character (e.g., type: dog, breed: Shiba Inu, personality: energetic, main color: orange).

[0978] 2. The emotion engine analyzes the user's voice and facial expressions when inputting design requirements, such as a smile or bright tone to express a lively personality.

[0979] 3. The device sends the input data and the emotion engine's analysis results to the server.

[0980] 4. The server receives this data and uses a natural language generation model to generate detailed design prompts, such as "Shiba Inu, energetic, orange character."

[0981] 5. The generated design prompt is sent to an image generating means, and a character image is automatically generated.

[0982] 6. The generated character image is sent from the server to the device and displayed to the user.

[0983] 7. The user checks the generated image and re-enters any necessary corrections. Once the final image is confirmed, the user saves or downloads it.

[0984] This system enables more intuitive and effective character design that reflects the user's emotions, reducing expensive design costs and enabling high-quality character generation in a short period of time. The use of an emotion engine makes it possible to generate design prompts that more accurately reflect the user's intentions, making it particularly effective when character design that appeals to emotions is required.

[0985] The processing flow will be explained below.

[0986] Step 1:

[0987] The user inputs the character's design requirements (type, characteristics, color, theme) using a dedicated interface on the device. For example, the user might input "Shiba Inu character, lively personality, main color orange."

[0988] Step 2:

[0989] The device sends the user's input data to the emotion engine, which then analyzes the user's emotions using their voice input and facial expression recognition. For example, when the user says "I'm fine," the emotion is read from the tone of voice and facial expression.

[0990] Step 3:

[0991] The emotion engine returns the analysis results to the device. The analysis results include data indicating the user's emotions. For example, it may return information such as "The user is expressing a cheerful emotion."

[0992] Step 4:

[0993] The device sends the user input data and the analysis results of the emotion engine to the server, which then receives both the user's design requirements and emotions.

[0994] Step 5:

[0995] The server invokes a generation method (natural language generation model) based on the received data to generate a detailed design prompt. For example, a prompt such as "Shiba Inu, energetic, orange character" is generated.

[0996] Step 6:

[0997] The server transmits the generated design prompt to the image generating means, which reads the prompt and automatically generates a character image.

[0998] Step 7:

[0999] The image generating means returns the generated character image to the server, which receives and temporarily stores the image.

[1000] Step 8:

[1001] The server sends the generated character image to the terminal, where it is converted into a format that can be viewed by the user and sent to the terminal.

[1002] Step 9:

[1003] The device displays the generated character image to the user. The user can check the image and input corrections as needed. For example, they can input corrections such as "I want the expression to be a little brighter."

[1004] Step 10:

[1005] If the user is satisfied with the image, they can save or download the final character image. If any corrections are needed, they can repeat the process from step 1 again.

[1006] Step 11:

[1007] Once the user has finalized the character image, the system will save the image and make it available to the user for marketing and commercial use.

[1008] In this way, the system efficiently and effectively generates character designs that reflect the user's emotions. The introduction of an emotion engine makes it possible to generate design prompts that accurately reflect the user's intentions and emotions, resulting in results that are more satisfying for the user.

[1009] Example 2

[1010] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1011] In modern character design, there are only a limited number of systems that can accurately reflect the user's emotions and intentions. Conventional design systems rely solely on user input data, which often results in inaccurate reflection of the user's emotions and intentions. Furthermore, the design process is complex and time-consuming, making it difficult to generate high-quality characters quickly. This invention was developed to address these issues.

[1012] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1013] In this invention, the server includes means for receiving user input data, means for utilizing an emotion engine that analyzes the user's emotions, and means for generating design prompts based on the received input data and the analyzed emotion data, thereby enabling the rapid generation of high-quality character images that reflect the user's emotions and intentions.

[1014] "User input data" refers to design requirement information input by a user through a terminal for character creation, including the character's type, characteristics, color, and theme.

[1015] The "emotion engine" is a technology that analyzes the user's tone of voice and facial expressions when inputting information and recognizes the user's emotions in real time.

[1016] The "generator" is a system component for generating detailed design prompts based on received user input data and analyzed emotion data, primarily utilizing natural language generation models.

[1017] A "design prompt" is a set of instructions containing specific character design details that are generated based on user input data and emotion data.

[1018] The "image generation means" is a system component that automatically generates character images based on the generated design prompts. It mainly uses an image generation model.

[1019] "User device" refers to an electronic device used by a user to input data for character creation and to view the generated character image. Examples include personal computers and smartphones.

[1020] The system of the present invention generates intuitive and effective character design proposals by combining user input data with an emotion engine. A specific embodiment of this system will be described below.

[1021] Hardware and Software Configuration

[1022] Hardware used

[1023] User Device: Electronic devices such as computers, tablets, and smartphones.

[1024] Server: A cloud-based server with a powerful processor and lots of memory.

[1025] Software used

[1026] Emotion Engine: Speech and facial recognition software to analyze the user's tone of voice and facial expressions.

[1027] Generator: Software that generates design prompts using a natural language generation model (e.g., GPT-3).

[1028] Image generation means: Software that generates character images using an image generation model (e.g., DALL-E).

[1029] Details of the main process

[1030] Receiving User Input

[1031] The user inputs the character design requirements using the user device. For example, the user can specify details such as "Shiba Inu character, lively personality, main color is orange." The device receives this input data and temporarily stores it.

[1032] Use of emotion engine

[1033] When a user inputs design requirements, the emotion engine analyzes the user's voice and facial expressions in real time. For example, when a user inputs "a lively character," the engine analyzes their intention from the tone of their voice and facial expression. The device acquires this emotion data and saves the analysis results.

[1034] Generate Design Prompts

[1035] The device sends the user's input data and the emotion engine's analysis results to the server, which receives this data and uses a natural language generation model to generate detailed design prompts. For example, a specific prompt such as "Shiba Inu, energetic, orange character" is generated.

[1036] Executing image generation

[1037] The generated design prompt is sent from the server to the image generation means, which reads the prompt and automatically generates a character image. Through this process, multiple character image ideas are generated in a short period of time.

[1038] Return and display of generated images

[1039] The generated character images are sent from the server to the user device. The user device receives these images and displays them to the user. The user can review the generated images and re-enter correction requirements as needed. If no corrections are required, the user saves or downloads the final character image.

[1040] Specific examples

[1041] For example, if a pet supply company (user) wants to design a new dog character, the process would go something like this:

[1042] 1. The user uses the terminal interface to input detailed requirements for a dog character (e.g., type: dog, breed: Shiba Inu, personality: energetic, main color: orange).

[1043] 2. The emotion engine analyzes the voice and facial expressions used when users input design requirements, such as smiles and bright tones that express a lively personality.

[1044] 3. The device sends the input data and the emotion engine's analysis results to the server.

[1045] 4. The server receives this data and uses a natural language generation model to generate detailed design prompts, such as "Shiba Inu, energetic, orange character."

[1046] 5. The generated design prompt is sent to an image generating means, and a character image is automatically generated.

[1047] 6. The generated character image is sent from the server to the user device and displayed to the user.

[1048] 7. The user checks the generated image and re-enters any necessary corrections. Once the final image is confirmed, the user saves or downloads it.

[1049] Example prompt sentence:

[1050] "Shiba Inu, a lively, orange character"

[1051] This system enables more intuitive and effective character design that reflects the user's emotions, reducing expensive design costs and enabling high-quality character generation in a short period of time. Furthermore, the use of an emotion engine makes it possible to generate design prompts that more accurately reflect the user's intentions, making it particularly effective when character design is required to appeal to emotions.

[1052] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1053] Step 1: Receiving User Input

[1054] A user inputs design requirements (e.g., type, characteristics, color, theme) of a character using a user device. For example, the user inputs requirements such as "a Shiba Inu character with a lively personality and orange as the main color."

[1055] Input: Character type, traits, color, theme

[1056] Data processing: The device temporarily stores the entered data.

[1057] Output: Input design requirements data

[1058] Specific operation: The user enters "Dog breed: Shiba Inu," "Personality: Energetic," and "Main color: Orange" into the device's input form, and the device receives and temporarily stores this information.

[1059] Step 2: Use the Emotion Engine

[1060] As users input their design requirements, the emotion engine analyzes their tone of voice and facial expressions in real time.

[1061] Input: User's voice data, facial expression data

[1062] Data processing: The device sends the voice data and facial expression images to the analysis engine to obtain emotional data.

[1063] Output: User sentiment analysis results

[1064] Specific operation: When a user types "cheerful personality," the camera and microphone capture the tone of voice and facial expression, and the emotion engine analyzes that "the user has a cheerful emotion."

[1065] Step 3: Generate design prompts

[1066] The device sends the user input data and the emotion engine's analysis results to the server.

[1067] The server receives this data and uses natural language generation models to generate detailed design prompts.

[1068] Input: User input data and sentiment analysis results

[1069] Data processing: The server organizes this data and generates design prompts using a natural language generation model.

[1070] Output: Generated design prompts

[1071] Specific operation: Based on the data "Shiba Inu, lively personality, orange color, lively emotion," the server uses a natural language generation model to generate the prompt "Shiba Inu, lively, orange character."

[1072] Step 4: Running Image Generation

[1073] The server sends the generated design prompt to the image generating means.

[1074] An image generating means automatically generates a character image based on the design prompt.

[1075] Input: Generated design prompt

[1076] Data processing: The image generation means reads the design prompt and generates a character image using the image generation model.

[1077] Output: Multiple character image ideas

[1078] Specific operation: The server passes the prompt "Shiba Inu, energetic, orange character" to the image generation model, and multiple character images are automatically generated.

[1079] Step 5: Return and display the generated image

[1080] The server transmits the generated character image to the user device.

[1081] The terminal receives these images and displays them to the user.

[1082] Input: Generated character image

[1083] Data processing: The device receives the image data and prepares it for display to the user.

[1084] Output: Generated character image that can be viewed by the user

[1085] Specific operation: The server sends a character image to the terminal, and the terminal displays the image to the user.

[1086] Step 6: Re-enter the corrections and save the final image

[1087] The user can review the generated image and re-enter any necessary corrections. If no corrections are required, the user can save or download the final character image.

[1088] Input: User correction requirements or save / download requests

[1089] Data processing: The terminal sends the modified requirements back to the server, which generates a new design prompt.

[1090] Output: Revised character image or final character image

[1091] Specific operation: The user checks the character image and inputs any correction requirements, such as "I would like the color to be a little brighter." The device sends this information to the server, which then generates a new design prompt and generates a new character image. The user can then save or download the final image they are satisfied with.

[1092] (Application example 2)

[1093] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1094] Conventional character design systems generate character images based solely on user input data, which means they are unable to fully reflect the user's emotions and intentions. Furthermore, the quality and intuitive design of the generated characters often fall short of user satisfaction. This results in increased costs and time for character design, making it difficult to respond to situations where emotionally appealing characters are required.

[1095] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1096] In this invention, the server includes means for receiving user input data, means for generating design prompts based on the received input data and analyzed emotion data, means for transmitting the generated design prompts to an image generation means to generate a character image, and means for returning the generated character image to a user device for display. This enables the automatic generation of high-quality, intuitive character images that reflect the user's emotions and intentions.

[1097] "User-input data" refers to information containing specific requirements a user has for character generation, such as the character's category, attributes, primary color, and motif.

[1098] "Emotion data" is data obtained by analyzing the emotions and intentions expressed by users when designing characters. It is acquired using voice input and facial expression recognition technology.

[1099] "Design prompts" are instructions for character design that are generated based on user input data and emotion data. Images are generated based on these.

[1100] "Image generation means" refers to a system or software that reads design prompts and automatically generates character images, for example, using a generative AI model.

[1101] A "user device" is a terminal used by a user to display the generated character image, and includes, for example, mobile devices such as smartphones and tablets.

[1102] A "natural language generation model" is an AI technology that analyzes user-input data and generates design prompts in a format that is easy for humans to understand. A representative example is GPT-3.

[1103] An "emotion analysis engine" is a system that analyzes emotions from the user's voice, facial expressions, etc., and generates emotional data, which allows the user's intentions to be reflected more accurately.

[1104] The present invention relates to a system for generating a character image based on user input data and emotion data. Specific embodiments are described below.

[1105] Receiving User Input

[1106] A user uses a user device such as a smartphone or tablet to input character design requirements, including specific requirements such as the character's category (e.g., animal or human), attributes (e.g., friendliness or liveliness), primary color, and motif.

[1107] Emotional Data Analysis

[1108] When a user inputs design requirements, the emotion analysis engine uses voice input and facial expression recognition technology to analyze the user's emotions. This involves analyzing the tone of voice and facial expressions when the user inputs a "friendly character" and recording the emotional data.

[1109] Generate Design Prompts

[1110] The input data and emotion data sent from the user device are received by a server. The server uses a natural language generation model (e.g., GPT-3) to generate a design prompt based on the received data. The generated design prompt becomes a specific instruction sentence, such as "blue cat, friendly, smiling, simple design."

[1111] Executing image generation

[1112] The generated design prompts are sent from the server to an image generation means (e.g., a generative AI model such as DALL-E or Stable Diffusion). This image generation means reads the design prompts and automatically generates corresponding character images. Multiple image proposals are generated in a short period of time.

[1113] Return and display of generated images

[1114] The generated character image is sent back to the user's device via the server and displayed to the user. The user can check this image and re-enter correction requirements as necessary. When the user is finally satisfied with the character image, they can save or download it.

[1115] Specific example explanation

[1116] For example, if a user wants to design a blue cat character for a pet supplies online store, the following prompt sentence is generated:

[1117] Example prompt sentence:

[1118] "Blue cat, friendly, smiling, simple design"

[1119] This allows users to quickly obtain high-quality character images that reflect their intentions, reducing design costs and improving the efficiency of character generation.

[1120] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1121] Step 1:

[1122] The user inputs the character design requirements using the terminal.

[1123] Specific actions: Enter data such as the character's category, attributes, primary color, and motif in the terminal's application interface.

[1124] Input: Character design requirements (e.g. blue cat, friendly, smiling).

[1125] Output: The entered data is sent from the terminal to the server.

[1126] Step 2:

[1127] The server receives the user input data and analyzes the user's emotions using an emotion analysis engine.

[1128] Specific operation: The server receives the voice and facial expression data sent along with the input data and generates emotion data using an emotion analysis engine.

[1129] Input: User input data, voice data, facial expression data.

[1130] Output: Emotion data is generated and stored in the server.

[1131] Step 3:

[1132] A server generates design prompts based on the user input data and the emotion data.

[1133] Specific operation: Using a natural language generation model (e.g., GPT-3), analyze user input data and sentiment data to generate design prompts.

[1134] Input: User input data, emotion data.

[1135] Output: Design prompt (e.g., "Blue cat, friendly, smiling, simple design").

[1136] Step 4:

[1137] The server transmits the generated design prompt to the image generating means, which generates a character image.

[1138] Specific operations: Provide design prompts to an image generation means (e.g., DALL-E or Stable Diffusion) to generate a character image.

[1139] Input: Design prompt.

[1140] Output: Generated character image.

[1141] Step 5:

[1142] The server returns the generated character image to the terminal and displays it to the user.

[1143] Specific operation: The generated character image is transferred from the server to the terminal and displayed on the user device's display.

[1144] Input: Generated character image.

[1145] Output: The generated character image is displayed on the terminal.

[1146] Step 6:

[1147] The user checks the generated image and re-enters correction requirements if necessary.

[1148] Specific operation: The user checks the displayed character image, and if any corrections are necessary, corrects the input data and emotion data again and sends them.

[1149] Input: Modification requirements (e.g. change primary color, change attributes, etc.).

[1150] Output: The corrected input data is sent from the terminal to the server.

[1151] Step 7:

[1152] The server regenerates the design prompts based on the modified input data, updates the character image, and returns the final character image to the user for saving or download.

[1153] Specific operations: Emotion analysis and design prompt generation are performed again based on the corrected input data, the character image is updated using the image generation means, and the final character image is returned to the user.

[1154] Input: Modified input data, emotion data.

[1155] Output: The final character image is sent back to your device and can be saved or downloaded.

[1156] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1157] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1158] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1159] [Fourth embodiment]

[1160] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1161] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1162] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1163] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1164] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1165] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1166] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1167] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1168] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1169] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1170] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1171] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1172] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1173] The system of the present invention receives user input data, generates character design prompts based on the data, generates an image based on the prompts, and provides the generated character image to the user. This system is mainly composed of the following means:

[1174] Receiving User Input

[1175] Using a terminal, users can input detailed requirements for the character design, including the character's type, characteristics, color, theme, etc. For example, a pet supply company might input requirements such as "Shiba Inu character, lively personality, predominant color orange."

[1176] Generate Design Prompts

[1177] Requirements entered by the user are sent from the terminal to the server. The server receives this input data and generates a design prompt using a natural language generation model, which serves as a generation means. For example, a design prompt such as "Shiba Inu, energetic, orange character" may be generated based on the entered requirements.

[1178] Executing image generation

[1179] The server sends the generated design prompt to the image generation means, which reads the prompt and automatically generates a character image. Through this process, multiple character image ideas are generated in a short period of time.

[1180] Return and display of generated images

[1181] The generated character image is sent back to the terminal from the server. The terminal displays this character image to the user. The user can check the image and re-enter correction requirements if necessary. If no corrections are required, the user can save or download the final character image.

[1182] Specific examples

[1183] Scenario: A pet supply company (the user) wants to design a new dog character.

[1184] 1. "The user uses the terminal interface to input detailed requirements for a dog character (type: dog, breed: Shiba Inu, personality: energetic, primary color: orange)."

[1185] 2. "The terminal sends these input data to the server."

[1186] 3. "The server receives the user's input data and uses a natural language generation model to generate detailed design prompts. For example, 'A Shiba Inu, energetic, orange character.'"

[1187] 4. "The generated design prompt is sent by the server to the image generation means, and a character image is automatically generated."

[1188] 5. The generated character image is sent back to the device from the server and displayed to the user.

[1189] 6. "The user reviews the image and re-enters any corrections required, if necessary. Once the final image is confirmed, it is saved or downloaded."

[1190] In this way, by using the system of this invention, users can efficiently generate high-quality characters in a short period of time, reducing expensive design costs. This system can also be applied to the automatic creation of social media stamps and the design of avatars and items for the Metaverse market.

[1191] The processing flow will be explained below.

[1192] Step 1:

[1193] The user uses a dedicated interface on the device to input the character design requirements (type, characteristics, color, theme, etc.). For example, requirements such as "Shiba Inu character, lively personality, main color orange" can be input.

[1194] Step 2:

[1195] The device prepares to send the user's input data to the server. The input data is converted into a format such as JSON or XML and sent to the server.

[1196] Step 3:

[1197] The server receives the user-entered data, which is temporarily stored in a database or memory.

[1198] Step 4:

[1199] Based on the input data received by the server, a natural language generation model is invoked. The model generates detailed design prompts based on the user's requirements. For example, a prompt such as "Shiba Inu, energetic, orange character" may be generated.

[1200] Step 5:

[1201] The server sends the generated design prompts to the image generation means, which interprets the prompts using the image generation model and uses them to generate the character image.

[1202] Step 6:

[1203] The image generation means generates a character image based on the design prompt. The image generation model uses multiple neural networks to generate a character image that meets the user's requirements.

[1204] Step 7:

[1205] The generated character image is sent back from the image generating means to the server, which receives and temporarily stores this image.

[1206] Step 8:

[1207] The server sends the generated character image to the terminal, where it is converted into a format that the user can view (e.g., JPEG or PNG) and sent to the terminal.

[1208] Step 9:

[1209] The terminal displays the generated character image to the user, who can check the generated image and re-enter correction requirements as necessary.

[1210] Step 10:

[1211] If the user is satisfied with the image, they can save or download the final character image. If any corrections are needed, they will repeat the process from step 1.

[1212] Example 1

[1213] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1214] In conventional character design generation systems, even when users input specific design requirements, the generated character images often did not fully reflect the user's requests. Furthermore, the process for users to reflect revision requirements in the generated character images was inefficient, resulting in time-consuming and costly processes. This created the problem of being unable to generate, revise, and finalize character images quickly and efficiently.

[1215] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1216] In this invention, the server includes means for receiving user input data, means for generating design prompts based on the received input data, means for transmitting the generated design prompts to the image generating means and generating a plurality of character images, means for returning the generated character images to the user device and displaying them, and means for the user to input requirements for modifying the generated character images. This makes it possible to efficiently generate character images that reflect the specific requirements of the user and to efficiently perform the process of modifying the generated character images.

[1217] "User input data" refers to information such as the type, characteristics, color, and theme of a character that is input by a user through a terminal.

[1218] "Generation means" refers to a function within the system that generates design prompts based on user input data using a natural language generation model.

[1219] "Design prompt" refers to specific character design instructions that the generating means creates based on user input data.

[1220] "Image generation means" refers to a function within the system that analyzes design prompts and automatically generates character images based on them.

[1221] "User device" refers to a terminal operated by a user, and is an electronic device for displaying and checking the generated character image.

[1222] "Modification requirements" refer to specific design requirements that the user adds or changes as necessary after looking at the generated character image.

[1223] A "natural language generation model" is a type of artificial intelligence that has the ability to understand and generate language that humans naturally use, and refers to the technology used to generate prompts based on text data.

[1224] The system of the present invention receives user input data, generates character design prompts based on the data, generates an image based on the prompts, and provides the generated character image to the user. An embodiment of the system includes the following means.

[1225] Receiving User Input

[1226] Using a terminal, users can input detailed requirements for the character design. These requirements include the character's type, characteristics, color, theme, etc. For example, a pet supply company might input requirements such as "a Shiba Inu character with a lively personality and a primary color of orange." The terminal then sends this input data to a server. Terminals can be personal computers, smartphones, tablets, etc.

[1227] Generate Design Prompts

[1228] The server receives the user's input data sent from the device. The server uses a generative AI model (such as GPT-3 or BERT) to generate a design prompt based on the input data. The prompt is in the form of a "Shiba Inu, Genki, Orange character." A natural language generation model is used as the generation method, and an appropriate design prompt is generated based on the input data.

[1229] Executing image generation

[1230] The server sends the generated design prompt to an image generation tool (e.g., DALL-E or DeepArt). The image generation model analyzes the prompt and automatically generates a character image. Through this process, multiple character image ideas are generated in a short time. The server temporarily stores the generated image data.

[1231] Return and display of generated images

[1232] The generated character image is sent back to the terminal from the server. The terminal displays this character image to the user. The user can check the displayed image and re-enter correction requirements as necessary.

[1233] Specific examples

[1234] Take the example of a pet supply company (the user) wanting to design a new dog character.

[1235] 1. The user uses the terminal interface to input detailed requirements for the dog character (type: dog, breed: Shiba Inu, personality: energetic, main color: orange).

[1236] 2. The terminal sends the input data to the server.

[1237] 3. The server receives the user's input data and uses a generative AI model (e.g., GPT-3) to generate a detailed design prompt, such as "a Shiba Inu, energetic, orange character."

[1238] 4. The generated design prompt is sent by the server to an image generation means (for example, DALL-E), and a character image is automatically generated.

[1239] 5. The generated character image is sent back from the server to the device and displayed to the user.

[1240] 6. The user checks the image and re-enters any corrections if necessary. Once the final image is confirmed, it is saved or downloaded.

[1241] By using this system, users can efficiently generate high-quality character images in a short period of time, reducing design costs. This system can also be applied to the automatic creation of social media stamps and the design of avatars and items for the Metaverse market.

[1242] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1243] Step 1:

[1244] The user opens the terminal interface and inputs detailed requirements for the character design, including the character's type, characteristics, color, and theme. For example, the user might input "Type: Dog, Breed: Shiba Inu, Personality: Energetic, Main Color: Orange."

[1245] Input: Character type, traits, color, theme

[1246] Output: User input data

[1247] Step 2:

[1248] The terminal sends the data entered by the user to the server using the HTTPS protocol, and the data is encrypted during transmission.

[1249] Input: User-entered data

[1250] Output: The encrypted input data

[1251] Step 3:

[1252] The server receives and decrypts the encrypted data. Next, it uses a generative AI model (e.g., GPT-3) to generate design prompts based on the received data. For example, from the data "Type: Dog, Breed: Shiba Inu, Personality: Energetic, Primary Color: Orange," it generates the prompt "Shiba Inu, Energetic, Orange Character."

[1253] Input: Encrypted input data

[1254] Output: Design prompt

[1255] Step 4:

[1256] The server sends the generated design prompts to an image generation model (e.g., DALL-E), which analyzes the prompts and generates multiple character images based on them. The generated images are stored in the server's temporary storage.

[1257] Input: Design prompt

[1258] Output: Generated character image

[1259] Step 5:

[1260] The server converts the generated character image into an appropriate format (JPEG or PNG), compresses it if necessary, and then sends it to the terminal.

[1261] Input: Generated character image

[1262] Output: Converted and compressed character image

[1263] Step 6:

[1264] The terminal receives the character image sent from the server and displays it to the user. The user can check the displayed image and re-enter correction requirements if necessary. If no corrections are required, the user can save or download the image.

[1265] Input: Converted and compressed character image

[1266] Output: Character image that can be viewed by the user

[1267] Step 7 (if necessary):

[1268] If the user inputs modified requirements, the terminal retransmits the new design requirements to the server, and the processes from step 2 to step 6 are performed again.

[1269] Input: Correction Requirement

[1270] Output: New Design Requirements

[1271] In this way, each processing step works in conjunction with the others, allowing the user to generate a high-quality character image in a short period of time and even make corrections as needed.

[1272] (Application example 1)

[1273] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1274] Conventional character design systems have the problem that even if a user inputs detailed character requirements, they lack a means to immediately check the results in a virtual environment, limiting the user experience. Furthermore, re-entering revised requirements and determining the final character image requires multiple steps, resulting in low efficiency. The present invention aims to solve these problems by providing a system that quickly and efficiently generates high-quality characters and allows them to be checked in a virtual environment in real time.

[1275] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1276] In this invention, the server includes means for receiving user input data, means for generating design prompts based on the received input data, means for transmitting the generated design prompts to the image generation means to generate a character image, interface means for allowing the user to check the generated character image in real time within the virtual environment, means for allowing the user to re-input correction requirements, and means for saving or downloading the final character image, thereby enabling the user to quickly generate high-quality characters and check and correct them in real time within the virtual environment.

[1277] "User-input data" refers to data that the user inputs specific requirements such as the type, characteristics, color, and theme of the character.

[1278] A "generator" is a device used to generate design prompts based on user input data.

[1279] A "design prompt" is a command statement generated by the generating means for realizing a character design.

[1280] The "image generation means" is a means for generating a character image based on the design prompt.

[1281] A "user device" is a terminal device used by a user, on which a generated character image is displayed.

[1282] The "interface means" is a means for the user to check the character image generated in the virtual environment in real time.

[1283] The "correction requirements" are requirements that the user inputs in order to make corrections to the generated character image.

[1284] "Storage or download means" refers to a means for saving or downloading the final character image.

[1285] A "virtual environment" refers to a virtual space or interface that allows you to check and modify character images in real time.

[1286] The embodiment of the present invention is to build a system according to the following procedure, which specifically includes the entire process from receiving user input data to checking it in the virtual environment, and generating and correcting character images.

[1287] Program generation and natural language explanation

[1288] Receiving user-entered data

[1289] Users use a smartphone app to input detailed requirements for their character, including the character's type, characteristics, color, and theme. The smartphone app is developed using Flutter.

[1290] Generate Design Prompts

[1291] The received user input data is sent from the smartphone app to a server, which uses Flask (Python) to generate design prompts using a natural language generation model (TensorFlow or PyTorch).

[1292] Executing image generation

[1293] The generated prompts are sent to an image generation engine such as Stable Diffusion, which generates character images based on the prompts. This generation process takes place on the server.

[1294] Check in a virtual environment

[1295] The generated character image is displayed in a virtual environment through an HMD (e.g., Oculus Quest 2) so that it can be viewed in real time. The user can view the character in this virtual environment and re-enter correction requirements as needed.

[1296] Revision and Finalization

[1297] When the user re-enters the correction requirements using the smartphone app, the server generates a new design prompt and sends it to the image generation engine. The final generated character image is sent to the user device (smartphone app).

[1298] Specific examples

[1299] A user types in, "I want a character who is a blue-haired wizard in a fantasy world and has a kind expression."

[1300] Example of a generated design prompt: "Fantasy wizard, blue hair, kind expression, wearing robes"

[1301] The final generated avatar will be designed based on this prompt.

[1302] Hardware and software used

[1303] Smartphone app: Flutter

[1304] Server side: Flask (Python)

[1305] Natural language generation model: TensorFlow, PyTorch

[1306] Image generation engine: Stable Diffusion

[1307] Virtual Environment Interface: Oculus Quest 2

[1308] This system allows users to quickly and efficiently generate high-quality characters and then review and modify them in real time in a virtual environment. This system is expected to have a wide range of applications, not just in the entertainment industry.

[1309] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1310] Step 1:

[1311] A user opens a smartphone app and inputs requirements for the character type, features, color, and theme. The input requirements are saved as user-input data.

[1312] Input: User requirements for character type, traits, color, and theme

[1313] Output: User input data

[1314] Step 2:

[1315] The smartphone app sends user-entered data to the server, which receives the data in JSON format and stores it for processing.

[1316] Input: User input data (JSON format)

[1317] Output: User-entered data stored on the server

[1318] Step 3:

[1319] The server generates design prompts based on the received user input data, utilizing natural language generation models (TensorFlow or PyTorch) to translate user requirements into text prompts.

[1320] Input: User-entered data stored on the server

[1321] Output: Design prompt

[1322] Step 4:

[1323] The server sends the generated design prompt to an image generator (such as Stable Diffusion), which reads the prompt and automatically generates a character image.

[1324] Input: Design prompt

[1325] Output: Generated character image

[1326] Step 5:

[1327] The server sends the generated character image to a smartphone app and an HMD (such as Oculus Quest 2), where it is displayed in real time in the virtual environment. The user can view the character within this environment.

[1328] Input: Generated character image

[1329] Output: Image of the character displayed in the virtual environment

[1330] Step 6:

[1331] The user checks the character image in the virtual environment and re-enters any necessary corrections through the smartphone app, which then sends the re-entered data to the server.

[1332] Input: Correction Requirement

[1333] Output: Re-entered modified requirements data

[1334] Step 7:

[1335] The server again generates design prompts and sends them to the image generator to generate a modified character image, and this process is repeated until the user is satisfied.

[1336] Input: Re-entered modified requirement data

[1337] Output: Modified character image

[1338] Step 8:

[1339] Once the final character image has been finalized, the server sends this image data to a smartphone app, where the user can save or download it.

[1340] Input: Final generated character image

[1341] Output: Saved or downloaded character image

[1342] By following these steps, users can efficiently generate high-quality characters and then check and modify them in real time in a virtual environment.

[1343] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1344] The system of the present invention generates more optimal character design proposals by combining not only user input data but also an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described below.

[1345] Receiving User Input

[1346] Using a terminal, users can input the design requirements for the character (type, characteristics, color, theme, etc.) For example, a pet supply company might input requirements such as "a Shiba Inu character with a lively personality and orange as the main color."

[1347] Use of emotion engine

[1348] When a user inputs design requirements, the emotion engine analyzes the user's emotions. This can be done using voice input and facial expression recognition technology. For example, when a user inputs a "cheerful character," the engine analyzes their intention from their tone of voice and facial expression.

[1349] Generate Design Prompts

[1350] The server receives the user's input data and the analysis results of the emotion engine. Based on this data, a natural language generation model is invoked to generate detailed design prompts. For example, a specific prompt such as "Shiba Inu, energetic, orange character" is generated.

[1351] Executing image generation

[1352] The generated design prompt is sent from the server to the image generation means, which reads the prompt and automatically generates a character image. Through this process, multiple character image ideas are generated in a short period of time.

[1353] Return and display of generated images

[1354] The generated character image is sent from the server to the terminal and displayed to the user. The user can check the generated image and re-enter correction requirements if necessary. If no corrections are required, the user can save or download the final character image.

[1355] Specific examples

[1356] Scenario: A pet supply company (the user) wants to design a new dog character.

[1357] 1. The user uses the terminal interface to input detailed requirements for a dog character (e.g., type: dog, breed: Shiba Inu, personality: energetic, main color: orange).

[1358] 2. The emotion engine analyzes the user's voice and facial expressions when inputting design requirements, such as a smile or bright tone to express a lively personality.

[1359] 3. The device sends the input data and the emotion engine's analysis results to the server.

[1360] 4. The server receives this data and uses a natural language generation model to generate detailed design prompts, such as "Shiba Inu, energetic, orange character."

[1361] 5. The generated design prompt is sent to an image generating means, and a character image is automatically generated.

[1362] 6. The generated character image is sent from the server to the device and displayed to the user.

[1363] 7. The user checks the generated image and re-enters any necessary corrections. Once the final image is confirmed, the user saves or downloads it.

[1364] This system enables more intuitive and effective character design that reflects the user's emotions, reducing expensive design costs and enabling high-quality character generation in a short period of time. The use of an emotion engine makes it possible to generate design prompts that more accurately reflect the user's intentions, making it particularly effective when character design that appeals to emotions is required.

[1365] The processing flow will be explained below.

[1366] Step 1:

[1367] The user inputs the character's design requirements (type, characteristics, color, theme) using a dedicated interface on the device. For example, the user might input "Shiba Inu character, lively personality, main color orange."

[1368] Step 2:

[1369] The device sends the user's input data to the emotion engine, which then analyzes the user's emotions using their voice input and facial expression recognition. For example, when the user says "I'm fine," the emotion is read from the tone of voice and facial expression.

[1370] Step 3:

[1371] The emotion engine returns the analysis results to the device. The analysis results include data indicating the user's emotions. For example, it may return information such as "The user is expressing a cheerful emotion."

[1372] Step 4:

[1373] The device sends the user input data and the analysis results of the emotion engine to the server, which then receives both the user's design requirements and emotions.

[1374] Step 5:

[1375] The server invokes a generation method (natural language generation model) based on the received data to generate a detailed design prompt. For example, a prompt such as "Shiba Inu, energetic, orange character" is generated.

[1376] Step 6:

[1377] The server transmits the generated design prompt to the image generating means, which reads the prompt and automatically generates a character image.

[1378] Step 7:

[1379] The image generating means returns the generated character image to the server, which receives and temporarily stores the image.

[1380] Step 8:

[1381] The server sends the generated character image to the terminal, where it is converted into a format that can be viewed by the user and sent to the terminal.

[1382] Step 9:

[1383] The device displays the generated character image to the user. The user can check the image and input corrections as needed. For example, they can input corrections such as "I want the expression to be a little brighter."

[1384] Step 10:

[1385] If the user is satisfied with the image, they can save or download the final character image. If any corrections are needed, they can repeat the process from step 1 again.

[1386] Step 11:

[1387] Once the user has finalized the character image, the system will save the image and make it available to the user for marketing and commercial use.

[1388] In this way, the system efficiently and effectively generates character designs that reflect the user's emotions. The introduction of an emotion engine makes it possible to generate design prompts that accurately reflect the user's intentions and emotions, resulting in results that are more satisfying for the user.

[1389] Example 2

[1390] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1391] In modern character design, there are only a limited number of systems that can accurately reflect the user's emotions and intentions. Conventional design systems rely solely on user input data, which often results in inaccurate reflection of the user's emotions and intentions. Furthermore, the design process is complex and time-consuming, making it difficult to generate high-quality characters quickly. This invention was developed to address these issues.

[1392] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1393] In this invention, the server includes means for receiving user input data, means for utilizing an emotion engine that analyzes the user's emotions, and means for generating design prompts based on the received input data and the analyzed emotion data, thereby enabling the rapid generation of high-quality character images that reflect the user's emotions and intentions.

[1394] "User input data" refers to design requirement information input by a user through a terminal for character creation, including the character's type, characteristics, color, and theme.

[1395] The "emotion engine" is a technology that analyzes the user's tone of voice and facial expressions when inputting information and recognizes the user's emotions in real time.

[1396] The "generator" is a system component for generating detailed design prompts based on received user input data and analyzed emotion data, primarily utilizing natural language generation models.

[1397] A "design prompt" is a set of instructions containing specific character design details that are generated based on user input data and emotion data.

[1398] The "image generation means" is a system component that automatically generates character images based on the generated design prompts. It mainly uses an image generation model.

[1399] "User device" refers to an electronic device used by a user to input data for character creation and to view the generated character image. Examples include personal computers and smartphones.

[1400] The system of the present invention generates intuitive and effective character design proposals by combining user input data with an emotion engine. A specific embodiment of this system will be described below.

[1401] Hardware and Software Configuration

[1402] Hardware used

[1403] User Device: Electronic devices such as computers, tablets, and smartphones.

[1404] Server: A cloud-based server with a powerful processor and lots of memory.

[1405] Software used

[1406] Emotion Engine: Speech and facial recognition software to analyze the user's tone of voice and facial expressions.

[1407] Generator: Software that generates design prompts using a natural language generation model (e.g., GPT-3).

[1408] Image generation means: Software that generates character images using an image generation model (e.g., DALL-E).

[1409] Details of the main process

[1410] Receiving User Input

[1411] The user inputs the character design requirements using the user device. For example, the user can specify details such as "Shiba Inu character, lively personality, main color is orange." The device receives this input data and temporarily stores it.

[1412] Use of emotion engine

[1413] When a user inputs design requirements, the emotion engine analyzes the user's voice and facial expressions in real time. For example, when a user inputs "a lively character," the engine analyzes their intention from the tone of their voice and facial expression. The device acquires this emotion data and saves the analysis results.

[1414] Generate Design Prompts

[1415] The device sends the user's input data and the emotion engine's analysis results to the server, which receives this data and uses a natural language generation model to generate detailed design prompts. For example, a specific prompt such as "Shiba Inu, energetic, orange character" is generated.

[1416] Executing image generation

[1417] The generated design prompt is sent from the server to the image generation means, which reads the prompt and automatically generates a character image. Through this process, multiple character image ideas are generated in a short period of time.

[1418] Return and display of generated images

[1419] The generated character images are sent from the server to the user device. The user device receives these images and displays them to the user. The user can review the generated images and re-enter correction requirements as needed. If no corrections are required, the user saves or downloads the final character image.

[1420] Specific examples

[1421] For example, if a pet supply company (user) wants to design a new dog character, the process would go something like this:

[1422] 1. The user uses the terminal interface to input detailed requirements for a dog character (e.g., type: dog, breed: Shiba Inu, personality: energetic, main color: orange).

[1423] 2. The emotion engine analyzes the voice and facial expressions used when users input design requirements, such as smiles and bright tones that express a lively personality.

[1424] 3. The device sends the input data and the emotion engine's analysis results to the server.

[1425] 4. The server receives this data and uses a natural language generation model to generate detailed design prompts, such as "Shiba Inu, energetic, orange character."

[1426] 5. The generated design prompt is sent to an image generating means, and a character image is automatically generated.

[1427] 6. The generated character image is sent from the server to the user device and displayed to the user.

[1428] 7. The user checks the generated image and re-enters any necessary corrections. Once the final image is confirmed, the user saves or downloads it.

[1429] Example prompt sentence:

[1430] "Shiba Inu, a lively, orange character"

[1431] This system enables more intuitive and effective character design that reflects the user's emotions, reducing expensive design costs and enabling high-quality character generation in a short period of time. Furthermore, the use of an emotion engine makes it possible to generate design prompts that more accurately reflect the user's intentions, making it particularly effective when character design is required to appeal to emotions.

[1432] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1433] Step 1: Receiving User Input

[1434] A user inputs design requirements (e.g., type, characteristics, color, theme) of a character using a user device. For example, the user inputs requirements such as "a Shiba Inu character with a lively personality and orange as the main color."

[1435] Input: Character type, traits, color, theme

[1436] Data processing: The device temporarily stores the entered data.

[1437] Output: Input design requirements data

[1438] Specific operation: The user enters "Dog breed: Shiba Inu," "Personality: Energetic," and "Main color: Orange" into the device's input form, and the device receives and temporarily stores this information.

[1439] Step 2: Use the Emotion Engine

[1440] As users input their design requirements, the emotion engine analyzes their tone of voice and facial expressions in real time.

[1441] Input: User's voice data, facial expression data

[1442] Data processing: The device sends the voice data and facial expression images to the analysis engine to obtain emotional data.

[1443] Output: User sentiment analysis results

[1444] Specific operation: When a user types "cheerful personality," the camera and microphone capture the tone of voice and facial expression, and the emotion engine analyzes that "the user has a cheerful emotion."

[1445] Step 3: Generate design prompts

[1446] The device sends the user input data and the emotion engine's analysis results to the server.

[1447] The server receives this data and uses natural language generation models to generate detailed design prompts.

[1448] Input: User input data and sentiment analysis results

[1449] Data processing: The server organizes this data and generates design prompts using a natural language generation model.

[1450] Output: Generated design prompts

[1451] Specific operation: Based on the data "Shiba Inu, lively personality, orange color, lively emotion," the server uses a natural language generation model to generate the prompt "Shiba Inu, lively, orange character."

[1452] Step 4: Running Image Generation

[1453] The server sends the generated design prompt to the image generating means.

[1454] An image generating means automatically generates a character image based on the design prompt.

[1455] Input: Generated design prompt

[1456] Data processing: The image generation means reads the design prompt and generates a character image using the image generation model.

[1457] Output: Multiple character image ideas

[1458] Specific operation: The server passes the prompt "Shiba Inu, energetic, orange character" to the image generation model, and multiple character images are automatically generated.

[1459] Step 5: Return and display the generated image

[1460] The server transmits the generated character image to the user device.

[1461] The terminal receives these images and displays them to the user.

[1462] Input: Generated character image

[1463] Data processing: The device receives the image data and prepares it for display to the user.

[1464] Output: Generated character image that can be viewed by the user

[1465] Specific operation: The server sends a character image to the terminal, and the terminal displays the image to the user.

[1466] Step 6: Re-enter the corrections and save the final image

[1467] The user can review the generated image and re-enter any necessary corrections. If no corrections are required, the user can save or download the final character image.

[1468] Input: User correction requirements or save / download requests

[1469] Data processing: The terminal sends the modified requirements back to the server, which generates a new design prompt.

[1470] Output: Revised character image or final character image

[1471] Specific operation: The user checks the character image and inputs any correction requirements, such as "I would like the color to be a little brighter." The device sends this information to the server, which then generates a new design prompt and generates a new character image. The user can then save or download the final image they are satisfied with.

[1472] (Application example 2)

[1473] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1474] Conventional character design systems generate character images based solely on user input data, which means they are unable to fully reflect the user's emotions and intentions. Furthermore, the quality and intuitive design of the generated characters often fall short of user satisfaction. This results in increased costs and time for character design, making it difficult to respond to situations where emotionally appealing characters are required.

[1475] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1476] In this invention, the server includes means for receiving user input data, means for generating design prompts based on the received input data and analyzed emotion data, means for transmitting the generated design prompts to an image generation means to generate a character image, and means for returning the generated character image to a user device for display. This enables the automatic generation of high-quality, intuitive character images that reflect the user's emotions and intentions.

[1477] "User-input data" refers to information containing specific requirements a user has for character generation, such as the character's category, attributes, primary color, and motif.

[1478] "Emotion data" is data obtained by analyzing the emotions and intentions expressed by users when designing characters. It is acquired using voice input and facial expression recognition technology.

[1479] "Design prompts" are instructions for character design that are generated based on user input data and emotion data. Images are generated based on these.

[1480] "Image generation means" refers to a system or software that reads design prompts and automatically generates character images, for example, using a generative AI model.

[1481] A "user device" is a terminal used by a user to display the generated character image, and includes, for example, mobile devices such as smartphones and tablets.

[1482] A "natural language generation model" is an AI technology that analyzes user-input data and generates design prompts in a format that is easy for humans to understand. A representative example is GPT-3.

[1483] An "emotion analysis engine" is a system that analyzes emotions from the user's voice, facial expressions, etc., and generates emotional data, which allows the user's intentions to be reflected more accurately.

[1484] The present invention relates to a system for generating a character image based on user input data and emotion data. Specific embodiments are described below.

[1485] Receiving User Input

[1486] A user uses a user device such as a smartphone or tablet to input character design requirements, including specific requirements such as the character's category (e.g., animal or human), attributes (e.g., friendliness or liveliness), primary color, and motif.

[1487] Emotional Data Analysis

[1488] When a user inputs design requirements, the emotion analysis engine uses voice input and facial expression recognition technology to analyze the user's emotions. This involves analyzing the tone of voice and facial expressions when the user inputs a "friendly character" and recording the emotional data.

[1489] Generate Design Prompts

[1490] The input data and emotion data sent from the user device are received by a server. The server uses a natural language generation model (e.g., GPT-3) to generate a design prompt based on the received data. The generated design prompt becomes a specific instruction sentence, such as "blue cat, friendly, smiling, simple design."

[1491] Executing image generation

[1492] The generated design prompts are sent from the server to an image generation means (e.g., a generative AI model such as DALL-E or Stable Diffusion). This image generation means reads the design prompts and automatically generates corresponding character images. Multiple image proposals are generated in a short period of time.

[1493] Return and display of generated images

[1494] The generated character image is sent back to the user's device via the server and displayed to the user. The user can check this image and re-enter correction requirements as necessary. When the user is finally satisfied with the character image, they can save or download it.

[1495] Specific example explanation

[1496] For example, if a user wants to design a blue cat character for a pet supplies online store, the following prompt sentence is generated:

[1497] Example prompt sentence:

[1498] "Blue cat, friendly, smiling, simple design"

[1499] This allows users to quickly obtain high-quality character images that reflect their intentions, reducing design costs and improving the efficiency of character generation.

[1500] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1501] Step 1:

[1502] The user inputs the character design requirements using the terminal.

[1503] Specific actions: Enter data such as the character's category, attributes, primary color, and motif in the terminal's application interface.

[1504] Input: Character design requirements (e.g. blue cat, friendly, smiling).

[1505] Output: The entered data is sent from the terminal to the server.

[1506] Step 2:

[1507] The server receives the user input data and analyzes the user's emotions using an emotion analysis engine.

[1508] Specific operation: The server receives the voice and facial expression data sent along with the input data and generates emotion data using an emotion analysis engine.

[1509] Input: User input data, voice data, facial expression data.

[1510] Output: Emotion data is generated and stored in the server.

[1511] Step 3:

[1512] A server generates design prompts based on the user input data and the emotion data.

[1513] Specific operation: Using a natural language generation model (e.g., GPT-3), analyze user input data and sentiment data to generate design prompts.

[1514] Input: User input data, emotion data.

[1515] Output: Design prompt (e.g., "Blue cat, friendly, smiling, simple design").

[1516] Step 4:

[1517] The server transmits the generated design prompt to the image generating means, which generates a character image.

[1518] Specific operations: Provide design prompts to an image generation means (e.g., DALL-E or Stable Diffusion) to generate a character image.

[1519] Input: Design prompt.

[1520] Output: Generated character image.

[1521] Step 5:

[1522] The server returns the generated character image to the terminal and displays it to the user.

[1523] Specific operation: The generated character image is transferred from the server to the terminal and displayed on the user device's display.

[1524] Input: Generated character image.

[1525] Output: The generated character image is displayed on the terminal.

[1526] Step 6:

[1527] The user checks the generated image and re-enters correction requirements if necessary.

[1528] Specific operation: The user checks the displayed character image, and if any corrections are necessary, corrects the input data and emotion data again and sends them.

[1529] Input: Modification requirements (e.g. change primary color, change attributes, etc.).

[1530] Output: The corrected input data is sent from the terminal to the server.

[1531] Step 7:

[1532] The server regenerates the design prompts based on the modified input data, updates the character image, and returns the final character image to the user for saving or download.

[1533] Specific operations: Emotion analysis and design prompt generation are performed again based on the corrected input data, the character image is updated using the image generation means, and the final character image is returned to the user.

[1534] Input: Modified input data, emotion data.

[1535] Output: The final character image is sent back to your device and can be saved or downloaded.

[1536] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1537] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1538] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1539] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1540] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1541] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1542] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1543] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1544] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1545] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1546] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1547] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1548] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1549] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1550] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1551] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1552] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1553] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1554] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1555] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1556] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1557] The following is further disclosed regarding the above embodiment.

[1558] (Claim 1)

[1559] means for receiving user input data;

[1560] generating means for generating a design prompt based on the received input data;

[1561] means for transmitting the generated design prompt to an image generating means and generating a character image;

[1562] means for returning the generated character image to the user device and displaying it;

[1563] A system including:

[1564] (Claim 2)

[1565] 10. The system of claim 1, wherein the user input data includes a character type, characteristics, color, and theme.

[1566] (Claim 3)

[1567] 2. The system of claim 1, wherein the generating means generates the design prompts using a natural language generation model.

[1568]

[1569] "Example 1"

[1570] (Claim 1)

[1571] means for receiving user input data;

[1572] generating means for generating a design prompt based on the received input data;

[1573] means for transmitting the generated design prompt to an image generating means for generating a plurality of character images;

[1574] means for returning the generated character image to the user device and displaying it;

[1575] A means for a user to input correction requirements for the generated character image;

[1576] A system including:

[1577] (Claim 2)

[1578] 10. The system of claim 1, wherein the user input data includes a character type, characteristics, color, and theme.

[1579] (Claim 3)

[1580] 2. The system of claim 1, wherein the generating means generates the design prompts using a natural language generation model.

[1581] "Application Example 1"

[1582] (Claim 1)

[1583] means for receiving user input data;

[1584] generating means for generating a design prompt based on the received input data;

[1585] means for transmitting the generated design prompt to an image generating means and generating a character image;

[1586] means for returning the generated character image to the user device and displaying it;

[1587] an interface means that allows the user to check the character image generated in the virtual environment in real time;

[1588] a means by which the user can re-enter modified requirements;

[1589] A way to save or download the final character image;

[1590] A system including:

[1591] (Claim 2)

[1592] 10. The system of claim 1, wherein the user input data includes a character type, characteristics, color, and theme.

[1593] (Claim 3)

[1594] 2. The system of claim 1, wherein the generating means generates the design prompts using a natural language generation model.

[1595] "Example 2: Combining Emotion Engines"

[1596] (Claim 1)

[1597] means for receiving user input data;

[1598] a means for utilizing an emotion engine for analyzing user emotions;

[1599] generating means for generating design prompts based on the received input data and the analyzed emotion data;

[1600] means for transmitting the generated design prompt to an image generating means and generating a character image;

[1601] means for returning the generated character image to the user device and displaying it;

[1602] A system including:

[1603] (Claim 2)

[1604] 10. The system of claim 1, wherein the user input data includes a character type, characteristics, color, and theme.

[1605] (Claim 3)

[1606] 2. The system of claim 1, wherein the generating means generates the design prompts using a natural language generation model.

[1607] "Application example 2 when combining emotion engines"

[1608] (Claim 1)

[1609] means for receiving user input data;

[1610] means for generating design prompts based on the received input data and the analyzed emotion data;

[1611] means for transmitting the generated design prompt to an image generating means and generating a character image;

[1612] means for returning the generated character image to the user device and displaying it;

[1613] A system including:

[1614] (Claim 2)

[1615] 10. The system of claim 1, wherein the user input data includes a character's category, attributes, primary color, and motif.

[1616] (Claim 3)

[1617] 2. The system of claim 1, wherein the generating means generates the design prompts using a natural language generation model and a sentiment analysis engine. [Explanation of symbols]

[1618] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving user input data; generating means for generating a design prompt based on the received input data; means for transmitting the generated design prompt to an image generating means and generating a character image; means for returning the generated character image to the user device and displaying it; A system including:

2. 10. The system of claim 1, wherein the user input data includes a character type, characteristics, color, and theme.

3. 2. The system of claim 1, wherein the generating means generates the design prompts using a natural language generation model.

Images