System

Abstract

To provide a system.SOLUTION: The system includes: means for acquiring a prompt sentence including an instruction or a request to input to a generative AI model; means for generating the figure of a character by using an image generative AI on the basis of the prompt sentence; means for generating a self-consciousness of the character by using a sentence generative AI on the basis of the prompt sentence; means for generating voice of the character by using a voice generative AI on the basis of the prompt sentence; and means for integrating the figure, the self-consciousness, and voice of the generated character and forming a consistent character setting.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional advertising characters are expensive to film and carry the risk of scandals involving the actors. Furthermore, it is difficult to meet the demands of customizing characters, global compatibility, continuous availability, and compatibility with various media. [Means for solving the problem]

[0005] This invention uses an image generation AI to generate appearances, a text generation AI to generate ego, and a voice generation AI to generate voices, to generate characters that can be used on social media. This reduces filming costs and avoids the risk of scandals. Furthermore, the characters are customizable, globally compatible, always-on, and compatible with various media, and can appear in online events and have their own unique story and worldview. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disk), or magnetic tape.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] One aspect of the present invention is a system that uses image generation AI, text generation AI, and voice generation AI to generate characters that can be used on social media. The image generation AI generates the character's appearance. The text generation AI generates the character's ego, that is, its personality and thoughts. The voice generation AI generates the character's voice. These AIs generate characters with consistent character settings that can be used on social media.

[0029] "Example 2"

[0030] In one embodiment of the present invention, the generated characters are used to advertise apparel, food, furniture, and home appliances. Specifically, the products are promoted by posting on social media a scene of the character wearing a particular apparel item or eating a particular food. This reduces filming costs and avoids the risk of scandals.

[0031] "Example 3"

[0032] In one embodiment of the present invention, the generated character has characteristics such as customizability, global compatibility, continuous operation, and compatibility with various media, and can appear in online events and have its own unique story and worldview. Specifically, the character's appearance, personality, voice, etc. can be customized according to the user's requests. Furthermore, the character can be operated 24 hours a day and can be accessed from anywhere in the world. Furthermore, the character can be active in various media such as television, radio, and the Internet.

[0033] The processing flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: Generate the character's appearance using image generation AI. Specifically, set the character's features such as hair color, eye color, and body type.

[0036] Step 2: The text generation AI generates the character's ego, specifically setting the character's personality, hobbies, likes, dislikes, and other characteristics.

[0037] Step 3: The voice generation AI generates the character's voice. Specifically, the pitch, tone, speaking style, and other characteristics of the character's voice are set.

[0038] "Example 2"

[0039] Step 1: Create an advertisement for an apparel product using the generated character. Specifically, generate an image of the character wearing a specific apparel product.

[0040] Step 2: Post the generated advertising image to social media. Specifically, post an image of a character wearing the product along with detailed product information and how to purchase it.

[0041] "Example 3"

[0042] Step 1: Customize the generated character's appearance, personality, voice, etc. according to the user's requests. Specifically, change the character's hair color, personality, etc., based on the user's requests.

[0043] Step 2: The customized character will appear in online events. Specifically, the character will participate in online events, interact with users, promote products, etc.

[0044] Example 1

[0045] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0046] With conventional character generation systems, it was difficult to consistently generate a character's appearance, personality, and voice, making it impossible to efficiently create characters suitable for social media management. Furthermore, in the case of advertisements and online events, the high cost of filming and the risk of scandals meant that customization and global support were required.

[0047] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0048] In this invention, the server includes a means for receiving a prompt text entered by a user, a means for generating a character's appearance using an image generation AI based on the prompt text, a means for generating a character's ego using a sentence generation AI based on the prompt text, a means for generating a character's voice using a voice generation AI based on the prompt text, a means for integrating the generated appearance, personality, and voice to create a consistent character profile, and a means for providing the character profile to the user. This allows users to efficiently generate characters with consistent profile settings suitable for use on social media, reducing filming costs and avoiding the risk of scandals in advertisements and online events. Furthermore, characters with characteristics such as customizability, global compatibility, continuous operation, and compatibility with various media can be generated.

[0049] A "prompt sentence" is text containing instructions or requests that a user inputs to a generative AI model.

[0050] "Image generation AI" is an artificial intelligence model that generates images based on input prompt text.

[0051] "Sentence generation AI" is an artificial intelligence model that generates sentences based on input prompts.

[0052] "Speech generation AI" is an artificial intelligence model that generates speech based on an input prompt.

[0053] "Character appearance" refers to a character's appearance and physical characteristics.

[0054] "Character ego" refers to a character's personality, thoughts, and behavior patterns.

[0055] "Character voice" refers to the sounds and speaking characteristics of a character.

[0056] "Consistent character settings" refers to the overall image of a character with consistent characteristics in appearance, personality, and voice.

[0057] "SNS-operable character" refers to a character that can be active on a social networking service.

[0058] "Reducing filming costs" refers to reducing the costs of actual filming by using characters.

[0059] "Scandal risk avoidance" refers to the avoidance of the risk of scandals related to real people by using characters.

[0060] "Customizability" refers to the ability to change a character's characteristics according to the user's needs.

[0061] "Global compatibility" refers to the characteristics of a character that can adapt to different languages ​​and cultures.

[0062] "Always-on" refers to the ability of a character to be active 24 hours a day, 365 days a year.

[0063] "Multimedia compatible" refers to the ability of a character to operate in different media formats (e.g., images, video, audio).

[0064] "Appearing at an online event" refers to a character's participation in an event held via the Internet.

[0065] "Unique story and worldview" refers to the unique background, story, and setting that each character has.

[0066] This invention is a system that generates a character with consistent character settings that can be used in SNS management using image generation AI, text generation AI, and voice generation AI based on a prompt sentence entered by a user. A specific embodiment of this system is described below.

[0067] Hardware and software used

[0068] The server uses the following hardware and software:

[0069] Hardware: High-performance server computer

[0070] Software: Image generation AI (e.g., DALL-E, Stable Diffusion), text generation AI (e.g., GPT-4 (registered trademark)), speech generation AI (e.g., Tacotron 2, WaveNet)

[0071] Data processing and calculation

[0072] The server receives the prompt text entered by the user from the terminal and passes it to each generative AI model in an appropriate format. Specifically, it receives the following prompt text:

[0073] Appearance: "Young female character, blue hair, casual clothing"

[0074] Personality: "This character is curious and loves adventure."

[0075] Voice: "This character's voice has a bright, cheerful tone."

[0076] The server analyzes the received prompt sentence and performs the following process.

[0077] Appearance generation using image generation AI

[0078] The server passes a prompt about appearance to the image generation AI. The image generation AI generates an image of a character with the specified characteristics and returns it to the server. For example, based on the prompt "young female character, blue hair, casual clothing," it generates an image of the corresponding character.

[0079] Personality generation using sentence generation AI

[0080] The server passes a prompt about the character's personality to the sentence generation AI. The sentence generation AI generates a sentence about the character's personality and thoughts with the specified characteristics and returns it to the server. For example, based on the prompt "This character is curious and loves adventure," it generates a sentence that describes the character's personality in detail.

[0081] Voice generation using AI

[0082] The server passes a voice prompt to the voice generation AI. The voice generation AI generates a character's voice with the specified characteristics and returns it to the server. For example, based on the prompt, "This character's voice has a bright, cheerful tone," it generates a voice file for the corresponding character.

[0083] Character Settings Integration

[0084] The server integrates the generated images, text, and audio to create a consistent character profile, which is then saved in a format suitable for SNS management.

[0085] Character provision

[0086] The server sends the generated character settings to the device. The user can then check the character through the device and start using it on the SNS. For example, the user can post on the SNS using the generated character.

[0087] In this way, users can efficiently create characters with consistent character settings that can be used on social media, reducing filming costs and avoiding the risk of scandals when it comes to advertising and online events. Characters can also be created with features such as customizability, global compatibility, continuous operation, and compatibility with various media.

[0088] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0089] Step 1:

[0090] The user inputs a prompt into the input form on the terminal. For example, the following prompt is input:

[0091] Appearance: "Young female character, blue hair, casual clothing"

[0092] Personality: "This character is curious and loves adventure."

[0093] Voice: "This character's voice has a bright, cheerful tone."

[0094] The entered prompt sentence is sent from the terminal to the server.

[0095] Step 2:

[0096] The server analyzes the prompts received from the device. The analyzed prompts are passed to each generation AI model in an appropriate format. For example, prompts about appearance are passed to the image generation AI, prompts about personality are passed to the sentence generation AI, and prompts about voice are passed to the voice generation AI.

[0097] Step 3:

[0098] The server passes a prompt about appearance to the image generation AI. The image generation AI generates an image of a character with the specified characteristics. For example, based on the prompt "young female character, blue hair, casual clothing," it generates an image of the corresponding character. The generated image is returned to the server.

[0099] Step 4:

[0100] The server passes a prompt about the character's personality to the sentence generation AI. The sentence generation AI generates a sentence about the character's personality and thoughts with the specified characteristics. For example, based on the prompt "This character is curious and loves adventure," it generates a sentence that describes the character's personality in detail. The generated sentence is returned to the server.

[0101] Step 5:

[0102] The server passes a voice prompt to the voice generation AI. The voice generation AI generates a character's voice with the specified characteristics. For example, based on the prompt, "This character's voice has a bright, cheerful tone," it generates a voice file for the corresponding character. The generated voice file is returned to the server.

[0103] Step 6:

[0104] The server integrates the generated images, text, and audio to create a consistent character profile. For example, the server combines the generated images, text about the character's personality, and audio files to create a unified character profile. This character profile is stored in a format suitable for SNS management.

[0105] Step 7:

[0106] The server sends the generated character settings to the device. The user can then check the character through the device and start using it on the SNS. For example, the user can post on the SNS using the generated character.

[0107] (Application example 1)

[0108] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0109] With conventional character generation systems, it was difficult to generate virtual influencers with consistent character settings for social media management and content distribution services. Furthermore, the cost and time required to individually set the character's appearance, personality, and voice made efficient management difficult. Furthermore, filming costs and the risk of scandals were also issues for advertising and online events.

[0110] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0111] In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a text generation AI, a voice generation means using a voice generation AI, and a means for managing a virtual influencer in a content distribution service using the generated character. This allows for efficient generation of virtual influencers with consistent character settings and their use in social media management and content distribution services. Furthermore, it also enables reductions in filming costs and avoidance of scandal risks in advertising and online events.

[0112] "Image generation AI" is an artificial intelligence technology that generates images based on input text or prompts.

[0113] "Sentence generation AI" is an artificial intelligence technology that generates natural-sounding sentences based on input text and prompts.

[0114] "Speech generation AI" is an artificial intelligence technology that generates speech based on input text.

[0115] "Appearance generation means" refers to a means of generating a character's appearance or figure using image generation AI.

[0116] "Self-generation means" is a means of generating a character's personality and thoughts using text generation AI.

[0117] "Voice generation means" refers to a means of generating a character's voice using voice generation AI.

[0118] A "virtual influencer" is a virtual influencer who uses a generated character to be active on social media and content distribution services.

[0119] A "content distribution service" is a service that distributes content such as video, audio, and text via the Internet.

[0120] "Filming costs" are the expenses required to shoot characters and advertising materials.

[0121] "Scandal risk" refers to the risk of scandal or problematic behavior that a character or influencer may cause.

[0122] An "online event" is an event held over the internet.

[0123] "Customizability" refers to the ability of users to freely change a character's appearance, personality, voice, etc.

[0124] "Global compatibility" refers to the ability of characters and systems to support multiple languages ​​and cultures.

[0125] "Always-on operation" refers to the characteristic of characters and systems operating 24 hours a day, 365 days a year.

[0126] "Multimedia compatibility" refers to the ability of characters and systems to support multiple media formats such as video, audio, and text.

[0127] A "unique story" is a character's unique background or story.

[0128] A "worldview" is the virtual environment or setting in which a character operates.

[0129] The following system configuration and processing procedure will be described as an embodiment of the present invention.

[0130] System Configuration

[0131] This system uses the following hardware and software:

[0132] Hardware: Servers, smartphones, PCs

[0133] Software: Python, OpenAI® API, PIL (Python Imaging Library), pyttsx3 (Python Text-to-Speech)

[0134] Processing steps

[0135] The server includes the following means:

[0136] 1. Image generation method: Using image generation AI, generate the character's appearance based on the input prompt.

[0137] 2. Self-generation method: Using text generation AI, generate a character's personality and thoughts based on the input prompts.

[0138] 3. Voice generation method: Using voice generation AI, a character's voice is generated based on the input text.

[0139] 4. Virtual influencer management means: Using the generated characters, virtual influencers are managed in content distribution services.

[0140] Data processing and calculation

[0141] The server processes and calculates the data as follows.

[0142] 1. Image Generation: The server uses the OpenAI API to generate an image of the character based on the prompt, and then converts the image to the appropriate format using PIL.

[0143] 2. Sentence Generation: The server uses the OpenAI API to generate the character's personality and thoughts based on the prompts. The generated sentences are saved as the character's ego.

[0144] 3. Voice generation: The server uses pyttsx3 to convert the generated text into an audio file, which generates the character's voice.

[0145] 4. Virtual Influencer Management: The server uses the generated character to automate activities on social media and content distribution services, including scheduling posts and interacting with fans.

[0146] Specific examples

[0147] Below are some examples of prompt sentences.

[0148] Image prompt: "A friendly and charismatic virtual influencer"

[0149] Writing prompt: "Describe the personality and daily activities of a virtual influencer who loves fashion and technology."

[0150] Based on this prompt, the server generates the character's appearance, personality, and voice, and allows the character to act as a virtual influencer on social media and content distribution services.

[0151] In this way, this invention makes it possible to efficiently generate virtual influencers with consistent character settings and utilize them in social media management and content distribution services. Furthermore, it also makes it possible to reduce filming costs and avoid the risk of scandals in advertising and online events.

[0152] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0153] Step 1:

[0154] The user inputs prompts about the character's appearance, personality, and voice using a smartphone or PC, which are then sent to the server.

[0155] Input: Prompts about the character's appearance, personality, and voice

[0156] Output: The prompt sent to the server

[0157] Step 2:

[0158] The server uses image generation AI to generate the character's appearance based on the input prompt, and the generated image is converted to the appropriate format using PIL.

[0159] Input: Prompt for character appearance

[0160] Output: An image of the generated character

[0161] Step 3:

[0162] The server uses a sentence generation AI to generate the character's personality and thoughts based on the input prompt sentence, and the generated sentence is saved as the character's ego.

[0163] Input: Character personality prompt

[0164] Output: Text about the generated character's personality and thoughts

[0165] Step 4:

[0166] The server uses a speech generation AI to convert the generated text into an audio file, which then generates the character's voice.

[0167] Input: Text about the character's personality and thoughts

[0168] Output: The generated character's voice

[0169] Step 5:

[0170] The server uses the generated character to automate activities on social media and content distribution services, including scheduling posts and interacting with fans.

[0171] Input: Image of the generated character, text about its personality and thoughts, and voice

[0172] Output: Character activities on social media and content distribution services

[0173] In this way, users can easily create virtual influencers and have them active on social media and content distribution services.

[0174] Example 2

[0175] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0176] Traditional advertising methods require the use of real models and locations, which not only increases filming costs but also poses the risk of scandal. While there is a demand for systems that offer features such as character customization, global compatibility, continuous operation, and compatibility with a variety of media, there is a lack of efficient methods to achieve this.

[0177] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0178] In this invention, the server includes an image generation means, a sentence generation means, a voice generation means, a means for inputting a prompt sentence, a means for analyzing the prompt sentence, a means for converting the generated data into a format for posting to an SNS, and a means for posting the formatted data to the SNS. This reduces filming costs, avoids the risk of scandals, and enables character customization, global compatibility, continuous operation, and compatibility with a variety of media.

[0179] "Image generation means" refers to a technique for generating an image based on a prompt sentence.

[0180] A "sentence generator" is a technique for generating text based on a prompt sentence.

[0181] The "voice generation means" is a technique for generating voice based on a prompt sentence.

[0182] The "means for inputting a prompt sentence" is an interface for the user to input the content they wish to generate in text format.

[0183] "Means for analyzing prompt sentences" refers to technology for analyzing input prompt sentences and converting them into a format suitable for a generative AI model.

[0184] "Means for converting generated data into a format for posting on SNS" refers to technology for converting generated images and text into a format suitable for SNS.

[0185] "Means for posting formatted data to a social networking site" refers to a technology for posting formatted data to a specified social networking site account.

[0186] This invention relates to a system for generating characters using a generative AI model and posting the characters to a social networking service. Specific embodiments of this system are described below.

[0187] System configuration

[0188] Hardware

[0189] Server: Runs the generative AI model, analyzes prompts, and converts data formats.

[0190] Terminal: A device on which a user enters a prompt, such as a computer, smartphone, or tablet.

[0191] software

[0192] Generative AI model: An AI model that generates images, text, and speech. Specifically, OpenAI's GPT-4 and DALL-E are used.

[0193] Social media posting formatter: Software that converts the generated data into a format suitable for social media.

[0194] Program processing

[0195] Entering a prompt statement

[0196] Using a dedicated application on a device or a web interface, a user enters a prompt that describes the scene they want to create for the character, for example, "Create a scene where the character is wearing a red dress."

[0197] Sending a prompt

[0198] The terminal sends the prompt text entered by the user to the server, where it is converted into an appropriate format and sent to the server.

[0199] Parsing the prompt statement

[0200] The server analyzes the received prompt and converts it into a format suitable for the generative AI model, for example, extracting keywords such as "character wearing a red dress."

[0201] Requests to generative AI models

[0202] The server sends a request to the generative AI model based on the analysis results, which includes the content of the prompt sentence.

[0203] Character image generation

[0204] The generative AI model receives requests from the server and generates a character image based on the specified scene, for example, an image of a character wearing a red dress.

[0205] Receiving the generated image

[0206] The server receives the character image returned by the generative AI model, which is generated based on the prompt text.

[0207] Image format conversion

[0208] The server then converts the received image into a format suitable for posting to the social networking site, for example by adjusting the size of the image and adding a caption.

[0209] Posting to social media

[0210] The server posts the formatted image to a specified social media account, which then publishes a scene of the generated character wearing a specific apparel item or eating a specific food item on the social media.

[0211] Specific examples

[0212] For example, if the user enters the prompt "Generate a scene where the character is wearing a red dress," the following specific actions will occur:

[0213] 1. The user types into the device application, "Generate a scene where the character is wearing a red dress."

[0214] 2. The terminal sends this prompt to the server.

[0215] 3. The server parses the prompt and extracts the keyword "character wearing a red dress."

[0216] 4. The server sends a request to the generation AI model saying, "Generate a character wearing a red dress."

[0217] 5. The generative AI model generates an image of a character wearing a red dress.

[0218] 6. The server receives the generated image and adjusts the size of the image for posting on social media.

[0219] 7. The server adds the caption "Check out our new red dress!" to the adjusted image.

[0220] 8. The server posts this image to the company's social media accounts.

[0221] This system allows companies to advertise efficiently, reduce filming costs, and avoid the risk of scandals.

[0222] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0223] Step 1:

[0224] The user enters a prompt statement.

[0225] Using a dedicated application or web interface on the device, the user inputs a prompt that describes the scene they want to create for the character. For example, they might input "Create a scene where the character is wearing a red dress." The input prompt is then stored in the device's memory.

[0226] Step 2:

[0227] The terminal sends the prompt to the server.

[0228] The terminal sends the prompt text entered by the user to the server. At this time, the prompt text is converted into an appropriate format and sent to the server over the network. The input is the prompt text, and the output is the prompt text sent to the server.

[0229] Step 3:

[0230] The server parses the prompt.

[0231] The server parses the received prompt and converts it into a format suitable for the generative AI model. For example, it extracts keywords such as "character wearing a red dress." The input is the prompt, and the output is the parsed keywords.

[0232] Step 4:

[0233] The server sends a request to the generative AI model.

[0234] The server sends a request to the generative AI model based on the analysis results. This request includes the contents of the prompt sentence. The input is the analyzed keyword, and the output is the request to the generative AI model.

[0235] Step 5:

[0236] A generative AI model generates character images.

[0237] The generative AI model receives a request from the server and generates a character image based on the specified scene. For example, generate an image of a character wearing a red dress. The input is the request to the generative AI model, and the output is the generated character image.

[0238] Step 6:

[0239] The server receives the generated image.

[0240] The server receives the character image returned by the generative AI model. This image is generated based on the prompt. The input is the generated character image, and the output is the image stored on the server.

[0241] Step 7:

[0242] The server formats the image for posting to social media.

[0243] The server converts the received image into a format suitable for posting to a social networking site, for example by adjusting the size of the image or adding a caption. The input is the generated character image, and the output is the formatted image.

[0244] Step 8:

[0245] The server posts the formatted image to a social networking site.

[0246] The server posts the formatted image to the specified social media account. This allows a scene of the generated character wearing a specific apparel item or eating a specific food to be published on the social media. The input is the formatted image, and the output is the image posted to the social media.

[0247] (Application example 2)

[0248] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0249] Traditional advertising production involves high filming costs and the risk of scandals. Furthermore, character-based advertising requires character customization and compatibility with a variety of media, but the technology required to achieve this is lacking. Furthermore, there are limited ways to automate posting to social media, making efficient advertising management difficult.

[0250] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a sentence generation AI, a voice generation means using a voice generation AI, and a means for automatically creating a scene in which the generated character uses a specific product and posting it to a social networking site. This enables reduction in photography costs, avoidance of the risk of scandals, character customization, compatibility with various media, and automatic posting to a social networking site.

[0251] "Image generation AI" is a system that generates images using artificial intelligence technology.

[0252] "Appearance generation means" is a function that generates a character's appearance using image generation AI.

[0253] "Text generation AI" is a system that generates text using artificial intelligence technology.

[0254] "Self-generation means" is a function that uses text generation AI to generate character personalities and background stories.

[0255] "Voice generation AI" is a system that generates voice using artificial intelligence technology.

[0256] "Voice generation means" is a function that generates a character's voice using voice generation AI.

[0257] The "means for automatically creating a scene in which a specific product is used" is a function for automatically creating a scene in which a generated character uses a specific product.

[0258] "Means for posting to SNS" is a function that automatically posts the generated scene to SNS.

[0259] "Apparel" refers to clothing and fashion items.

[0260] "Food" means food and drink intended for human consumption.

[0261] "Furniture and home appliances" refers to furniture and electrical appliances used in the home.

[0262] "Photography costs" refer to the costs incurred when taking photos or videos for advertising or promotional purposes.

[0263] "Scandal risk" refers to the risk that people or companies involved in advertising may become involved in a scandal.

[0264] "Customizability" refers to the ability for users to freely change the appearance and personality of their characters.

[0265] "Global compatibility" refers to the ability to adapt to different languages ​​and cultures.

[0266] "Continuous operation" refers to the system's ability to operate 24 hours a day, 365 days a year.

[0267] "Multimedia compatibility" refers to the ability of a character to be compatible with multiple media formats (images, videos, audio, etc.).

[0268] "Appearing at an online event" refers to the generated character participating in an event held via the Internet.

[0269] "Unique story and worldview" refers to the unique background story and setting of each character.

[0270] This invention is a system that generates a character using a generative AI model, automatically creates a scene of the character using a specific product, and posts the scene to a social networking site. Specific embodiments of this system are described below.

[0271] System configuration

[0272] The system consists of the following main components:

[0273] 1. Image generation AI: This is an artificial intelligence technology for generating character appearances. Specifically, it uses image generation models such as OpenAI's DALL-E.

[0274] 2. Text generation AI: This is an artificial intelligence technology for generating character personalities and background stories. Specifically, it uses text generation models such as OpenAI's GPT-4.

[0275] 3. Voice Generation AI: This is an artificial intelligence technology for generating character voices. Specifically, it uses voice generation models such as Google's WaveNet.

[0276] 4. SNS posting module: This module automatically posts scenes of generated characters using specific products to SNS.

[0277] System Operation

[0278] 1. Character Generation:

[0279] The server generates the character's appearance using image generation AI.

[0280] Next, a text generation AI is used to generate the character's ego and background story.

[0281] Finally, a voice generation AI is used to generate the character's voice.

[0282] 2. Create a scene:

[0283] The server automatically creates a scene in which a generated character uses a specific product, with the user providing a prompt such as:

[0284] Apparel: "Generate an image of a character wearing a new t-shirt."

[0285] Food: "Generate an image of a character eating a new snack."

[0286] Furniture Appliances: "Generate an image of a character sitting on a new sofa."

[0287] 3. Posting to social media:

[0288] The SNS posting module automatically posts the generated scenes to SNS, saving users the trouble of manually posting.

[0289] Hardware and software used

[0290] Hardware: The server uses a computer equipped with a high-performance GPU, which enables high-speed processing of generative AI models.

[0291] Software: OpenAI's DALL-E, GPT-4, and Google's WaveNet are used as generative AI models. The SNS posting module also uses the APIs of each SNS.

[0292] Specific examples

[0293] For example, if a clothing brand wants to advertise a new t-shirt, the user might enter the following prompt text:

[0294] Generate an image of the character wearing the new T-shirt.

[0295] Based on this prompt, the server uses image generation AI to generate a character image, text generation AI to create a background story for the character, and voice generation AI to generate a voice for the character. It then automatically creates a scene of the generated character wearing the new T-shirt and posts it to social media via the social media posting module.

[0296] In this way, it is possible to reduce filming costs, avoid the risk of scandals, customize characters, support a variety of media, and automatically post to social media.

[0297] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0298] Step 1:

[0299] The user uses the terminal to input a prompt, for example, "Please generate an image of a character wearing the new T-shirt." This prompt is then sent to the server.

[0300] Step 2:

[0301] The server analyzes the received prompt text and inputs it into the image generation AI. At this time, data processing is performed to generate the character's appearance based on the prompt text. Specifically, the character image is generated using OpenAI's DALL-E model. The input is the prompt text, and the output is an image of the generated character.

[0302] Step 3:

[0303] The server saves the generated character image and then inputs the prompt text into the text generation AI. During this process, data processing is performed to generate the character's ego and background story. Specifically, the character's background story is generated using OpenAI's GPT-4 model. The input is the prompt text, and the output is the generated background story.

[0304] Step 4:

[0305] The server saves the generated background story and then inputs the prompt text into the voice generation AI. At this time, data processing is performed to generate the character's voice. Specifically, the character's voice is generated using Google's WaveNet model. The input is the prompt text, and the output is the generated voice data.

[0306] Step 5:

[0307] The server integrates the generated character's image, background story, and voice data to automatically create a scene using a specific product. The scene is set based on the prompt entered by the user. The input is the generated character data, and the output is the scene data using a specific product.

[0308] Step 6:

[0309] The server inputs the generated scene data into the SNS posting module and automatically posts it to the SNS. The posting is done using the API of each SNS. The input is the generated scene data, and the output is the result of posting to the SNS.

[0310] Step 7:

[0311] Users can use their devices to check scenes posted to SNS. This saves users the trouble of manually posting. The input is the SNS posting result, and the output is the user's confirmation result.

[0312] Example 3

[0313] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0314] Conventional character generation systems lacked sufficient character customization and compatibility with diverse media, making it difficult to meet the diverse needs of users. Furthermore, it was difficult for users to reflect their specific requests when generating characters, and the generated characters often did not meet users' expectations. Furthermore, there were limitations on the character's operating hours and access regions, making global use difficult.

[0315] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0316] In this invention, the server includes a means for a user to input a prompt sentence, a means for receiving and analyzing the prompt sentence, a means for generating a character profile using a generative artificial intelligence model, a means for generating a voice for the character using speech synthesis software, and a means for providing the generated character to the user, thereby enabling character customization, global compatibility, continuous operation, and compatibility with a variety of media.

[0317] "Image generation artificial intelligence" is an artificial intelligence technology that generates images based on input data.

[0318] "Text generation artificial intelligence" is an artificial intelligence technology that generates text based on input data.

[0319] "Speech generation artificial intelligence" is an artificial intelligence technology that generates speech based on input data.

[0320] A "prompt sentence" is a sentence that describes instructions or requests that a user inputs to a system.

[0321] A "server" is a computer system that processes data and provides services over a network.

[0322] "Means of analysis" refers to the techniques and methods used to understand the input data and extract the necessary information.

[0323] A "generative artificial intelligence model" is a model for generating data using artificial intelligence technology.

[0324] "Speech synthesis software" is software for generating speech based on text data.

[0325] A "character profile" is information that describes a character's detailed settings and characteristics.

[0326] "Customizability" refers to the ability to change and adjust systems and characters according to the user's needs.

[0327] "Global compatibility" refers to the ability to access and use the system from anywhere in the world.

[0328] "Continuous operation" refers to the ability of a system to operate continuously 24 hours a day, 365 days a year.

[0329] "Multimedia compatibility" refers to the ability of a system or character to operate across multiple media, such as television, radio, and the Internet.

[0330] This invention is a system in which a user inputs a prompt sentence, and a server analyzes the prompt sentence and generates a character. A specific embodiment of this system will be described below.

[0331] First, the user inputs a prompt through the system interface, which includes detailed requests for the character's appearance, personality, voice, etc. For example, the following prompt can be input:

[0332] "Please create a character to appear at the online event. The character should be a woman in her 20s with a bright and lively personality, long blonde hair, and blue eyes. Please use a high-pitched, friendly voice."

[0333] Next, the server receives the prompt text sent by the user. The received prompt text is passed to an analysis module within the server. The server uses natural language processing technology to analyze the prompt text and extract each element of the character (appearance, personality, voice, etc.). Specifically, it uses a natural language processing library (e.g., spaCy).

[0334] Based on the analysis results, the server uses a generative AI model (e.g., OpenAI GPT-4) to generate a text-based profile for the character, which includes detailed settings for the character. The generated profile details the character's appearance, personality, background story, etc.

[0335] The server then uses speech synthesis software (e.g., Google Text-to-Speech) to generate a character voice based on the generated text-based profile, reflecting the tone and characteristics specified in the prompt.

[0336] Finally, the server provides the generated character's profile and voice to the user, who can then download the character and use it in online events and other media. This system enables character customization, global compatibility, continuous operation, and compatibility with a variety of media.

[0337] It is recommended to use a server equipped with a high-performance GPU (e.g., NVIDIA Tesla V100) to implement this system, as this will enable fast and efficient processing of the generative AI model and speech synthesis software.

[0338] As described above, the present invention provides a specific means for generating a customizable character in response to various user needs. The flow of the specification process in the third embodiment will be described with reference to FIG.

[0339] Step 1:

[0340] The user enters a prompt statement

[0341] The user inputs a prompt statement describing the character's requirements through the system interface. For example, the user can input a prompt statement such as, "Please create a character to appear in an online event. The character should be a woman in her twenties with a bright and lively personality, long blonde hair, and blue eyes. The voice should be high-pitched and friendly." The input data is text data containing detailed requirements such as the character's appearance, personality, and voice.

[0342] Step 2:

[0343] The server receives the prompt.

[0344] The server receives the prompt text sent by the user as an HTTP request. The received prompt text is passed to the analysis module in the server. The input data is the prompt text entered by the user, and the output data is the text data passed to the analysis module.

[0345] Step 3:

[0346] The server parses the prompt.

[0347] The server analyzes the prompt using natural language processing technology. Specifically, it uses a natural language processing library (e.g., spaCy) to understand the content of the prompt and extract each element of the character (appearance, personality, voice, etc.). The input data is the text data of the prompt, and the output data is data containing each element of the character as the analysis result.

[0348] Step 4:

[0349] The server generates a character profile using a generative AI model.

[0350] The server uses a generative AI model (e.g., OpenAI GPT-4) based on the analysis results to generate a text-based profile for the character. The generated profile contains detailed settings for the character. The input data is data containing each element of the character as a result of the analysis, and the output data is the generated character profile.

[0351] Step 5:

[0352] The server generates the character's voice using speech synthesis software.

[0353] The server generates a character voice using speech synthesis software (e.g., Google Text-to-Speech) based on the generated text-based profile. The generated voice reflects the tone and characteristics specified in the prompt. The input data is the generated character profile, and the output data is the generated character voice.

[0354] Step 6:

[0355] The server provides the generated character to the user.

[0356] The server provides the generated character's profile and voice to the user. The user can download the provided character and use it in online events or other media. The input data is the generated character's profile and voice, and the output data is the character data provided to the user.

[0357] (Application example 3)

[0358] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0359] Conventional character generation systems have limited user customization and interactive dialogue capabilities, and are not yet fully compatible with global platforms, 24 / 7 operation, or diverse media. Furthermore, there is a lack of systems that allow users to interact with characters in real time and advance the story. This limits the ability to improve the user experience and apply them to a variety of purposes.

[0360] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0361] In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a text generation AI, a voice generation means using a voice generation AI, an interactive dialogue system means, a multilingual support means, a server infrastructure means that can operate 24 hours a day, a means for a user-customized character to tell a unique story, and a means for updating the story in real time according to user selections. As a result, the user-customized character can interactively tell a story and be updated in real time, improving the user experience and enabling applications in a variety of uses.

[0362] "Image generation AI" is an artificial intelligence technology that generates a character's appearance based on user input.

[0363] "Text generation AI" is an artificial intelligence technology for generating character egos and stories.

[0364] "Voice generation AI" is an artificial intelligence technology for generating character voices.

[0365] An "interactive dialogue system" is a system that allows users and characters to interact in real time.

[0366] "Multilingual support" is a function that supports multiple languages ​​and translates what characters say according to the user's language.

[0367] "24-hour server infrastructure" refers to server infrastructure that can be kept running at all times.

[0368] "A means for user-customized characters to tell their own stories" refers to a means for users to create and tell their own stories with characters they have created.

[0369] "Means for updating a story in real time according to user selection" means means for changing and updating a story in real time based on user selection.

[0370] The following system configuration and processing procedure will be described as an embodiment of the present invention.

[0371] System Configuration

[0372] This system uses the following hardware and software:

[0373] Hardware

[0374] Smartphone (iOS, ANDROID (registered trademark))

[0375] Head-mounted displays (Oculus Quest, HTC Vive, etc.)

[0376] Server infrastructure (AWS (registered trademark), etc.)

[0377] software

[0378] Unity (game engine)

[0379] TENSORFLOW (registered trademark) (generative AI model)

[0380] Google Translate API (multilingual support)

[0381] Processing steps

[0382] 1. Character Creation and Customization

[0383] The user launches the application using a smartphone or head-mounted display.

[0384] Users customize their character's appearance, personality, and voice within the application.

[0385] Unity is used to generate characters in real time, and TensorFlow is used to generate the character's voice.

[0386] 2. Story generation and management

[0387] It uses a generative AI model (such as GPT-4) to generate a story based on a theme selected by the user.

[0388] Stories are managed on a server and updated in real time based on user choices.

[0389] 3. Interactive dialogue systems

[0390] When a user interacts with the character, it uses voice recognition technology to analyze the user's input and generate an appropriate response.

[0391] The content of the dialogue is reflected in the story in real time.

[0392] 4. Multilingual support

[0393] Uses Google Translate API to translate what the characters say based on the user's language.

[0394] 5. 24 / 7 server infrastructure

[0395] Using AWS, we will build a server infrastructure that operates 24 hours a day and allows users around the world to access our services.

[0396] Specific examples

[0397] The user launches the "Virtual Character Storyteller" and customizes a character set in a fantasy world. The character's name is "Elise the Elf," its personality is "brave and intelligent," and its voice is "gentle, female." When the user says, "Elise, tell me a story about your adventure," Elise begins telling the story of a new adventure using a generative AI model. When the user asks, "What happens next?", Elise continues the story, and the story progresses according to the user's choices.

[0398] Prompt Sentence Examples

[0399] The character "Elise the Elf" is brave, intelligent, and speaks with a gentle, female voice. Generate a scene in which Elise narrates a fantasy adventure story. Include a response when the user asks, "What happens next?"

[0400] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0401] Step 1:

[0402] A user starts an application using a smartphone or a head-mounted display.

[0403] Input: User device operations

[0404] Output: Application launch screen

[0405] Specific behavior: The user taps the application icon and the application launches.

[0406] Step 2:

[0407] Users customize the character's appearance, personality, and voice within the application.

[0408] Input: User customization settings (appearance, personality, voice)

[0409] Output: Customized character data

[0410] How it works: The user sets each of the character's attributes using sliders and drop-down menus, and the settings are updated in real time.

[0411] Step 3:

[0412] The server uses Unity to generate characters in real time and TensorFlow to generate the character's voice.

[0413] Input: Customized character data

[0414] Output: 3D model and voice data of the generated character

[0415] How it works: The server launches the Unity engine and generates a 3D model of the character based on the user's settings, while simultaneously generating the character's voice using TensorFlow.

[0416] Step 4:

[0417] It uses a generative AI model (such as GPT-4) to generate a story based on a theme selected by the user.

[0418] Input: Theme selected by the user

[0419] Output: Text data of the generated story

[0420] Specific operation: The server inputs the theme into the generative AI model and generates a prompt sentence. A story is generated based on the generated prompt sentence.

[0421] Step 5:

[0422] The server manages the stories and updates them in real time according to the user's selections.

[0423] Input: Text data of the generated story, user selection

[0424] Output: Updated story text data

[0425] What happens: The server receives the user's selections and updates the story progress in real time. The updated story is displayed to the user.

[0426] Step 6:

[0427] As the user interacts with the character, voice recognition technology is used to analyze the user's input and generate appropriate responses.

[0428] Input: User voice input

[0429] Output: Character voice response

[0430] How it works: When a user speaks into the microphone, speech recognition technology converts what they say into text, and a generative AI model generates an appropriate response, which is then played back as the character's voice.

[0431] Step 7:

[0432] Uses the Google Translate API to translate what the characters say based on the user's language.

[0433] Input: Character's spoken text data, user's language settings

[0434] Output: Translated speech text data

[0435] Specific operation: The server sends the character's speech to the Google Translate API, receives the text data translated into the user's language, and plays the translated speech as the character's voice.

[0436] Step 8:

[0437] Using AWS, we will build a server infrastructure that operates 24 hours a day and allows users around the world to access it.

[0438] Input: Server infrastructure configuration data

[0439] Output: 24-hour server infrastructure

[0440] How it works: A server administrator uses AWS to set up server infrastructure and create a 24 / 7 environment, allowing users around the world to access the service at any time.

[0441] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0442] "Example 1"

[0443] One embodiment of the present invention is an AI character generation system incorporating an emotion engine. This system is capable of recognizing a user's emotions and adjusting the character's behavior and speech accordingly. Specifically, if the user expresses joy, the character will also express joy. Similarly, if the user expresses sadness, the character will act in accordance with the user's emotions, such as offering words of comfort.

[0444] "Example 2"

[0445] The emotion engine can also detect changes in the user's emotions in real time and instantly adjust the character's behavior and speech. For example, if the user suddenly expresses anger, the character will immediately apologize and respond appropriately to the situation.

[0446] "Example 3"

[0447] Furthermore, the emotion engine can learn the user's emotional patterns and predict changes in the user's emotions. This allows the character to act in advance of the user's emotional changes, enabling more natural interactions. For example, if the user feels tired from daily work, the character can predict the user's emotions by playing relaxing music when the user returns home.

[0448] The processing flow of each embodiment will be described below.

[0449] "Example 1"

[0450] Step 1: The emotion engine recognizes the user's emotion.

[0451] Step 2: Adjust the AI ​​character's behavior and speech based on the emotions recognized by the emotion engine.

[0452] Step 3: Present the adjusted actions and statements to the user.

[0453] "Example 2"

[0454] Step 1: The emotion engine captures changes in the user's emotions in real time.

[0455] Step 2: The emotion engine instantly adjusts the AI ​​character's behavior and speech based on the changes in emotions detected.

[0456] Step 3: Present the adjusted actions and statements to the user.

[0457] "Example 3"

[0458] Step 1: The emotion engine learns the user's emotional patterns.

[0459] Step 2: Based on the learned emotional patterns, the emotion engine predicts changes in the user's emotions.

[0460] Step 3: Adjust the AI ​​character's behavior and speech based on the predicted emotional changes.

[0461] Step 4: Present the adjusted actions and statements to the user.

[0462] Example 1

[0463] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0464] Conventional character generation systems can individually generate a character's appearance, personality, and voice, but lack the ability to adjust the character's behavior and speech according to the user's emotions. This results in inconsistent interaction with the user and makes it difficult to respond to emotions in real time. Furthermore, in an age where diverse media such as advertising and online events are required, the lack of character customization and global compatibility is also a problem.

[0465] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a sentence generation AI, a voice generation means using a voice generation AI, a user emotion recognition means using an emotion engine, and a means for adjusting the character's actions and remarks according to the user's emotions. This makes it possible to generate a character that can respond to the user's emotions in real time and has consistent character settings suitable for use on SNS. In addition, the character's customizability, global compatibility, and compatibility with various media are improved, making it easier to use in advertisements, online events, etc.

[0466] "Image generation artificial intelligence" is an artificial intelligence technology that generates images based on input text or prompts.

[0467] "Appearance generation means" refers to techniques or methods for generating a character's appearance or shape.

[0468] "Sentence generation artificial intelligence" is an artificial intelligence technology that generates sentences based on input text and prompts.

[0469] "Self-generation means" refers to techniques and methods for generating a character's personality and thoughts.

[0470] "Speech generation artificial intelligence" is an artificial intelligence technology that generates speech based on input text and prompts.

[0471] "Voice generation means" refers to the techniques and methods used to generate the voice of a character.

[0472] An "emotion engine" is a technology that recognizes emotions from user input and behavior, and adjusts responses and behavior according to those emotions.

[0473] "User emotion recognition means" refers to a technique or method for analyzing and recognizing a user's emotions.

[0474] "Means for adjusting behavior and speech" refers to techniques and methods for changing a character's behavior and speech based on the user's recognized emotions.

[0475] An "SNS-operable character" is a character that can be active on a social networking service and interact with users.

[0476] "Customizability" refers to the ability to change a character's appearance, personality, voice, etc. according to the user's needs and purpose.

[0477] "Global compatibility" refers to the ability to accommodate different languages ​​and cultures and interact with users around the world.

[0478] "Media-versatile" refers to the ability of a character to operate in a variety of media formats (e.g., images, video, audio).

[0479] "Online event appearance" is a feature that allows a character to participate in events held over the Internet and interact with users in real time.

[0480] "Unique story and worldview" refers to the character's unique background setting and narrative, as well as the virtual environment and setting in which the character exists.

[0481] MODE FOR CARRYING OUT THE INVENTION

[0482] This invention is a system for generating characters capable of managing social networking sites using image generation AI, text generation AI, voice generation AI, and an emotion engine. This system can adjust the character's behavior and comments according to the user's emotions.

[0483] 1. System Overview

[0484] The system includes the following main components:

[0485] Image generation artificial intelligence (e.g., DALL-E, Stable Diffusion)

[0486] Text generation artificial intelligence (e.g. GPT-4)

[0487] Speech generation artificial intelligence (e.g., Tacotron 2, WaveNet)

[0488] Emotion Engine

[0489] 2. Hardware and Software Used

[0490] The server performs its processing using the following hardware and software:

[0491] A server equipped with a high-performance GPU (e.g., NVIDIA GPU)

[0492] Image generation AI models (e.g., DALL-E, Stable Diffusion)

[0493] Sentence generation AI model (e.g. GPT-4)

[0494] Speech generation AI models (e.g., Tacotron 2, WaveNet)

[0495] Emotion Recognition Software

[0496] The terminal is a device that the user uses to request character generation, and includes a smartphone, a PC, etc.

[0497] 3. Program Processing Flow

[0498] The user uses a device to submit a character creation request, for example, by entering "a female character in her 20s with a bright smile" into a form on a dedicated app or website, and then pressing the submit button.

[0499] The server performs the following actions based on the request:

[0500] 1. Image generation:

[0501] The server calls an image generation AI model and generates the character's appearance.

[0502] Example: Send the prompt "A female character in her 20s with a bright smile" to an image generation AI and receive the generated image.

[0503] 2. Ego generation:

[0504] The server calls a sentence generation AI model to generate the character's personality and thoughts.

[0505] Example: Send the prompt "A cheerful and sociable person whose hobbies are reading and traveling" to a text generation AI and receive the generated text.

[0506] 3. Voice generation:

[0507] The server calls a voice generation AI model to generate the character's voice.

[0508] Example: Send a prompt such as "a cheerful voice of a woman in her 20s" to a voice generation AI and receive the generated voice data.

[0509] 4. Emotion recognition:

[0510] The server uses an emotion engine to recognize emotions from the user's input (text and / or voice).

[0511] Example: If a user types "I'm so happy today!", the server uses an emotion engine to recognize "joy."

[0512] 5. Behavioral and speech adjustment:

[0513] The server adjusts the character's actions and speech based on the user's recognized emotions.

[0514] Example: If the user types "I'm so happy today!", the server has a character reply "That's great! I'm happy too!"

[0515] In this way, the server can generate a character with consistent character settings that can be used in an SNS and provide interactions that correspond to the user's emotions.

[0516] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0517] Step 1:

[0518] The user uses a device to send a request for character generation. Specifically, they enter "a female character in her 20s with a bright smile" into a form on a dedicated app or website, and then press the send button. The input data is the character's basic attribute information.

[0519] Step 2:

[0520] The server receives a request from a user and calls the image generation AI model. The input data is the character's attribute information sent by the user. Based on this information, the server sends a prompt to the image generation AI model and generates the character's appearance. The output data is an image of the generated character. Specifically, the server sends a prompt to the image generation AI saying, "A female character in her twenties with a bright smile," and receives the generated image.

[0521] Step 3:

[0522] The server saves an image of the generated character and then calls the sentence generation AI model. The input data is information about the character's personality and thoughts sent by the user. Based on this information, the server sends a prompt to the sentence generation AI model to generate the character's ego. The output data is text about the generated character's personality and thoughts. Specifically, the server sends a prompt to the sentence generation AI saying, "He has a cheerful and sociable personality, and his hobbies are reading and traveling," and receives the generated text.

[0523] Step 4:

[0524] The server saves the self information of the generated character and then calls the voice generation AI model. The input data is information about the character's voice sent by the user. Based on this information, the server sends a prompt to the voice generation AI model to generate the character's voice. The output data is the voice data of the generated character. Specifically, the server sends a prompt, "the cheerful voice of a woman in her twenties," to the voice generation AI and receives the generated voice data.

[0525] Step 5:

[0526] The server saves the voice data of the generated character and then calls the emotion engine. The input data is the user's text input or voice input. Based on this information, the server uses the emotion engine to recognize the user's emotion. The output data is the recognized user's emotion information. In concrete terms, if the user inputs "I'm so happy today!", the server will recognize "joy" using the emotion engine.

[0527] Step 6:

[0528] The server adjusts the character's behavior and speech based on the recognized user's emotional information. The input data is the user's emotional information. The server generates the character's behavior and speech based on this information. The output data is the adjusted character's behavior and speech. In concrete terms, if the user inputs "I'm so happy today!", the server will have the character reply "That's great! I'm happy too!"

[0529] In this way, the server can generate a character with consistent character settings that can be used in an SNS and provide interactions that correspond to the user's emotions.

[0530] (Application example 1)

[0531] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0532] Conventional SNS management characters were unable to recognize emotions in conversations with users and act or speak accordingly. This limited the user experience and lacked emotionally sensitive interactions. It was also difficult to personalize advertisements and content delivery based on user emotions. This led to problems such as a decline in user satisfaction and engagement.

[0533] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0534] In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a sentence generation AI, a voice generation means using a voice generation AI, an emotion recognition and behavior adjustment means using an emotion engine, and a means for adjusting the character's behavior and speech according to the user's emotions. This enables interactions according to the user's emotions, providing a more personalized experience.

[0535] "Image generation AI" is an artificial intelligence technology for generating character appearances.

[0536] "Text generation AI" is an artificial intelligence technology for generating a character's ego, that is, their personality and thoughts.

[0537] "Voice generation AI" is an artificial intelligence technology for generating character voices.

[0538] An "emotion engine" is a technology that recognizes the user's emotions and adjusts the character's actions and speech in response to those emotions.

[0539] "Emotion recognition and behavior adjustment means" is a function for detecting the user's emotions and changing the character's behavior and speech based on those emotions.

[0540] The "means for adjusting the character's actions and speech in accordance with the user's emotions" is a function for dynamically changing the character's responses and actions in accordance with the user's emotional state.

[0541] The following system configuration will be described as an embodiment of the present invention.

[0542] First, the server includes an image generation AI, a text generation AI, a voice generation AI, and an emotion engine. The image generation AI is used to generate the character's appearance. Specifically, the image generation AI generates the character's appearance based on the character's characteristics input by the user. The text generation AI is used to generate the character's ego, that is, its personality and thoughts. This allows the character to have a consistent personality. The voice generation AI is used to generate the character's voice. This allows the character to interact with the user in a natural voice.

[0543] The emotion engine is used to recognize the user's emotions and adjust the character's behavior and speech accordingly. Specifically, it analyzes emotions from the text or voice entered by the user and changes the character's response based on the results. For example, if the user enters "happy," the character will respond with "I'm so glad to see you happy!"

[0544] This system runs on devices such as smartphones and PCs. The device receives input from the user and sends it to a server. The server generates the character's appearance, personality, and voice based on the received input, and adjusts the character's responses using an emotion engine. The generated character's image and voice are sent to the device and displayed to the user.

[0545] As a concrete example, consider a scenario in which a user launches an application using a smartphone and interacts with a character. If the user inputs "I'm so happy today," the emotion engine recognizes the user's emotion as "happy." Based on this, the character responds with "I'm so glad to see you happy!", which is displayed along with the generated voice.

[0546] An example of a prompt is as follows:

[0547] Describe a friendly and supportive personality.

[0548] By inputting this prompt into a generative AI model, the character's personality and thoughts are generated.

[0549] The above is an embodiment of the present invention.

[0550] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0551] Step 1:

[0552] A user launches the application using a device such as a smartphone or PC. The user provides input (text or voice) to the device to interact with the character. Input can include text such as "I'm very happy today."

[0553] Step 2:

[0554] The terminal sends input from the user to the server. The input data is sent to the server in text format. The server analyzes the received input data and uses an emotion engine to recognize the user's emotion. The emotion engine analyzes emotions from the input text and outputs an emotion label such as "happy."

[0555] Step 3:

[0556] The server adjusts the character's behavior and speech based on the output of the emotion engine. Specifically, it inputs prompt sentences corresponding to emotion labels into a generative AI model to generate the character's responses. For example, the prompt sentence "Describe a friendly and supportive personality" is used to generate the character's personality and thoughts.

[0557] Step 4:

[0558] The server inputs the generated character's responses into the voice generation AI to generate the character's voice. The voice generation AI converts the text responses into audio data, allowing the character to respond in a natural voice.

[0559] Step 5:

[0560] The server inputs the generated character's appearance into the image generation AI to generate an image of the character. The image generation AI generates image data based on the character's characteristics and determines the character's appearance.

[0561] Step 6:

[0562] The server sends the generated character's image, voice, and response text to the terminal, which displays the received data to the user, and the character responds in a natural voice.

[0563] Step 7:

[0564] The user can continue to interact with the character on the device. Steps 1 to 6 are repeated depending on the user's new input, allowing the user to have a personalized experience based on their emotions.

[0565] Example 2

[0566] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0567] Traditional advertising methods have high filming costs and the risk of scandals. They also have the problem of being difficult to respond to users' emotions immediately, limiting real-time interaction. Furthermore, the lack of character customization and compatibility with various media limits the effectiveness of advertising.

[0568] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0569] In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a text generation AI, a voice generation means using a voice generation AI, a user emotion detection means using an emotion engine, a means for adjusting the character's actions and remarks in real time, and a means for posting the generated character to a social media platform. This reduces filming costs, avoids the risk of scandals, and enables immediate response to user emotions. It also improves character customization and compatibility with various media, maximizing advertising effectiveness.

[0570] "Image generation AI" is a technology that uses artificial intelligence to generate images.

[0571] "Appearance generation means" is a means of generating a character's appearance using image generation AI.

[0572] "Text generation AI" is a technology that uses artificial intelligence to generate text.

[0573] "Self-generation means" is a means of generating a character's personality and speech using text generation AI.

[0574] "Voice generation AI" is a technology that uses artificial intelligence to generate voice.

[0575] "Voice generation means" refers to a means of generating a character's voice using voice generation AI.

[0576] An "emotion engine" is a technology that detects and analyzes a user's emotions.

[0577] The "user emotion detection means" is a means for detecting the user's emotion in real time using an emotion engine.

[0578] "Means for adjusting the character's actions and statements in real time" refers to means for instantly changing the character's actions and statements in response to the user's emotions.

[0579] "SNS Platform" means an online platform that provides social networking services.

[0580] "Means for posting the generated character on a social media platform" refers to means for posting images and videos of the generated character on a social media platform.

[0581] MODE FOR CARRYING OUT THE INVENTION

[0582] This invention is a system that generates characters using a generative AI model and performs advertising and user support on a social networking platform. Specific embodiments of this system are described below.

[0583] Character Generation

[0584] The server generates characters using generative AI models (e.g., image generation AI, text generation AI, and voice generation AI). For image generation AI, it uses DALL-E, for example, and for text generation AI, it uses GPT-4. For voice generation AI, it uses Google Cloud Text-to-Speech, for example. By entering the prompt "Generate a character wearing the new jacket," the character's appearance, personality, and voice are generated.

[0585] Character action settings

[0586] The server assigns specific actions to the generated character. For example, it creates a scene in which the character is walking around town wearing a new jacket. It then uses image processing software (e.g., Adobe Photoshop) to adjust the character's pose and the background. Specifically, it creates a scene in which the character is walking down the sidewalk and looking at a smartphone.

[0587] Posting to social media

[0588] The server posts the generated images and videos to a social media platform (e.g., Instagram). The post includes a caption such as "New jacket is here! Check it out now!" Specifically, the user uploads the image to the Instagram posting screen, enters a caption, and clicks the "Post" button.

[0589] User Emotion Detection

[0590] The device uses an emotion engine (e.g., Microsoft® Azure® Emotion API) to detect the user's emotions in real time. The emotion engine captures the user's facial expressions and voice using a camera and microphone and analyzes them. Specifically, it detects the moment when the user frowns or their voice tone rises.

[0591] Character reaction adjustments

[0592] When a user expresses anger, the device notifies the server. The server then instantly adjusts the character's actions and speech, for example, making the character say, "Sorry, did I say something that offend you?" Specifically, it generates an audio file for the character and sends it to the user's device.

[0593] Examples of concrete examples and prompts

[0594] Example 1: Advertising apparel products

[0595] A scene is generated in which a character is wearing a new jacket and walking around town.

[0596] Post the generated image to Instagram with the caption, "New jacket here! Check it out now!"

[0597] Example 2: Responding to user emotions

[0598] If the user suddenly expresses anger, the emotion engine detects this and notifies the server.

[0599] The server has the character say, "Sorry, did I say something to offend you?"

[0600] Prompt Sentence Examples

[0601] "Generate an image of a character wearing the jacket of a new work."

[0602] "Generate a scenario where a character apologizes when the user expresses anger."

[0603] In this way, the server, terminal, and user work together to create a system that uses the generated characters to advertise and respond to users.

[0604] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0605] Step 1:

[0606] The server generates a character using a generative AI model. The prompt sentence "Generate a character wearing the new jacket" is used as input. The appearance, personality, and voice of the character are generated by the generative AI model (e.g., image generation AI, text generation AI, voice generation AI). The output is an image, text, and voice data of the generated character. Specifically, the server inputs the prompt sentence into the generative AI model and saves the generated data.

[0607] Step 2:

[0608] The server assigns specific actions to the generated character. As input, it uses the data of the generated character and instructions for the action (e.g., "scene of walking through town"). It uses image processing software (e.g., Adobe Photoshop) to adjust the character's pose and background. As output, it obtains images and videos of the character with the assigned action. Specifically, the server inputs the character's data into the image processing software and generates data with the assigned action.

[0609] Step 3:

[0610] The server posts the generated images and videos to the SNS platform. As input, it uses the image or video of the character with the action set and the post content (e.g., "New jacket is here! Check it out now!"). It posts using the SNS platform's API. As output, it obtains the content posted to the SNS platform. Specifically, the server inputs the image or video and post content into the SNS platform's API and executes the post.

[0611] Step 4:

[0612] The device detects the user's emotions in real time. It uses the user's facial expressions and voice data as input. It analyzes the user's emotions using an emotion engine (e.g., Microsoft Azure's Emotion API). The output is the user's emotional state. Specifically, the device captures the user's facial expressions and voice using a camera and microphone, inputs them into the emotion engine, and obtains the analysis results.

[0613] Step 5:

[0614] When the user expresses anger, the device notifies the server. The user's emotional state is used as input. The server immediately adjusts the character's actions and speech. The adjusted character actions and speech are obtained as output. Specifically, the server receives the user's emotional state, generates an audio file that makes the character say, "I'm sorry, did I say something that offend you?" and sends it to the user's device.

[0615] (Application example 2)

[0616] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0617] Conventional advertising systems have high filming costs and the risk of scandals. They also face the challenge of not being able to provide personalized ads that reflect users' emotions, making effective marketing impossible. Furthermore, while character-based advertising requires responses that reflect users' emotions in real time, there is a lack of technology to achieve this.

[0618] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0619] In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a sentence generation AI, a voice generation means using a voice generation AI, an emotion recognition means, an emotion-based advertisement content generation means, a voice output means, a camera image acquisition means, a face detection means, a face landmark detection means, an emotion prediction means, and an emotion-based character behavior determination means, thereby enabling advertisements to be provided in real time according to the user's emotions.

[0620] "Image generation AI" is a method of generating images using artificial intelligence technology.

[0621] "Appearance generation means" is a means for generating the appearance and shape of a character.

[0622] "Text generation AI" is a method of generating text using artificial intelligence technology.

[0623] "Self-generation means" is a means for generating a character's individuality and personality.

[0624] "Voice generation AI" is a method of generating voice using artificial intelligence technology.

[0625] "Voice generation means" is a means for generating the voice of a character.

[0626] The "emotion recognition means" is a means for recognizing the user's emotions.

[0627] The "means for generating advertisement content according to emotions" is a means for generating advertisement content based on the emotions of a user.

[0628] The "audio output means" is a means for outputting the generated audio.

[0629] The "camera image acquisition means" is a means for acquiring an image using a camera.

[0630] The "face detection means" is a means for detecting a face from the acquired image.

[0631] The "face landmark detection means" is a means for detecting feature points of a detected face.

[0632] The "emotion prediction means" is a means for predicting the user's emotion based on the detected facial feature points.

[0633] The "means for determining character behavior based on emotions" is a means for determining the behavior of a character based on predicted emotions.

[0634] The system for implementing this invention includes an image generation AI, a text generation AI, a voice generation AI, an emotion recognition means, an emotion-based advertising content generation means, a voice output means, a camera image acquisition means, a face detection means, a face landmark detection means, an emotion prediction means, and an emotion-based character behavior determination means.

[0635] System configuration

[0636] The server uses image generation AI to generate the character's appearance, text generation AI to generate the character's ego, and voice generation AI to generate the character's voice, allowing the character to converse naturally with the user.

[0637] The terminal acquires an image of the user using the camera image acquisition means, detects a face from the image using the face detection means, detects facial feature points using the face landmark detection means, and predicts the user's emotion using the emotion prediction means.

[0638] The server generates advertisement content using emotion-based advertisement content generation means based on the predicted emotion, and determines the character's behavior using emotion-based character behavior determination means, thereby enabling the character to provide advertisements that correspond to the user's emotions in real time.

[0639] Hardware and software used

[0640] Hardware: Smartphone camera, smartphone speaker

[0641] Software: OpenCV (camera image acquisition and face detection), dlib (facial landmark detection), Keras (emotion recognition model loading and prediction), text_to_speech (converting character speech into audio)

[0642] Data processing and calculation

[0643] 1. Camera image acquisition: Capture the user's image in real time using the device's camera.

[0644] 2. Face detection: Detect faces from captured video using OpenCV and dlib and identify facial landmarks.

[0645] 3. Emotion prediction: The detected facial landmark data is input into an emotion recognition model using Keras to predict the user's emotion.

[0646] 4. Advertising content generation: Based on the predicted emotions, advertising content is generated using an emotion-based advertising content generation means.

[0647] 5. Character behavior determination: Determine the character's behavior using emotion-based character behavior determination means.

[0648] 6. Audio output: Using the text_to_speech library, the character's speech is converted into audio and played through the device's speaker.

[0649] Specific examples

[0650] When the user opens their smartphone, the camera activates and a character cheerfully speaks to them, saying, "Hello! It's a wonderful day today. How do you like our new apparel?"

[0651] If the user looks sad, the character will gently say, "Cheer up! These delicious sweets are sure to cheer you up."

[0652] Prompt Sentence Examples

[0653] Generate what the character will say when the user is happy.

[0654] Generate what the character will say if the user is sad.

[0655] Generate what the character will say if the user is angry.

[0656] In this way, the system can deliver ads in real time that respond to the user's emotions, creating a more personalized experience.

[0657] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0658] Step 1:

[0659] The terminal uses a camera image capture means to capture the user's image in real time, where the input is the camera image and the output is the captured image data, which is used in subsequent processing steps.

[0660] Step 2:

[0661] The device detects faces from captured video data using a face detection mechanism. The input is the captured video data, and the output is the position information of the detected face. The device uses OpenCV to identify the position of the face and extract the face area.

[0662] Step 3:

[0663] The device identifies the detected facial feature points using a facial landmark detection means. The input is face position information, and the output is facial landmark data. 68 facial feature points are detected using dlib, and this data is acquired.

[0664] Step 4:

[0665] The device uses emotion prediction means to predict the user's emotion based on facial landmark data. The input is facial landmark data, and the output is a predicted emotion label. The landmark data is input into an emotion recognition model using Keras to predict emotions.

[0666] Step 5:

[0667] The server generates advertising content based on the predicted emotion using an emotion-based advertising content generation means. The input is the predicted emotion label, and the output is the generated advertising content. The advertising content corresponding to the emotion label is created using a generative AI model.

[0668] Step 6:

[0669] The server determines the character's behavior using emotion-based character behavior determination means. The input is the generated advertisement content, and the output is instructions for the character's behavior. The server determines the character's remarks and actions based on the advertisement content.

[0670] Step 7:

[0671] The device uses a voice generation AI to convert the character's speech into audio and plays it back using an audio output means. The input is the character's action instructions, and the output is audio data. The text_to_speech library is used to convert the speech into audio and play it back through the smartphone speaker.

[0672] Step 8:

[0673] The user listens to the voice played on the device and takes action based on what the character says. The input is the voice data, and the output is the user's reaction. The user's reaction is captured again in the next cycle, forming a feedback loop for the entire system.

[0674] Example 3

[0675] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0676] Conventional character generation systems had difficulty responding to user emotions, and were unable to fully utilize features such as character customization, global compatibility, continuous operation, and compatibility with a variety of media. Furthermore, they lacked the ability to learn and predict user emotional patterns, so more natural dialogue and an improved user experience were needed.

[0677] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0678] In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a sentence generation AI, a voice generation means using a voice generation AI, a user emotion pattern learning means using an emotion engine, a constant character operation means using a cloud service, a character activity means in various media, and a means for predicting and responding to changes in the user's emotions. This enables character customization, global compatibility, constant operation, and compatibility with various media, and also enables natural dialogue and responses according to the user's emotions.

[0679] "Image generation artificial intelligence" is an artificial intelligence technology that generates images based on user input.

[0680] "Sentence generation artificial intelligence" is an artificial intelligence technology that generates sentences based on user input.

[0681] "Speech generation artificial intelligence" is an artificial intelligence technology that generates speech based on user input.

[0682] An "emotion engine" is a technology that learns a user's emotional patterns and predicts changes in emotions.

[0683] "Cloud services" is a general term for computing resources and services provided via the Internet.

[0684] "Means for keeping characters running at all times" refers to the technology and infrastructure required to keep characters running 24 hours a day.

[0685] "Means for character activity in various media" refers to the technology and interfaces that allow characters to be active in various media such as television, radio, and the Internet.

[0686] The "means for predicting and responding to changes in the user's emotions" is a technology for predicting changes in the user's emotions and responding accordingly.

[0687] The invention begins when a user submits a character generation request using a device. The user inputs details such as the character's appearance, personality, and voice. The server generates the character based on the user's request using a generative AI model. Specifically, it uses an image generation AI (e.g., DALL-E) to generate the character's image, and a text generation AI (e.g., GPT-4) to generate the character's personality and voice characteristics. The generated character data is stored on the server, and a confirmation link is sent to the user.

[0688] The user opens a confirmation link on their device to view the generated character. If desired, the user can further customize the character's details, such as changing hair color or brightening the character's personality. The server updates these changes in real time and provides the updated character to the user again.

[0689] The server uses cloud services to build infrastructure to run the generated characters 24 hours a day, ensuring that the characters are always online and accessible from any region. For example, AWS's EC2 instances are used to provide the character's operating environment.

[0690] Users input their daily emotional data through their devices. For example, they might enter "I'm tired today" into a dedicated application. The server sends this data to the emotion engine, which learns the user's emotional patterns. The emotion engine analyzes the user's input data and updates the model to predict changes in emotions.

[0691] The server predicts changes in the user's emotions based on the learned emotional data. For example, if the user inputs "I'm tired today," the emotion engine will determine that the user needs to relax. Based on this prediction, the character will respond by playing relaxing music when the user gets home. Specifically, the character will say, "Thank you for your hard work. I'll play some relaxing music for you," and then actually play the music.

[0692] Examples of concrete examples and prompts

[0693] Specific examples

[0694] When a user types "I'm tired today" into the device, the server uses an emotion engine to analyze the user's emotions. The emotion engine determines that the user needs to relax and issues instructions to the character. The character says, "Thank you for your hard work. I'll play some relaxing music for you," and uses the API of a music streaming service to play relaxing music.

[0695] Prompt Sentence Examples

[0696] "Generate a prompt that instructs a character to play relaxing music when the user gets home from work."

[0697] In this way, the system operates in cooperation with the server, the terminal, and the user. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0698] Step 1:

[0699] The user uses a terminal to send a request for character generation. The user inputs details such as the character's appearance, personality, and voice. The input data is sent to the server. Specifically, the user inputs hairstyle, eye color, clothing, personality, and voice characteristics into a form in a dedicated application, and then presses the send button.

[0700] Step 2:

[0701] The server receives requests sent by users and generates characters using a generative AI model. Specifically, it uses image generation AI to generate character images, and text generation AI to generate the character's personality and voice characteristics. The generated character data is stored on the server. For example, DALL-E is used to generate character images, and GPT-4 is used to generate the character's personality and voice characteristics.

[0702] Step 3:

[0703] The server generates a confirmation link and sends it to the user to provide the generated character data to the user. The user opens the confirmation link on their device and confirms the generated character. Specifically, the server saves the generated character data and sends the user an access link to that data by email.

[0704] Step 4:

[0705] The user can review the generated character through their device and further customize it as needed. The user can then change the character's details and send them back to the server. The server then updates these changes in real time and provides the updated character back to the user. For example, the user can change the hair color or make the character a little brighter.

[0706] Step 5:

[0707] The server uses a cloud service to build infrastructure to run the generated characters 24 hours a day. This allows the characters to be always online and accessible from any region. Specifically, AWS EC2 instances are used to provide the character's operating environment.

[0708] Step 6:

[0709] Users input their daily emotional data through their devices. For example, they might enter "I'm tired today" into a dedicated application. The input emotional data is sent to the server. The server then sends this data to the emotion engine, which learns the user's emotional patterns. The emotion engine analyzes the user's input data and updates the model to predict changes in emotions.

[0710] Step 7:

[0711] The server predicts changes in the user's emotions based on the learned emotional data. For example, if a user inputs "I'm tired today," the emotion engine determines that the user needs to relax. Based on this prediction, the character responds by playing relaxing music when the user returns home. Specifically, the character says, "Thank you for your hard work. I'll play some relaxing music for you," and uses the API of a music streaming service to play relaxing music.

[0712] (Application example 3)

[0713] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0714] Conventional character generation systems lack the ability to predict user emotions and provide appropriate content, which leaves the user experience unsatisfied. Furthermore, there is a need for systems that offer features such as character customization, global compatibility, continuous operation, and compatibility with various media.

[0715] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a sentence generation AI, a voice generation means using a voice generation AI, an emotion prediction means, a content recommendation means, and a means for interacting with the user. This makes it possible to predict the user's emotions and provide appropriate content, improving the user experience. Furthermore, it is possible to realize a system with characteristics such as character customization, global compatibility, continuous operation, and compatibility with various media.

[0716] "Image generation AI" is a method of automatically generating a character's appearance using artificial intelligence technology.

[0717] "Text generation AI" is a method of automatically generating text that expresses a character's ego and personality using artificial intelligence technology.

[0718] "Voice generation AI" is a method of automatically generating character voices using artificial intelligence technology.

[0719] "Emotion prediction means" is a technology for predicting emotions from user input and behavior.

[0720] A "content recommendation means" is a technology for recommending appropriate content based on predicted user emotions.

[0721] "Means for interacting with the user" refers to technology that allows the character to have natural conversations with the user.

[0722] "Customizability" refers to the ability to change a character's appearance, personality, voice, etc. according to the user's wishes.

[0723] "Global compatibility" refers to the ability to access a website from anywhere in the world.

[0724] "Continuous operation" means that the system is operational 24 hours a day.

[0725] "Diverse media compatibility" refers to the ability to operate across multiple media, including television, radio, and the Internet.

[0726] "Online event appearance" is a feature that allows a character to participate in events held over the Internet.

[0727] "Unique story and worldview" refers to the unique background, story, and setting that each character has.

[0728] The system for implementing this invention includes an image generation AI, a text generation AI, a voice generation AI, an emotion prediction means, a content recommendation means, and a means for interacting with a user. Specific embodiments of each means will be described below.

[0729] 1. Appearance generation method using image generation AI

[0730] The server uses image generation AI to generate the character's appearance. This image generation AI can customize the character's appearance according to the user's request. For example, if the user requests an "anime-style character," the image generation AI will generate a character that meets that request.

[0731] 2. Self-generation using text generation AI

[0732] The server uses a text generation AI to generate sentences that express the character's ego and personality. This text generation AI can customize the character's personality and background story according to the user's requests. For example, if the user requests a "friendly personality," the text generation AI will generate dialogue that matches that personality.

[0733] 3. Voice generation using AI

[0734] The server uses a voice generation AI to generate the character's voice. This voice generation AI can customize the character's voice quality and speaking style according to the user's request. For example, if the user requests a "soft voice," the voice generation AI will generate a voice that meets that request.

[0735] 4. Emotion Prediction Methods

[0736] The server uses an emotion prediction means to predict emotions from the user's input and actions. The emotion prediction means analyzes the user's text input and voice input and estimates the user's emotional state. For example, if the user inputs "I'm tired today," the emotion prediction means predicts that the user is tired.

[0737] 5. Content Recommendation Methods

[0738] The server recommends appropriate content based on the user's emotion predicted by the emotion prediction means. This content recommendation means suggests relaxing music, videos, news, entertainment content, etc. according to the user's emotional state. For example, if the server predicts that the user is tired, it recommends a relaxing music playlist.

[0739] 6. Means of interaction with users

[0740] The server uses a user interaction means to allow the character to have a natural conversation with the user. This interaction means generates an appropriate response according to the user's input, allowing the character to interact with the user. For example, if the user inputs "I'm tired today," the character responds, "I recommend a relaxing music playlist."

[0741] Examples of concrete examples and prompts

[0742] For example, if a user types, "I feel tired today," the character will respond, "I recommend a relaxing music playlist." In this way, predicting the user's emotions and providing appropriate content improves the user experience.

[0743] Prompt Sentence Examples

[0744] User: I'm tired today.

[0745] Character: I recommend a relaxing music playlist.

[0746] This invention makes it possible to predict user emotions and provide appropriate content, improving the user experience. It also makes it possible to realize a system with features such as character customization, global compatibility, continuous operation, and compatibility with a variety of media.

[0747] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0748] Step 1:

[0749] The user accesses the character generation system using a terminal and inputs their character customization requests. The input data includes customization information such as the character's appearance, personality, and voice. The server receives this input data and passes it on to the image generation AI, text generation AI, and voice generation AI.

[0750] Step 2:

[0751] The server uses image generation AI to generate a character's appearance based on the user's requests. The server receives the user's customization requests as input data, and the image generation AI generates the character's appearance based on those requests. Image data of the generated character is output.

[0752] Step 3:

[0753] The server uses a text generation AI to generate text that expresses the character's ego and personality. It receives the user's customization requests as input data, and the text generation AI generates the character's personality and background story based on those requests. The generated text data is output.

[0754] Step 4:

[0755] The server uses a voice generation AI to generate the character's voice. It receives the user's customization requests as input data, and the AI ​​generates the character's voice quality and speaking style based on those requests. The generated voice data is output.

[0756] Step 5:

[0757] The user starts a conversation with the character using the terminal. The server receives the user's input (text or voice) and passes it to the emotion prediction means. The content of the user's conversation is used as input data.

[0758] Step 6:

[0759] The server predicts the user's emotions using emotion prediction means. The server receives the user's dialogue content as input data, analyzes the content, and estimates the user's emotional state. The server outputs predicted emotion data.

[0760] Step 7:

[0761] The server recommends appropriate content based on the user's emotion predicted using the content recommendation means. The server receives the predicted emotion data as input data, and the content recommendation means selects content according to the emotion. The recommended content data is output.

[0762] Step 8:

[0763] The server uses a means for interacting with the user to have the character present recommended content to the user. The server receives the recommended content data as input data, and the interactive means conveys the content to the user. The user receives a response from the character through the terminal.

[0764] Step 9:

[0765] The user confirms the recommended content and initiates further dialogue if necessary. The server receives the user's input again and repeats the process from step 5.

[0766] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0767] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0768] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0769] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0770] [Second embodiment]

[0771] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0772] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0773] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0774] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0775] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0776] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0777] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0778] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0779] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0780] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0781] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0782] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0783] "Example 1"

[0784] One aspect of the present invention is a system that uses image generation AI, text generation AI, and voice generation AI to generate characters that can be used on social media. The image generation AI generates the character's appearance. The text generation AI generates the character's ego, that is, its personality and thoughts. The voice generation AI generates the character's voice. These AIs generate characters with consistent character settings that can be used on social media.

[0785] "Example 2"

[0786] In one embodiment of the present invention, the generated characters are used to advertise apparel, food, furniture, and home appliances. Specifically, the products are promoted by posting on social media a scene of the character wearing a particular apparel item or eating a particular food. This reduces filming costs and avoids the risk of scandals.

[0787] "Example 3"

[0788] In one embodiment of the present invention, the generated character has characteristics such as customizability, global compatibility, continuous operation, and compatibility with various media, and can appear in online events and have its own unique story and worldview. Specifically, the character's appearance, personality, voice, etc. can be customized according to the user's requests. Furthermore, the character can be operated 24 hours a day and can be accessed from anywhere in the world. Furthermore, the character can be active in various media such as television, radio, and the Internet.

[0789] The processing flow of each embodiment will be described below.

[0790] "Example 1"

[0791] Step 1: Generate the character's appearance using image generation AI. Specifically, set the character's features such as hair color, eye color, and body type.

[0792] Step 2: The text generation AI generates the character's ego, specifically setting the character's personality, hobbies, likes, dislikes, and other characteristics.

[0793] Step 3: The voice generation AI generates the character's voice. Specifically, the pitch, tone, speaking style, and other characteristics of the character's voice are set.

[0794] "Example 2"

[0795] Step 1: Create an advertisement for an apparel product using the generated character. Specifically, generate an image of the character wearing a specific apparel product.

[0796] Step 2: Post the generated advertising image to social media. Specifically, post an image of a character wearing the product along with detailed product information and how to purchase it.

[0797] "Example 3"

[0798] Step 1: Customize the generated character's appearance, personality, voice, etc. according to the user's requests. Specifically, change the character's hair color, personality, etc., based on the user's requests.

[0799] Step 2: The customized character will appear in online events. Specifically, the character will participate in online events, interact with users, promote products, etc.

[0800] Example 1

[0801] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0802] With conventional character generation systems, it was difficult to consistently generate a character's appearance, personality, and voice, making it impossible to efficiently create characters suitable for social media management. Furthermore, in the case of advertisements and online events, the high cost of filming and the risk of scandals meant that customization and global support were required.

[0803] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0804] In this invention, the server includes a means for receiving a prompt text entered by a user, a means for generating a character's appearance using an image generation AI based on the prompt text, a means for generating a character's ego using a sentence generation AI based on the prompt text, a means for generating a character's voice using a voice generation AI based on the prompt text, a means for integrating the generated appearance, personality, and voice to create a consistent character profile, and a means for providing the character profile to the user. This allows users to efficiently generate characters with consistent profile settings suitable for use on social media, reducing filming costs and avoiding the risk of scandals in advertisements and online events. Furthermore, characters with characteristics such as customizability, global compatibility, continuous operation, and compatibility with various media can be generated.

[0805] A "prompt sentence" is text containing instructions or requests that a user inputs to a generative AI model.

[0806] "Image generation AI" is an artificial intelligence model that generates images based on input prompt text.

[0807] "Sentence generation AI" is an artificial intelligence model that generates sentences based on input prompts.

[0808] "Speech generation AI" is an artificial intelligence model that generates speech based on an input prompt.

[0809] "Character appearance" refers to a character's appearance and physical characteristics.

[0810] "Character ego" refers to a character's personality, thoughts, and behavior patterns.

[0811] "Character voice" refers to the sounds and speaking characteristics of a character.

[0812] "Consistent character settings" refers to the overall image of a character with consistent characteristics in appearance, personality, and voice.

[0813] "SNS-operable character" refers to a character that can be active on a social networking service.

[0814] "Reducing filming costs" refers to reducing the costs of actual filming by using characters.

[0815] "Scandal risk avoidance" refers to the avoidance of the risk of scandals related to real people by using characters.

[0816] "Customizability" refers to the ability to change a character's characteristics according to the user's needs.

[0817] "Global compatibility" refers to the characteristics of a character that can adapt to different languages ​​and cultures.

[0818] "Always-on" refers to the ability of a character to be active 24 hours a day, 365 days a year.

[0819] "Multimedia compatible" refers to the ability of a character to operate in different media formats (e.g., images, video, audio).

[0820] "Appearing at an online event" refers to a character's participation in an event held via the Internet.

[0821] "Unique story and worldview" refers to the unique background, story, and setting that each character has.

[0822] This invention is a system that generates a character with consistent character settings that can be used in SNS management using image generation AI, text generation AI, and voice generation AI based on a prompt sentence entered by a user. A specific embodiment of this system is described below.

[0823] Hardware and software used

[0824] The server uses the following hardware and software:

[0825] Hardware: High-performance server computer

[0826] Software: Image generation AI (e.g., DALL-E, Stable Diffusion), text generation AI (e.g., GPT-4), speech generation AI (e.g., Tacotron 2, WaveNet)

[0827] Data processing and calculation

[0828] The server receives the prompt text entered by the user from the terminal and passes it to each generative AI model in an appropriate format. Specifically, it receives the following prompt text:

[0829] Appearance: "Young female character, blue hair, casual clothing"

[0830] Personality: "This character is curious and loves adventure."

[0831] Voice: "This character's voice has a bright, cheerful tone."

[0832] The server analyzes the received prompt sentence and performs the following process.

[0833] Appearance generation using image generation AI

[0834] The server passes a prompt about appearance to the image generation AI. The image generation AI generates an image of a character with the specified characteristics and returns it to the server. For example, based on the prompt "young female character, blue hair, casual clothing," it generates an image of the corresponding character.

[0835] Personality generation using sentence generation AI

[0836] The server passes a prompt about the character's personality to the sentence generation AI. The sentence generation AI generates a sentence about the character's personality and thoughts with the specified characteristics and returns it to the server. For example, based on the prompt "This character is curious and loves adventure," it generates a sentence that describes the character's personality in detail.

[0837] Voice generation using AI

[0838] The server passes a voice prompt to the voice generation AI. The voice generation AI generates a character's voice with the specified characteristics and returns it to the server. For example, based on the prompt, "This character's voice has a bright, cheerful tone," it generates a voice file for the corresponding character.

[0839] Character Settings Integration

[0840] The server integrates the generated images, text, and audio to create a consistent character profile, which is then saved in a format suitable for SNS management.

[0841] Character provision

[0842] The server sends the generated character settings to the device. The user can then check the character through the device and start using it on the SNS. For example, the user can post on the SNS using the generated character.

[0843] In this way, users can efficiently create characters with consistent character settings that can be used on social media, reducing filming costs and avoiding the risk of scandals when it comes to advertising and online events. Characters can also be created with features such as customizability, global compatibility, continuous operation, and compatibility with various media.

[0844] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0845] Step 1:

[0846] The user inputs a prompt into the input form on the terminal. For example, the following prompt is input:

[0847] Appearance: "Young female character, blue hair, casual clothing"

[0848] Personality: "This character is curious and loves adventure."

[0849] Voice: "This character's voice has a bright, cheerful tone."

[0850] The entered prompt sentence is sent from the terminal to the server.

[0851] Step 2:

[0852] The server analyzes the prompts received from the device. The analyzed prompts are passed to each generation AI model in an appropriate format. For example, prompts about appearance are passed to the image generation AI, prompts about personality are passed to the sentence generation AI, and prompts about voice are passed to the voice generation AI.

[0853] Step 3:

[0854] The server passes a prompt about appearance to the image generation AI. The image generation AI generates an image of a character with the specified characteristics. For example, based on the prompt "young female character, blue hair, casual clothing," it generates an image of the corresponding character. The generated image is returned to the server.

[0855] Step 4:

[0856] The server passes a prompt about the character's personality to the sentence generation AI. The sentence generation AI generates a sentence about the character's personality and thoughts with the specified characteristics. For example, based on the prompt "This character is curious and loves adventure," it generates a sentence that describes the character's personality in detail. The generated sentence is returned to the server.

[0857] Step 5:

[0858] The server passes a voice prompt to the voice generation AI. The voice generation AI generates a character's voice with the specified characteristics. For example, based on the prompt, "This character's voice has a bright, cheerful tone," it generates a voice file for the corresponding character. The generated voice file is returned to the server.

[0859] Step 6:

[0860] The server integrates the generated images, text, and audio to create a consistent character profile. For example, the server combines the generated images, text about the character's personality, and audio files to create a unified character profile. This character profile is stored in a format suitable for SNS management.

[0861] Step 7:

[0862] The server sends the generated character settings to the device. The user can then check the character through the device and start using it on the SNS. For example, the user can post on the SNS using the generated character.

[0863] (Application example 1)

[0864] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0865] With conventional character generation systems, it was difficult to generate virtual influencers with consistent character settings for social media management and content distribution services. Furthermore, the cost and time required to individually set the character's appearance, personality, and voice made efficient management difficult. Furthermore, filming costs and the risk of scandals were also issues for advertising and online events.

[0866] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0867] In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a text generation AI, a voice generation means using a voice generation AI, and a means for managing a virtual influencer in a content distribution service using the generated character. This allows for efficient generation of virtual influencers with consistent character settings and their use in social media management and content distribution services. Furthermore, it also enables reductions in filming costs and avoidance of scandal risks in advertising and online events.

[0868] "Image generation AI" is an artificial intelligence technology that generates images based on input text or prompts.

[0869] "Sentence generation AI" is an artificial intelligence technology that generates natural-sounding sentences based on input text and prompts.

[0870] "Speech generation AI" is an artificial intelligence technology that generates speech based on input text.

[0871] "Appearance generation means" refers to a means of generating a character's appearance or figure using image generation AI.

[0872] "Self-generation means" is a means of generating a character's personality and thoughts using text generation AI.

[0873] "Voice generation means" refers to a means of generating a character's voice using voice generation AI.

[0874] A "virtual influencer" is a virtual influencer who uses a generated character to be active on social media and content distribution services.

[0875] A "content distribution service" is a service that distributes content such as video, audio, and text via the Internet.

[0876] "Filming costs" are the expenses required to shoot characters and advertising materials.

[0877] "Scandal risk" refers to the risk of scandal or problematic behavior that a character or influencer may cause.

[0878] An "online event" is an event held over the internet.

[0879] "Customizability" refers to the ability of users to freely change a character's appearance, personality, voice, etc.

[0880] "Global compatibility" refers to the ability of characters and systems to support multiple languages ​​and cultures.

[0881] "Always-on operation" refers to the characteristic of characters and systems operating 24 hours a day, 365 days a year.

[0882] "Multimedia compatibility" refers to the ability of characters and systems to support multiple media formats such as video, audio, and text.

[0883] A "unique story" is a character's unique background or story.

[0884] A "worldview" is the virtual environment or setting in which a character operates.

[0885] The following system configuration and processing procedure will be described as an embodiment of the present invention.

[0886] System Configuration

[0887] This system uses the following hardware and software:

[0888] Hardware: Servers, smartphones, PCs

[0889] Software: Python, OpenAI API, PIL (Python Imaging Library), pyttsx3 (Python Text-to-Speech)

[0890] Processing steps

[0891] The server includes the following means:

[0892] 1. Image generation method: Using image generation AI, generate the character's appearance based on the input prompt.

[0893] 2. Self-generation method: Using text generation AI, generate a character's personality and thoughts based on the input prompts.

[0894] 3. Voice generation method: Using voice generation AI, a character's voice is generated based on the input text.

[0895] 4. Virtual influencer management means: Using the generated characters, virtual influencers are managed in content distribution services.

[0896] Data processing and calculation

[0897] The server processes and calculates the data as follows.

[0898] 1. Image Generation: The server uses the OpenAI API to generate an image of the character based on the prompt, and then converts the image to the appropriate format using PIL.

[0899] 2. Sentence Generation: The server uses the OpenAI API to generate the character's personality and thoughts based on the prompts. The generated sentences are saved as the character's ego.

[0900] 3. Voice generation: The server uses pyttsx3 to convert the generated text into an audio file, which generates the character's voice.

[0901] 4. Virtual Influencer Management: The server uses the generated character to automate activities on social media and content distribution services, including scheduling posts and interacting with fans.

[0902] Specific examples

[0903] Below are some examples of prompt sentences.

[0904] Image prompt: "A friendly and charismatic virtual influencer"

[0905] Writing prompt: "Describe the personality and daily activities of a virtual influencer who loves fashion and technology."

[0906] Based on this prompt, the server generates the character's appearance, personality, and voice, and allows the character to act as a virtual influencer on social media and content distribution services.

[0907] In this way, this invention makes it possible to efficiently generate virtual influencers with consistent character settings and utilize them in social media management and content distribution services. Furthermore, it also makes it possible to reduce filming costs and avoid the risk of scandals in advertising and online events.

[0908] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0909] Step 1:

[0910] The user inputs prompts about the character's appearance, personality, and voice using a smartphone or PC, which are then sent to the server.

[0911] Input: Prompts about the character's appearance, personality, and voice

[0912] Output: The prompt sent to the server

[0913] Step 2:

[0914] The server uses image generation AI to generate the character's appearance based on the input prompt, and the generated image is converted to the appropriate format using PIL.

[0915] Input: Prompt for character appearance

[0916] Output: An image of the generated character

[0917] Step 3:

[0918] The server uses a sentence generation AI to generate the character's personality and thoughts based on the input prompt sentence, and the generated sentence is saved as the character's ego.

[0919] Input: Character personality prompt

[0920] Output: Text about the generated character's personality and thoughts

[0921] Step 4:

[0922] The server uses a speech generation AI to convert the generated text into an audio file, which then generates the character's voice.

[0923] Input: Text about the character's personality and thoughts

[0924] Output: The generated character's voice

[0925] Step 5:

[0926] The server uses the generated character to automate activities on social media and content distribution services, including scheduling posts and interacting with fans.

[0927] Input: Image of the generated character, text about its personality and thoughts, and voice

[0928] Output: Character activities on social media and content distribution services

[0929] In this way, users can easily create virtual influencers and have them active on social media and content distribution services.

[0930] Example 2

[0931] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0932] Traditional advertising methods require the use of real models and locations, which not only increases filming costs but also poses the risk of scandal. While there is a demand for systems that offer features such as character customization, global compatibility, continuous operation, and compatibility with a variety of media, there is a lack of efficient methods to achieve this.

[0933] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0934] In this invention, the server includes an image generation means, a sentence generation means, a voice generation means, a means for inputting a prompt sentence, a means for analyzing the prompt sentence, a means for converting the generated data into a format for posting to an SNS, and a means for posting the formatted data to the SNS. This reduces filming costs, avoids the risk of scandals, and enables character customization, global compatibility, continuous operation, and compatibility with a variety of media.

[0935] "Image generation means" refers to a technique for generating an image based on a prompt sentence.

[0936] A "sentence generator" is a technique for generating text based on a prompt sentence.

[0937] The "voice generation means" is a technique for generating voice based on a prompt sentence.

[0938] The "means for inputting a prompt sentence" is an interface for the user to input the content they wish to generate in text format.

[0939] "Means for analyzing prompt sentences" refers to technology for analyzing input prompt sentences and converting them into a format suitable for a generative AI model.

[0940] "Means for converting generated data into a format for posting on SNS" refers to technology for converting generated images and text into a format suitable for SNS.

[0941] "Means for posting formatted data to a social networking site" refers to a technology for posting formatted data to a specified social networking site account.

[0942] This invention relates to a system for generating characters using a generative AI model and posting the characters to a social networking service. Specific embodiments of this system are described below.

[0943] System configuration

[0944] Hardware

[0945] Server: Runs the generative AI model, analyzes prompts, and converts data formats.

[0946] Terminal: A device on which a user enters a prompt, such as a computer, smartphone, or tablet.

[0947] software

[0948] Generative AI model: An AI model that generates images, text, and speech. Specifically, OpenAI's GPT-4 and DALL-E are used.

[0949] Social media posting formatter: Software that converts the generated data into a format suitable for social media.

[0950] Program processing

[0951] Entering a prompt statement

[0952] Using a dedicated application on a device or a web interface, a user enters a prompt that describes the scene they want to create for the character, for example, "Create a scene where the character is wearing a red dress."

[0953] Sending a prompt

[0954] The terminal sends the prompt text entered by the user to the server, where it is converted into an appropriate format and sent to the server.

[0955] Parsing the prompt statement

[0956] The server analyzes the received prompt and converts it into a format suitable for the generative AI model, for example, extracting keywords such as "character wearing a red dress."

[0957] Requests to generative AI models

[0958] The server sends a request to the generative AI model based on the analysis results, which includes the content of the prompt sentence.

[0959] Character image generation

[0960] The generative AI model receives requests from the server and generates a character image based on the specified scene, for example, an image of a character wearing a red dress.

[0961] Receiving the generated image

[0962] The server receives the character image returned by the generative AI model, which is generated based on the prompt text.

[0963] Image format conversion

[0964] The server then converts the received image into a format suitable for posting to the social networking site, for example by adjusting the size of the image and adding a caption.

[0965] Posting to social media

[0966] The server posts the formatted image to a specified social media account, which then publishes a scene of the generated character wearing a specific apparel item or eating a specific food item on the social media.

[0967] Specific examples

[0968] For example, if the user enters the prompt "Generate a scene where the character is wearing a red dress," the following specific actions will occur:

[0969] 1. The user types into the device application, "Generate a scene where the character is wearing a red dress."

[0970] 2. The terminal sends this prompt to the server.

[0971] 3. The server parses the prompt and extracts the keyword "character wearing a red dress."

[0972] 4. The server sends a request to the generation AI model saying, "Generate a character wearing a red dress."

[0973] 5. The generative AI model generates an image of a character wearing a red dress.

[0974] 6. The server receives the generated image and adjusts the size of the image for posting on social media.

[0975] 7. The server adds the caption "Check out our new red dress!" to the adjusted image.

[0976] 8. The server posts this image to the company's social media accounts.

[0977] This system allows companies to advertise efficiently, reduce filming costs, and avoid the risk of scandals.

[0978] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0979] Step 1:

[0980] The user enters a prompt statement.

[0981] Using a dedicated application or web interface on the device, the user inputs a prompt that describes the scene they want to create for the character. For example, they might input "Create a scene where the character is wearing a red dress." The input prompt is then stored in the device's memory.

[0982] Step 2:

[0983] The terminal sends the prompt to the server.

[0984] The terminal sends the prompt text entered by the user to the server. At this time, the prompt text is converted into an appropriate format and sent to the server over the network. The input is the prompt text, and the output is the prompt text sent to the server.

[0985] Step 3:

[0986] The server parses the prompt.

[0987] The server parses the received prompt and converts it into a format suitable for the generative AI model. For example, it extracts keywords such as "character wearing a red dress." The input is the prompt, and the output is the parsed keywords.

[0988] Step 4:

[0989] The server sends a request to the generative AI model.

[0990] The server sends a request to the generative AI model based on the analysis results. This request includes the contents of the prompt sentence. The input is the analyzed keyword, and the output is the request to the generative AI model.

[0991] Step 5:

[0992] A generative AI model generates character images.

[0993] The generative AI model receives a request from the server and generates a character image based on the specified scene. For example, generate an image of a character wearing a red dress. The input is the request to the generative AI model, and the output is the generated character image.

[0994] Step 6:

[0995] The server receives the generated image.

[0996] The server receives the character image returned by the generative AI model. This image is generated based on the prompt. The input is the generated character image, and the output is the image stored on the server.

[0997] Step 7:

[0998] The server formats the image for posting to social media.

[0999] The server converts the received image into a format suitable for posting to a social networking site, for example by adjusting the size of the image or adding a caption. The input is the generated character image, and the output is the formatted image.

[1000] Step 8:

[1001] The server posts the formatted image to a social networking site.

[1002] The server posts the formatted image to the specified social media account. This allows a scene of the generated character wearing a specific apparel item or eating a specific food to be published on the social media. The input is the formatted image, and the output is the image posted to the social media.

[1003] (Application example 2)

[1004] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1005] Traditional advertising production involves high filming costs and the risk of scandals. Furthermore, character-based advertising requires character customization and compatibility with a variety of media, but the technology required to achieve this is lacking. Furthermore, there are limited ways to automate posting to social media, making efficient advertising management difficult.

[1006] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a sentence generation AI, a voice generation means using a voice generation AI, and a means for automatically creating a scene in which the generated character uses a specific product and posting it to a social networking site. This enables reduction in photography costs, avoidance of the risk of scandals, character customization, compatibility with various media, and automatic posting to a social networking site.

[1007] "Image generation AI" is a system that generates images using artificial intelligence technology.

[1008] "Appearance generation means" is a function that generates a character's appearance using image generation AI.

[1009] "Text generation AI" is a system that generates text using artificial intelligence technology.

[1010] "Self-generation means" is a function that uses text generation AI to generate character personalities and background stories.

[1011] "Voice generation AI" is a system that generates voice using artificial intelligence technology.

[1012] "Voice generation means" is a function that generates a character's voice using voice generation AI.

[1013] The "means for automatically creating a scene in which a specific product is used" is a function for automatically creating a scene in which a generated character uses a specific product.

[1014] "Means for posting to SNS" is a function that automatically posts the generated scene to SNS.

[1015] "Apparel" refers to clothing and fashion items.

[1016] "Food" means food and drink intended for human consumption.

[1017] "Furniture and home appliances" refers to furniture and electrical appliances used in the home.

[1018] "Photography costs" refer to the costs incurred when taking photos or videos for advertising or promotional purposes.

[1019] "Scandal risk" refers to the risk that people or companies involved in advertising may become involved in a scandal.

[1020] "Customizability" refers to the ability for users to freely change the appearance and personality of their characters.

[1021] "Global compatibility" refers to the ability to adapt to different languages ​​and cultures.

[1022] "Continuous operation" refers to the system's ability to operate 24 hours a day, 365 days a year.

[1023] "Multimedia compatibility" refers to the ability of a character to be compatible with multiple media formats (images, videos, audio, etc.).

[1024] "Appearing at an online event" refers to the generated character participating in an event held via the Internet.

[1025] "Unique story and worldview" refers to the unique background story and setting of each character.

[1026] This invention is a system that generates a character using a generative AI model, automatically creates a scene of the character using a specific product, and posts the scene to a social networking site. Specific embodiments of this system are described below.

[1027] System configuration

[1028] The system consists of the following main components:

[1029] 1. Image generation AI: This is an artificial intelligence technology for generating character appearances. Specifically, it uses image generation models such as OpenAI's DALL-E.

[1030] 2. Text generation AI: This is an artificial intelligence technology for generating character personalities and background stories. Specifically, it uses text generation models such as OpenAI's GPT-4.

[1031] 3. Voice Generation AI: This is an artificial intelligence technology for generating character voices. Specifically, it uses voice generation models such as Google's WaveNet.

[1032] 4. SNS posting module: This module automatically posts scenes of generated characters using specific products to SNS.

[1033] System Operation

[1034] 1. Character Generation:

[1035] The server generates the character's appearance using image generation AI.

[1036] Next, a text generation AI is used to generate the character's ego and background story.

[1037] Finally, a voice generation AI is used to generate the character's voice.

[1038] 2. Create a scene:

[1039] The server automatically creates a scene in which a generated character uses a specific product, with the user providing a prompt such as:

[1040] Apparel: "Generate an image of a character wearing a new t-shirt."

[1041] Food: "Generate an image of a character eating a new snack."

[1042] Furniture Appliances: "Generate an image of a character sitting on a new sofa."

[1043] 3. Posting to social media:

[1044] The SNS posting module automatically posts the generated scenes to SNS, saving users the trouble of manually posting.

[1045] Hardware and software used

[1046] Hardware: The server uses a computer equipped with a high-performance GPU, which enables high-speed processing of generative AI models.

[1047] Software: OpenAI's DALL-E, GPT-4, and Google's WaveNet are used as generative AI models. The SNS posting module also uses the APIs of each SNS.

[1048] Specific examples

[1049] For example, if a clothing brand wants to advertise a new t-shirt, the user might enter the following prompt text:

[1050] Generate an image of the character wearing the new T-shirt.

[1051] Based on this prompt, the server uses image generation AI to generate a character image, text generation AI to create a background story for the character, and voice generation AI to generate a voice for the character. It then automatically creates a scene of the generated character wearing the new T-shirt and posts it to social media via the social media posting module.

[1052] In this way, it is possible to reduce filming costs, avoid the risk of scandals, customize characters, support a variety of media, and automatically post to social media.

[1053] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1054] Step 1:

[1055] The user uses the terminal to input a prompt, for example, "Please generate an image of a character wearing the new T-shirt." This prompt is then sent to the server.

[1056] Step 2:

[1057] The server analyzes the received prompt text and inputs it into the image generation AI. At this time, data processing is performed to generate the character's appearance based on the prompt text. Specifically, the character image is generated using OpenAI's DALL-E model. The input is the prompt text, and the output is an image of the generated character.

[1058] Step 3:

[1059] The server saves the generated character image and then inputs the prompt text into the text generation AI. During this process, data processing is performed to generate the character's ego and background story. Specifically, the character's background story is generated using OpenAI's GPT-4 model. The input is the prompt text, and the output is the generated background story.

[1060] Step 4:

[1061] The server saves the generated background story and then inputs the prompt text into the voice generation AI. At this time, data processing is performed to generate the character's voice. Specifically, the character's voice is generated using Google's WaveNet model. The input is the prompt text, and the output is the generated voice data.

[1062] Step 5:

[1063] The server integrates the generated character's image, background story, and voice data to automatically create a scene using a specific product. The scene is set based on the prompt entered by the user. The input is the generated character data, and the output is the scene data using a specific product.

[1064] Step 6:

[1065] The server inputs the generated scene data into the SNS posting module and automatically posts it to the SNS. The posting is done using the API of each SNS. The input is the generated scene data, and the output is the result of posting to the SNS.

[1066] Step 7:

[1067] Users can use their devices to check scenes posted to SNS. This saves users the trouble of manually posting. The input is the SNS posting result, and the output is the user's confirmation result.

[1068] Example 3

[1069] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1070] Conventional character generation systems lacked sufficient character customization and compatibility with diverse media, making it difficult to meet the diverse needs of users. Furthermore, it was difficult for users to reflect their specific requests when generating characters, and the generated characters often did not meet users' expectations. Furthermore, there were limitations on the character's operating hours and access regions, making global use difficult.

[1071] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1072] In this invention, the server includes a means for a user to input a prompt sentence, a means for receiving and analyzing the prompt sentence, a means for generating a character profile using a generative artificial intelligence model, a means for generating a voice for the character using speech synthesis software, and a means for providing the generated character to the user, thereby enabling character customization, global compatibility, continuous operation, and compatibility with a variety of media.

[1073] "Image generation artificial intelligence" is an artificial intelligence technology that generates images based on input data.

[1074] "Text generation artificial intelligence" is an artificial intelligence technology that generates text based on input data.

[1075] "Speech generation artificial intelligence" is an artificial intelligence technology that generates speech based on input data.

[1076] A "prompt sentence" is a sentence that describes instructions or requests that a user inputs to a system.

[1077] A "server" is a computer system that processes data and provides services over a network.

[1078] "Means of analysis" refers to the techniques and methods used to understand the input data and extract the necessary information.

[1079] A "generative artificial intelligence model" is a model for generating data using artificial intelligence technology.

[1080] "Speech synthesis software" is software for generating speech based on text data.

[1081] A "character profile" is information that describes a character's detailed settings and characteristics.

[1082] "Customizability" refers to the ability to change and adjust systems and characters according to the user's needs.

[1083] "Global compatibility" refers to the ability to access and use the system from anywhere in the world.

[1084] "Continuous operation" refers to the ability of a system to operate continuously 24 hours a day, 365 days a year.

[1085] "Multimedia compatibility" refers to the ability of a system or character to operate across multiple media, such as television, radio, and the Internet.

[1086] This invention is a system in which a user inputs a prompt sentence, and a server analyzes the prompt sentence and generates a character. A specific embodiment of this system will be described below.

[1087] First, the user inputs a prompt through the system interface, which includes detailed requests for the character's appearance, personality, voice, etc. For example, the following prompt can be input:

[1088] "Please create a character to appear at the online event. The character should be a woman in her 20s with a bright and lively personality, long blonde hair, and blue eyes. Please use a high-pitched, friendly voice."

[1089] Next, the server receives the prompt text sent by the user. The received prompt text is passed to an analysis module within the server. The server uses natural language processing technology to analyze the prompt text and extract each element of the character (appearance, personality, voice, etc.). Specifically, it uses a natural language processing library (e.g., spaCy).

[1090] Based on the analysis results, the server uses a generative AI model (e.g., OpenAI GPT-4) to generate a text-based profile for the character, which includes detailed settings for the character. The generated profile details the character's appearance, personality, background story, etc.

[1091] The server then uses speech synthesis software (e.g., Google Text-to-Speech) to generate a character voice based on the generated text-based profile, reflecting the tone and characteristics specified in the prompt.

[1092] Finally, the server provides the generated character's profile and voice to the user, who can then download the character and use it in online events and other media. This system enables character customization, global compatibility, continuous operation, and compatibility with a variety of media.

[1093] It is recommended to use a server equipped with a high-performance GPU (e.g., NVIDIA Tesla V100) to implement this system, as this will enable fast and efficient processing of the generative AI model and speech synthesis software.

[1094] As described above, the present invention provides a specific means for generating a customizable character in response to various user needs. The flow of the specification process in the third embodiment will be described with reference to FIG.

[1095] Step 1:

[1096] The user enters a prompt statement

[1097] The user inputs a prompt statement describing the character's requirements through the system interface. For example, the user can input a prompt statement such as, "Please create a character to appear in an online event. The character should be a woman in her twenties with a bright and lively personality, long blonde hair, and blue eyes. The voice should be high-pitched and friendly." The input data is text data containing detailed requirements such as the character's appearance, personality, and voice.

[1098] Step 2:

[1099] The server receives the prompt.

[1100] The server receives the prompt text sent by the user as an HTTP request. The received prompt text is passed to the analysis module in the server. The input data is the prompt text entered by the user, and the output data is the text data passed to the analysis module.

[1101] Step 3:

[1102] The server parses the prompt.

[1103] The server analyzes the prompt using natural language processing technology. Specifically, it uses a natural language processing library (e.g., spaCy) to understand the content of the prompt and extract each element of the character (appearance, personality, voice, etc.). The input data is the text data of the prompt, and the output data is data containing each element of the character as the analysis result.

[1104] Step 4:

[1105] The server generates a character profile using a generative AI model.

[1106] The server uses a generative AI model (e.g., OpenAI GPT-4) based on the analysis results to generate a text-based profile for the character. The generated profile contains detailed settings for the character. The input data is data containing each element of the character as a result of the analysis, and the output data is the generated character profile.

[1107] Step 5:

[1108] The server generates the character's voice using speech synthesis software.

[1109] The server generates a character voice using speech synthesis software (e.g., Google Text-to-Speech) based on the generated text-based profile. The generated voice reflects the tone and characteristics specified in the prompt. The input data is the generated character profile, and the output data is the generated character voice.

[1110] Step 6:

[1111] The server provides the generated character to the user.

[1112] The server provides the generated character's profile and voice to the user. The user can download the provided character and use it in online events or other media. The input data is the generated character's profile and voice, and the output data is the character data provided to the user.

[1113] (Application example 3)

[1114] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1115] Conventional character generation systems have limited user customization and interactive dialogue capabilities, and are not yet fully compatible with global platforms, 24 / 7 operation, or diverse media. Furthermore, there is a lack of systems that allow users to interact with characters in real time and advance the story. This limits the ability to improve the user experience and apply them to a variety of purposes.

[1116] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1117] In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a text generation AI, a voice generation means using a voice generation AI, an interactive dialogue system means, a multilingual support means, a server infrastructure means that can operate 24 hours a day, a means for a user-customized character to tell a unique story, and a means for updating the story in real time according to user selections. As a result, the user-customized character can interactively tell a story and be updated in real time, improving the user experience and enabling applications in a variety of uses.

[1118] "Image generation AI" is an artificial intelligence technology that generates a character's appearance based on user input.

[1119] "Text generation AI" is an artificial intelligence technology for generating character egos and stories.

[1120] "Voice generation AI" is an artificial intelligence technology for generating character voices.

[1121] An "interactive dialogue system" is a system that allows users and characters to interact in real time.

[1122] "Multilingual support" is a function that supports multiple languages ​​and translates what characters say according to the user's language.

[1123] "24-hour server infrastructure" refers to server infrastructure that can be kept running at all times.

[1124] "A means for user-customized characters to tell their own stories" refers to a means for users to create and tell their own stories with characters they have created.

[1125] "Means for updating a story in real time according to user selection" means means for changing and updating a story in real time based on user selection.

[1126] The following system configuration and processing procedure will be described as an embodiment of the present invention.

[1127] System Configuration

[1128] This system uses the following hardware and software:

[1129] Hardware

[1130] Smartphone (iOS, Android)

[1131] Head-mounted displays (Oculus Quest, HTC Vive, etc.)

[1132] Server infrastructure (AWS, etc.)

[1133] software

[1134] Unity (game engine)

[1135] TensorFlow (generative AI model)

[1136] Google Translate API (multilingual support)

[1137] Processing steps

[1138] 1. Character Creation and Customization

[1139] The user launches the application using a smartphone or head-mounted display.

[1140] Users customize their character's appearance, personality, and voice within the application.

[1141] Unity is used to generate characters in real time, and TensorFlow is used to generate the character's voice.

[1142] 2. Story generation and management

[1143] It uses a generative AI model (such as GPT-4) to generate a story based on a theme selected by the user.

[1144] Stories are managed on a server and updated in real time based on user choices.

[1145] 3. Interactive dialogue systems

[1146] When a user interacts with the character, it uses voice recognition technology to analyze the user's input and generate an appropriate response.

[1147] The content of the dialogue is reflected in the story in real time.

[1148] 4. Multilingual support

[1149] Uses Google Translate API to translate what the characters say based on the user's language.

[1150] 5. 24 / 7 server infrastructure

[1151] Using AWS, we will build a server infrastructure that operates 24 hours a day and allows users around the world to access our services.

[1152] Specific examples

[1153] The user launches the "Virtual Character Storyteller" and customizes a character set in a fantasy world. The character's name is "Elise the Elf," its personality is "brave and intelligent," and its voice is "gentle, female." When the user says, "Elise, tell me a story about your adventure," Elise begins telling the story of a new adventure using a generative AI model. When the user asks, "What happens next?", Elise continues the story, and the story progresses according to the user's choices.

[1154] Prompt Sentence Examples

[1155] The character "Elise the Elf" is brave, intelligent, and speaks with a gentle, female voice. Generate a scene in which Elise narrates a fantasy adventure story. Include a response when the user asks, "What happens next?"

[1156] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1157] Step 1:

[1158] A user starts an application using a smartphone or a head-mounted display.

[1159] Input: User device operations

[1160] Output: Application launch screen

[1161] Specific behavior: The user taps the application icon and the application launches.

[1162] Step 2:

[1163] Users customize the character's appearance, personality, and voice within the application.

[1164] Input: User customization settings (appearance, personality, voice)

[1165] Output: Customized character data

[1166] How it works: The user sets each of the character's attributes using sliders and drop-down menus, and the settings are updated in real time.

[1167] Step 3:

[1168] The server uses Unity to generate characters in real time and TensorFlow to generate the character's voice.

[1169] Input: Customized character data

[1170] Output: 3D model and voice data of the generated character

[1171] How it works: The server launches the Unity engine and generates a 3D model of the character based on the user's settings, while simultaneously generating the character's voice using TensorFlow.

[1172] Step 4:

[1173] It uses a generative AI model (such as GPT-4) to generate a story based on a theme selected by the user.

[1174] Input: Theme selected by the user

[1175] Output: Text data of the generated story

[1176] Specific operation: The server inputs the theme into the generative AI model and generates a prompt sentence. A story is generated based on the generated prompt sentence.

[1177] Step 5:

[1178] The server manages the stories and updates them in real time according to the user's selections.

[1179] Input: Text data of the generated story, user selection

[1180] Output: Updated story text data

[1181] What happens: The server receives the user's selections and updates the story progress in real time. The updated story is displayed to the user.

[1182] Step 6:

[1183] As the user interacts with the character, voice recognition technology is used to analyze the user's input and generate appropriate responses.

[1184] Input: User voice input

[1185] Output: Character voice response

[1186] How it works: When a user speaks into the microphone, speech recognition technology converts what they say into text, and a generative AI model generates an appropriate response, which is then played back as the character's voice.

[1187] Step 7:

[1188] Uses the Google Translate API to translate what the characters say based on the user's language.

[1189] Input: Character's spoken text data, user's language settings

[1190] Output: Translated speech text data

[1191] Specific operation: The server sends the character's speech to the Google Translate API, receives the text data translated into the user's language, and plays the translated speech as the character's voice.

[1192] Step 8:

[1193] Using AWS, we will build a server infrastructure that operates 24 hours a day and allows users around the world to access it.

[1194] Input: Server infrastructure configuration data

[1195] Output: 24-hour server infrastructure

[1196] How it works: A server administrator uses AWS to set up server infrastructure and create a 24 / 7 environment, allowing users around the world to access the service at any time.

[1197] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1198] "Example 1"

[1199] One embodiment of the present invention is an AI character generation system incorporating an emotion engine. This system is capable of recognizing a user's emotions and adjusting the character's behavior and speech accordingly. Specifically, if the user expresses joy, the character will also express joy. Similarly, if the user expresses sadness, the character will act in accordance with the user's emotions, such as offering words of comfort.

[1200] "Example 2"

[1201] The emotion engine can also detect changes in the user's emotions in real time and instantly adjust the character's behavior and speech. For example, if the user suddenly expresses anger, the character will immediately apologize and respond appropriately to the situation.

[1202] "Example 3"

[1203] Furthermore, the emotion engine can learn the user's emotional patterns and predict changes in the user's emotions. This allows the character to act in advance of the user's emotional changes, enabling more natural interactions. For example, if the user feels tired from daily work, the character can predict the user's emotions by playing relaxing music when the user returns home.

[1204] The processing flow of each embodiment will be described below.

[1205] "Example 1"

[1206] Step 1: The emotion engine recognizes the user's emotion.

[1207] Step 2: Adjust the AI ​​character's behavior and speech based on the emotions recognized by the emotion engine.

[1208] Step 3: Present the adjusted actions and statements to the user.

[1209] "Example 2"

[1210] Step 1: The emotion engine captures changes in the user's emotions in real time.

[1211] Step 2: The emotion engine instantly adjusts the AI ​​character's behavior and speech based on the changes in emotions detected.

[1212] Step 3: Present the adjusted actions and statements to the user.

[1213] "Example 3"

[1214] Step 1: The emotion engine learns the user's emotional patterns.

[1215] Step 2: Based on the learned emotional patterns, the emotion engine predicts changes in the user's emotions.

[1216] Step 3: Adjust the AI ​​character's behavior and speech based on the predicted emotional changes.

[1217] Step 4: Present the adjusted actions and statements to the user.

[1218] Example 1

[1219] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1220] Conventional character generation systems can individually generate a character's appearance, personality, and voice, but lack the ability to adjust the character's behavior and speech according to the user's emotions. This results in inconsistent interaction with the user and makes it difficult to respond to emotions in real time. Furthermore, in an age where diverse media such as advertising and online events are required, the lack of character customization and global compatibility is also a problem.

[1221] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a sentence generation AI, a voice generation means using a voice generation AI, a user emotion recognition means using an emotion engine, and a means for adjusting the character's actions and remarks according to the user's emotions. This makes it possible to generate a character that can respond to the user's emotions in real time and has consistent character settings suitable for use on SNS. In addition, the character's customizability, global compatibility, and compatibility with various media are improved, making it easier to use in advertisements, online events, etc.

[1222] "Image generation artificial intelligence" is an artificial intelligence technology that generates images based on input text or prompts.

[1223] "Appearance generation means" refers to techniques or methods for generating a character's appearance or shape.

[1224] "Sentence generation artificial intelligence" is an artificial intelligence technology that generates sentences based on input text and prompts.

[1225] "Self-generation means" refers to techniques and methods for generating a character's personality and thoughts.

[1226] "Speech generation artificial intelligence" is an artificial intelligence technology that generates speech based on input text and prompts.

[1227] "Voice generation means" refers to the techniques and methods used to generate the voice of a character.

[1228] An "emotion engine" is a technology that recognizes emotions from user input and behavior, and adjusts responses and behavior according to those emotions.

[1229] "User emotion recognition means" refers to a technique or method for analyzing and recognizing a user's emotions.

[1230] "Means for adjusting behavior and speech" refers to techniques and methods for changing a character's behavior and speech based on the user's recognized emotions.

[1231] An "SNS-operable character" is a character that can be active on a social networking service and interact with users.

[1232] "Customizability" refers to the ability to change a character's appearance, personality, voice, etc. according to the user's needs and purpose.

[1233] "Global compatibility" refers to the ability to accommodate different languages ​​and cultures and interact with users around the world.

[1234] "Media-versatile" refers to the ability of a character to operate in a variety of media formats (e.g., images, video, audio).

[1235] "Online event appearance" is a feature that allows a character to participate in events held over the Internet and interact with users in real time.

[1236] "Unique story and worldview" refers to the character's unique background setting and narrative, as well as the virtual environment and setting in which the character exists.

[1237] MODE FOR CARRYING OUT THE INVENTION

[1238] This invention is a system for generating characters capable of managing social networking sites using image generation AI, text generation AI, voice generation AI, and an emotion engine. This system can adjust the character's behavior and comments according to the user's emotions.

[1239] 1. System Overview

[1240] The system includes the following main components:

[1241] Image generation artificial intelligence (e.g., DALL-E, Stable Diffusion)

[1242] Text generation artificial intelligence (e.g. GPT-4)

[1243] Speech generation artificial intelligence (e.g., Tacotron 2, WaveNet)

[1244] Emotion Engine

[1245] 2. Hardware and Software Used

[1246] The server performs its processing using the following hardware and software:

[1247] A server equipped with a high-performance GPU (e.g., NVIDIA GPU)

[1248] Image generation AI models (e.g., DALL-E, Stable Diffusion)

[1249] Sentence generation AI model (e.g. GPT-4)

[1250] Speech generation AI models (e.g., Tacotron 2, WaveNet)

[1251] Emotion Recognition Software

[1252] The terminal is a device that the user uses to request character generation, and includes a smartphone, a PC, etc.

[1253] 3. Program Processing Flow

[1254] The user uses a device to submit a character creation request, for example, by entering "a female character in her 20s with a bright smile" into a form on a dedicated app or website, and then pressing the submit button.

[1255] The server performs the following actions based on the request:

[1256] 1. Image generation:

[1257] The server calls an image generation AI model and generates the character's appearance.

[1258] Example: Send the prompt "A female character in her 20s with a bright smile" to an image generation AI and receive the generated image.

[1259] 2. Ego generation:

[1260] The server calls a sentence generation AI model to generate the character's personality and thoughts.

[1261] Example: Send the prompt "A cheerful and sociable person whose hobbies are reading and traveling" to a text generation AI and receive the generated text.

[1262] 3. Voice generation:

[1263] The server calls a voice generation AI model to generate the character's voice.

[1264] Example: Send a prompt such as "a cheerful voice of a woman in her 20s" to a voice generation AI and receive the generated voice data.

[1265] 4. Emotion recognition:

[1266] The server uses an emotion engine to recognize emotions from the user's input (text and / or voice).

[1267] Example: If a user types "I'm so happy today!", the server uses an emotion engine to recognize "joy."

[1268] 5. Behavioral and speech adjustment:

[1269] The server adjusts the character's actions and speech based on the user's recognized emotions.

[1270] Example: If the user types "I'm so happy today!", the server has a character reply "That's great! I'm happy too!"

[1271] In this way, the server can generate a character with consistent character settings that can be used in an SNS and provide interactions that correspond to the user's emotions.

[1272] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1273] Step 1:

[1274] The user uses a device to send a request for character generation. Specifically, they enter "a female character in her 20s with a bright smile" into a form on a dedicated app or website, and then press the send button. The input data is the character's basic attribute information.

[1275] Step 2:

[1276] The server receives a request from a user and calls the image generation AI model. The input data is the character's attribute information sent by the user. Based on this information, the server sends a prompt to the image generation AI model and generates the character's appearance. The output data is an image of the generated character. Specifically, the server sends a prompt to the image generation AI saying, "A female character in her twenties with a bright smile," and receives the generated image.

[1277] Step 3:

[1278] The server saves an image of the generated character and then calls the sentence generation AI model. The input data is information about the character's personality and thoughts sent by the user. Based on this information, the server sends a prompt to the sentence generation AI model to generate the character's ego. The output data is text about the generated character's personality and thoughts. Specifically, the server sends a prompt to the sentence generation AI saying, "He has a cheerful and sociable personality, and his hobbies are reading and traveling," and receives the generated text.

[1279] Step 4:

[1280] The server saves the self information of the generated character and then calls the voice generation AI model. The input data is information about the character's voice sent by the user. Based on this information, the server sends a prompt to the voice generation AI model to generate the character's voice. The output data is the voice data of the generated character. Specifically, the server sends a prompt, "the cheerful voice of a woman in her twenties," to the voice generation AI and receives the generated voice data.

[1281] Step 5:

[1282] The server saves the voice data of the generated character and then calls the emotion engine. The input data is the user's text input or voice input. Based on this information, the server uses the emotion engine to recognize the user's emotion. The output data is the recognized user's emotion information. In concrete terms, if the user inputs "I'm so happy today!", the server will recognize "joy" using the emotion engine.

[1283] Step 6:

[1284] The server adjusts the character's behavior and speech based on the recognized user's emotional information. The input data is the user's emotional information. The server generates the character's behavior and speech based on this information. The output data is the adjusted character's behavior and speech. In concrete terms, if the user inputs "I'm so happy today!", the server will have the character reply "That's great! I'm happy too!"

[1285] In this way, the server can generate a character with consistent character settings that can be used in an SNS and provide interactions that correspond to the user's emotions.

[1286] (Application example 1)

[1287] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1288] Conventional SNS management characters were unable to recognize emotions in conversations with users and act or speak accordingly. This limited the user experience and lacked emotionally sensitive interactions. It was also difficult to personalize advertisements and content delivery based on user emotions. This led to problems such as a decline in user satisfaction and engagement.

[1289] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1290] In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a sentence generation AI, a voice generation means using a voice generation AI, an emotion recognition and behavior adjustment means using an emotion engine, and a means for adjusting the character's behavior and speech according to the user's emotions. This enables interactions according to the user's emotions, providing a more personalized experience.

[1291] "Image generation AI" is an artificial intelligence technology for generating character appearances.

[1292] "Text generation AI" is an artificial intelligence technology for generating a character's ego, that is, their personality and thoughts.

[1293] "Voice generation AI" is an artificial intelligence technology for generating character voices.

[1294] An "emotion engine" is a technology that recognizes the user's emotions and adjusts the character's actions and speech in response to those emotions.

[1295] "Emotion recognition and behavior adjustment means" is a function for detecting the user's emotions and changing the character's behavior and speech based on those emotions.

[1296] The "means for adjusting the character's actions and speech in accordance with the user's emotions" is a function for dynamically changing the character's responses and actions in accordance with the user's emotional state.

[1297] The following system configuration will be described as an embodiment of the present invention.

[1298] First, the server includes an image generation AI, a text generation AI, a voice generation AI, and an emotion engine. The image generation AI is used to generate the character's appearance. Specifically, the image generation AI generates the character's appearance based on the character's characteristics input by the user. The text generation AI is used to generate the character's ego, that is, its personality and thoughts. This allows the character to have a consistent personality. The voice generation AI is used to generate the character's voice. This allows the character to interact with the user in a natural voice.

[1299] The emotion engine is used to recognize the user's emotions and adjust the character's behavior and speech accordingly. Specifically, it analyzes emotions from the text or voice entered by the user and changes the character's response based on the results. For example, if the user enters "happy," the character will respond with "I'm so glad to see you happy!"

[1300] This system runs on devices such as smartphones and PCs. The device receives input from the user and sends it to a server. The server generates the character's appearance, personality, and voice based on the received input, and adjusts the character's responses using an emotion engine. The generated character's image and voice are sent to the device and displayed to the user.

[1301] As a concrete example, consider a scenario in which a user launches an application using a smartphone and interacts with a character. If the user inputs "I'm so happy today," the emotion engine recognizes the user's emotion as "happy." Based on this, the character responds with "I'm so glad to see you happy!", which is displayed along with the generated voice.

[1302] An example of a prompt is as follows:

[1303] Describe a friendly and supportive personality.

[1304] By inputting this prompt into a generative AI model, the character's personality and thoughts are generated.

[1305] The above is an embodiment of the present invention.

[1306] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1307] Step 1:

[1308] A user launches the application using a device such as a smartphone or PC. The user provides input (text or voice) to the device to interact with the character. Input can include text such as "I'm very happy today."

[1309] Step 2:

[1310] The terminal sends input from the user to the server. The input data is sent to the server in text format. The server analyzes the received input data and uses an emotion engine to recognize the user's emotion. The emotion engine analyzes emotions from the input text and outputs an emotion label such as "happy."

[1311] Step 3:

[1312] The server adjusts the character's behavior and speech based on the output of the emotion engine. Specifically, it inputs prompt sentences corresponding to emotion labels into a generative AI model to generate the character's responses. For example, the prompt sentence "Describe a friendly and supportive personality" is used to generate the character's personality and thoughts.

[1313] Step 4:

[1314] The server inputs the generated character's responses into the voice generation AI to generate the character's voice. The voice generation AI converts the text responses into audio data, allowing the character to respond in a natural voice.

[1315] Step 5:

[1316] The server inputs the generated character's appearance into the image generation AI to generate an image of the character. The image generation AI generates image data based on the character's characteristics and determines the character's appearance.

[1317] Step 6:

[1318] The server sends the generated character's image, voice, and response text to the terminal, which displays the received data to the user, and the character responds in a natural voice.

[1319] Step 7:

[1320] The user can continue to interact with the character on the device. Steps 1 to 6 are repeated depending on the user's new input, allowing the user to have a personalized experience based on their emotions.

[1321] Example 2

[1322] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1323] Traditional advertising methods have high filming costs and the risk of scandals. They also have the problem of being difficult to respond to users' emotions immediately, limiting real-time interaction. Furthermore, the lack of character customization and compatibility with various media limits the effectiveness of advertising.

[1324] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1325] In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a text generation AI, a voice generation means using a voice generation AI, a user emotion detection means using an emotion engine, a means for adjusting the character's actions and remarks in real time, and a means for posting the generated character to a social media platform. This reduces filming costs, avoids the risk of scandals, and enables immediate response to user emotions. It also improves character customization and compatibility with various media, maximizing advertising effectiveness.

[1326] "Image generation AI" is a technology that uses artificial intelligence to generate images.

[1327] "Appearance generation means" is a means of generating a character's appearance using image generation AI.

[1328] "Text generation AI" is a technology that uses artificial intelligence to generate text.

[1329] "Self-generation means" is a means of generating a character's personality and speech using text generation AI.

[1330] "Voice generation AI" is a technology that uses artificial intelligence to generate voice.

[1331] "Voice generation means" refers to a means of generating a character's voice using voice generation AI.

[1332] An "emotion engine" is a technology that detects and analyzes a user's emotions.

[1333] The "user emotion detection means" is a means for detecting the user's emotion in real time using an emotion engine.

[1334] "Means for adjusting the character's actions and statements in real time" refers to means for instantly changing the character's actions and statements in response to the user's emotions.

[1335] "SNS Platform" means an online platform that provides social networking services.

[1336] "Means for posting the generated character on a social media platform" refers to means for posting images and videos of the generated character on a social media platform.

[1337] MODE FOR CARRYING OUT THE INVENTION

[1338] This invention is a system that generates characters using a generative AI model and performs advertising and user support on a social networking platform. Specific embodiments of this system are described below.

[1339] Character Generation

[1340] The server generates characters using generative AI models (e.g., image generation AI, text generation AI, and voice generation AI). For image generation AI, it uses DALL-E, for example, and for text generation AI, it uses GPT-4. For voice generation AI, it uses Google Cloud Text-to-Speech, for example. By entering the prompt "Generate a character wearing the new jacket," the character's appearance, personality, and voice are generated.

[1341] Character action settings

[1342] The server assigns specific actions to the generated character. For example, it creates a scene in which the character is walking around town wearing a new jacket. It then uses image processing software (e.g., Adobe Photoshop) to adjust the character's pose and the background. Specifically, it creates a scene in which the character is walking down the sidewalk and looking at a smartphone.

[1343] Posting to social media

[1344] The server posts the generated images and videos to a social media platform (e.g., Instagram). The post includes a caption such as "New jacket is here! Check it out now!" Specifically, the user uploads the image to the Instagram posting screen, enters a caption, and clicks the "Post" button.

[1345] User Emotion Detection

[1346] The device uses an emotion engine (e.g., Microsoft Azure's Emotion API) to detect the user's emotions in real time. The emotion engine analyzes the user's facial expressions and voice, capturing them with a camera and microphone. Specifically, it detects the moment when the user frowns or their voice tone rises.

[1347] Character reaction adjustments

[1348] When a user expresses anger, the device notifies the server. The server then instantly adjusts the character's actions and speech, for example, making the character say, "Sorry, did I say something that offend you?" Specifically, it generates an audio file for the character and sends it to the user's device.

[1349] Examples of concrete examples and prompts

[1350] Example 1: Advertising apparel products

[1351] A scene is generated in which a character is wearing a new jacket and walking around town.

[1352] Post the generated image to Instagram with the caption, "New jacket here! Check it out now!"

[1353] Example 2: Responding to user emotions

[1354] If the user suddenly expresses anger, the emotion engine detects this and notifies the server.

[1355] The server has the character say, "Sorry, did I say something to offend you?"

[1356] Prompt Sentence Examples

[1357] "Generate an image of a character wearing the jacket of a new work."

[1358] "Generate a scenario where a character apologizes when the user expresses anger."

[1359] In this way, the server, terminal, and user work together to create a system that uses the generated characters to advertise and respond to users.

[1360] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1361] Step 1:

[1362] The server generates a character using a generative AI model. The prompt sentence "Generate a character wearing the new jacket" is used as input. The appearance, personality, and voice of the character are generated by the generative AI model (e.g., image generation AI, text generation AI, voice generation AI). The output is an image, text, and voice data of the generated character. Specifically, the server inputs the prompt sentence into the generative AI model and saves the generated data.

[1363] Step 2:

[1364] The server assigns specific actions to the generated character. As input, it uses the data of the generated character and instructions for the action (e.g., "scene of walking through town"). It uses image processing software (e.g., Adobe Photoshop) to adjust the character's pose and background. As output, it obtains images and videos of the character with the assigned action. Specifically, the server inputs the character's data into the image processing software and generates data with the assigned action.

[1365] Step 3:

[1366] The server posts the generated images and videos to the SNS platform. As input, it uses the image or video of the character with the action set and the post content (e.g., "New jacket is here! Check it out now!"). It posts using the SNS platform's API. As output, it obtains the content posted to the SNS platform. Specifically, the server inputs the image or video and post content into the SNS platform's API and executes the post.

[1367] Step 4:

[1368] The device detects the user's emotions in real time. It uses the user's facial expressions and voice data as input. It analyzes the user's emotions using an emotion engine (e.g., Microsoft Azure's Emotion API). The output is the user's emotional state. Specifically, the device captures the user's facial expressions and voice using a camera and microphone, inputs them into the emotion engine, and obtains the analysis results.

[1369] Step 5:

[1370] When the user expresses anger, the device notifies the server. The user's emotional state is used as input. The server immediately adjusts the character's actions and speech. The adjusted character actions and speech are obtained as output. Specifically, the server receives the user's emotional state, generates an audio file that makes the character say, "I'm sorry, did I say something that offend you?" and sends it to the user's device.

[1371] (Application example 2)

[1372] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1373] Conventional advertising systems have high filming costs and the risk of scandals. They also face the challenge of not being able to provide personalized ads that reflect users' emotions, making effective marketing impossible. Furthermore, while character-based advertising requires responses that reflect users' emotions in real time, there is a lack of technology to achieve this.

[1374] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1375] In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a sentence generation AI, a voice generation means using a voice generation AI, an emotion recognition means, an emotion-based advertisement content generation means, a voice output means, a camera image acquisition means, a face detection means, a face landmark detection means, an emotion prediction means, and an emotion-based character behavior determination means, thereby enabling advertisements to be provided in real time according to the user's emotions.

[1376] "Image generation AI" is a method of generating images using artificial intelligence technology.

[1377] "Appearance generation means" is a means for generating the appearance and shape of a character.

[1378] "Text generation AI" is a method of generating text using artificial intelligence technology.

[1379] "Self-generation means" is a means for generating a character's individuality and personality.

[1380] "Voice generation AI" is a method of generating voice using artificial intelligence technology.

[1381] "Voice generation means" is a means for generating the voice of a character.

[1382] The "emotion recognition means" is a means for recognizing the user's emotions.

[1383] The "means for generating advertisement content according to emotions" is a means for generating advertisement content based on the emotions of a user.

[1384] The "audio output means" is a means for outputting the generated audio.

[1385] The "camera image acquisition means" is a means for acquiring an image using a camera.

[1386] The "face detection means" is a means for detecting a face from the acquired image.

[1387] The "face landmark detection means" is a means for detecting feature points of a detected face.

[1388] The "emotion prediction means" is a means for predicting the user's emotion based on the detected facial feature points.

[1389] The "means for determining character behavior based on emotions" is a means for determining the behavior of a character based on predicted emotions.

[1390] The system for implementing this invention includes an image generation AI, a text generation AI, a voice generation AI, an emotion recognition means, an emotion-based advertising content generation means, a voice output means, a camera image acquisition means, a face detection means, a face landmark detection means, an emotion prediction means, and an emotion-based character behavior determination means.

[1391] System configuration

[1392] The server uses image generation AI to generate the character's appearance, text generation AI to generate the character's ego, and voice generation AI to generate the character's voice, allowing the character to converse naturally with the user.

[1393] The terminal acquires an image of the user using the camera image acquisition means, detects a face from the image using the face detection means, detects facial feature points using the face landmark detection means, and predicts the user's emotion using the emotion prediction means.

[1394] The server generates advertisement content using emotion-based advertisement content generation means based on the predicted emotion, and determines the character's behavior using emotion-based character behavior determination means, thereby enabling the character to provide advertisements that correspond to the user's emotions in real time.

[1395] Hardware and software used

[1396] Hardware: Smartphone camera, smartphone speaker

[1397] Software: OpenCV (camera image acquisition and face detection), dlib (facial landmark detection), Keras (emotion recognition model loading and prediction), text_to_speech (converting character speech into audio)

[1398] Data processing and calculation

[1399] 1. Camera image acquisition: Capture the user's image in real time using the device's camera.

[1400] 2. Face detection: Detect faces from captured video using OpenCV and dlib and identify facial landmarks.

[1401] 3. Emotion prediction: The detected facial landmark data is input into an emotion recognition model using Keras to predict the user's emotion.

[1402] 4. Advertising content generation: Based on the predicted emotions, advertising content is generated using an emotion-based advertising content generation means.

[1403] 5. Character behavior determination: Determine the character's behavior using emotion-based character behavior determination means.

[1404] 6. Audio output: Using the text_to_speech library, the character's speech is converted into audio and played through the device's speaker.

[1405] Specific examples

[1406] When the user opens their smartphone, the camera activates and a character cheerfully speaks to them, saying, "Hello! It's a wonderful day today. How do you like our new apparel?"

[1407] If the user looks sad, the character will gently say, "Cheer up! These delicious sweets are sure to cheer you up."

[1408] Prompt Sentence Examples

[1409] Generate what the character will say when the user is happy.

[1410] Generate what the character will say if the user is sad.

[1411] Generate what the character will say if the user is angry.

[1412] In this way, the system can deliver ads in real time that respond to the user's emotions, creating a more personalized experience.

[1413] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1414] Step 1:

[1415] The terminal uses a camera image capture means to capture the user's image in real time, where the input is the camera image and the output is the captured image data, which is used in subsequent processing steps.

[1416] Step 2:

[1417] The device detects faces from captured video data using a face detection mechanism. The input is the captured video data, and the output is the position information of the detected face. The device uses OpenCV to identify the position of the face and extract the face area.

[1418] Step 3:

[1419] The device identifies the detected facial feature points using a facial landmark detection means. The input is face position information, and the output is facial landmark data. 68 facial feature points are detected using dlib, and this data is acquired.

[1420] Step 4:

[1421] The device uses emotion prediction means to predict the user's emotion based on facial landmark data. The input is facial landmark data, and the output is a predicted emotion label. The landmark data is input into an emotion recognition model using Keras to predict emotions.

[1422] Step 5:

[1423] The server generates advertising content based on the predicted emotion using an emotion-based advertising content generation means. The input is the predicted emotion label, and the output is the generated advertising content. The advertising content corresponding to the emotion label is created using a generative AI model.

[1424] Step 6:

[1425] The server determines the character's behavior using emotion-based character behavior determination means. The input is the generated advertisement content, and the output is instructions for the character's behavior. The server determines the character's remarks and actions based on the advertisement content.

[1426] Step 7:

[1427] The device uses a voice generation AI to convert the character's speech into audio and plays it back using an audio output means. The input is the character's action instructions, and the output is audio data. The text_to_speech library is used to convert the speech into audio and play it back through the smartphone speaker.

[1428] Step 8:

[1429] The user listens to the voice played on the device and takes action based on what the character says. The input is the voice data, and the output is the user's reaction. The user's reaction is captured again in the next cycle, forming a feedback loop for the entire system.

[1430] Example 3

[1431] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1432] Conventional character generation systems had difficulty responding to user emotions, and were unable to fully utilize features such as character customization, global compatibility, continuous operation, and compatibility with a variety of media. Furthermore, they lacked the ability to learn and predict user emotional patterns, so more natural dialogue and an improved user experience were needed.

[1433] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1434] In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a sentence generation AI, a voice generation means using a voice generation AI, a user emotion pattern learning means using an emotion engine, a constant character operation means using a cloud service, a character activity means in various media, and a means for predicting and responding to changes in the user's emotions. This enables character customization, global compatibility, constant operation, and compatibility with various media, and also enables natural dialogue and responses according to the user's emotions.

[1435] "Image generation artificial intelligence" is an artificial intelligence technology that generates images based on user input.

[1436] "Sentence generation artificial intelligence" is an artificial intelligence technology that generates sentences based on user input.

[1437] "Speech generation artificial intelligence" is an artificial intelligence technology that generates speech based on user input.

[1438] An "emotion engine" is a technology that learns a user's emotional patterns and predicts changes in emotions.

[1439] "Cloud services" is a general term for computing resources and services provided via the Internet.

[1440] "Means for keeping characters running at all times" refers to the technology and infrastructure required to keep characters running 24 hours a day.

[1441] "Means for character activity in various media" refers to the technology and interfaces that allow characters to be active in various media such as television, radio, and the Internet.

[1442] The "means for predicting and responding to changes in the user's emotions" is a technology for predicting changes in the user's emotions and responding accordingly.

[1443] The invention begins when a user submits a character generation request using a device. The user inputs details such as the character's appearance, personality, and voice. The server generates the character based on the user's request using a generative AI model. Specifically, it uses an image generation AI (e.g., DALL-E) to generate the character's image, and a text generation AI (e.g., GPT-4) to generate the character's personality and voice characteristics. The generated character data is stored on the server, and a confirmation link is sent to the user.

[1444] The user opens a confirmation link on their device to view the generated character. If desired, the user can further customize the character's details, such as changing hair color or brightening the character's personality. The server updates these changes in real time and provides the updated character to the user again.

[1445] The server uses cloud services to build infrastructure to run the generated characters 24 hours a day, ensuring that the characters are always online and accessible from any region. For example, AWS's EC2 instances are used to provide the character's operating environment.

[1446] Users input their daily emotional data through their devices. For example, they might enter "I'm tired today" into a dedicated application. The server sends this data to the emotion engine, which learns the user's emotional patterns. The emotion engine analyzes the user's input data and updates the model to predict changes in emotions.

[1447] The server predicts changes in the user's emotions based on the learned emotional data. For example, if the user inputs "I'm tired today," the emotion engine will determine that the user needs to relax. Based on this prediction, the character will respond by playing relaxing music when the user gets home. Specifically, the character will say, "Thank you for your hard work. I'll play some relaxing music for you," and then actually play the music.

[1448] Examples of concrete examples and prompts

[1449] Specific examples

[1450] When a user types "I'm tired today" into the device, the server uses an emotion engine to analyze the user's emotions. The emotion engine determines that the user needs to relax and issues instructions to the character. The character says, "Thank you for your hard work. I'll play some relaxing music for you," and uses the API of a music streaming service to play relaxing music.

[1451] Prompt Sentence Examples

[1452] "Generate a prompt that instructs a character to play relaxing music when the user gets home from work."

[1453] In this way, the system operates in cooperation with the server, the terminal, and the user. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1454] Step 1:

[1455] The user uses a terminal to send a request for character generation. The user inputs details such as the character's appearance, personality, and voice. The input data is sent to the server. Specifically, the user inputs hairstyle, eye color, clothing, personality, and voice characteristics into a form in a dedicated application, and then presses the send button.

[1456] Step 2:

[1457] The server receives requests sent by users and generates characters using a generative AI model. Specifically, it uses image generation AI to generate character images, and text generation AI to generate the character's personality and voice characteristics. The generated character data is stored on the server. For example, DALL-E is used to generate character images, and GPT-4 is used to generate the character's personality and voice characteristics.

[1458] Step 3:

[1459] The server generates a confirmation link and sends it to the user to provide the generated character data to the user. The user opens the confirmation link on their device and confirms the generated character. Specifically, the server saves the generated character data and sends the user an access link to that data by email.

[1460] Step 4:

[1461] The user can review the generated character through their device and further customize it as needed. The user can then change the character's details and send them back to the server. The server then updates these changes in real time and provides the updated character back to the user. For example, the user can change the hair color or make the character a little brighter.

[1462] Step 5:

[1463] The server uses a cloud service to build infrastructure to run the generated characters 24 hours a day. This allows the characters to be always online and accessible from any region. Specifically, AWS EC2 instances are used to provide the character's operating environment.

[1464] Step 6:

[1465] Users input their daily emotional data through their devices. For example, they might enter "I'm tired today" into a dedicated application. The input emotional data is sent to the server. The server then sends this data to the emotion engine, which learns the user's emotional patterns. The emotion engine analyzes the user's input data and updates the model to predict changes in emotions.

[1466] Step 7:

[1467] The server predicts changes in the user's emotions based on the learned emotional data. For example, if a user inputs "I'm tired today," the emotion engine determines that the user needs to relax. Based on this prediction, the character responds by playing relaxing music when the user returns home. Specifically, the character says, "Thank you for your hard work. I'll play some relaxing music for you," and uses the API of a music streaming service to play relaxing music.

[1468] (Application example 3)

[1469] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1470] Conventional character generation systems lack the ability to predict user emotions and provide appropriate content, which leaves the user experience unsatisfied. Furthermore, there is a need for systems that offer features such as character customization, global compatibility, continuous operation, and compatibility with various media.

[1471] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a sentence generation AI, a voice generation means using a voice generation AI, an emotion prediction means, a content recommendation means, and a means for interacting with the user. This makes it possible to predict the user's emotions and provide appropriate content, improving the user experience. Furthermore, it is possible to realize a system with characteristics such as character customization, global compatibility, continuous operation, and compatibility with various media.

[1472] "Image generation AI" is a method of automatically generating a character's appearance using artificial intelligence technology.

[1473] "Text generation AI" is a method of automatically generating text that expresses a character's ego and personality using artificial intelligence technology.

[1474] "Voice generation AI" is a method of automatically generating character voices using artificial intelligence technology.

[1475] "Emotion prediction means" is a technology for predicting emotions from user input and behavior.

[1476] A "content recommendation means" is a technology for recommending appropriate content based on predicted user emotions.

[1477] "Means for interacting with the user" refers to technology that allows the character to have natural conversations with the user.

[1478] "Customizability" refers to the ability to change a character's appearance, personality, voice, etc. according to the user's wishes.

[1479] "Global compatibility" refers to the ability to access a website from anywhere in the world.

[1480] "Continuous operation" means that the system is operational 24 hours a day.

[1481] "Diverse media compatibility" refers to the ability to operate across multiple media, including television, radio, and the Internet.

[1482] "Online event appearance" is a feature that allows a character to participate in events held over the Internet.

[1483] "Unique story and worldview" refers to the unique background, story, and setting that each character has.

[1484] The system for implementing this invention includes an image generation AI, a text generation AI, a voice generation AI, an emotion prediction means, a content recommendation means, and a means for interacting with a user. Specific embodiments of each means will be described below.

[1485] 1. Appearance generation method using image generation AI

[1486] The server uses image generation AI to generate the character's appearance. This image generation AI can customize the character's appearance according to the user's request. For example, if the user requests an "anime-style character," the image generation AI will generate a character that meets that request.

[1487] 2. Self-generation using text generation AI

[1488] The server uses a text generation AI to generate sentences that express the character's ego and personality. This text generation AI can customize the character's personality and background story according to the user's requests. For example, if the user requests a "friendly personality," the text generation AI will generate dialogue that matches that personality.

[1489] 3. Voice generation using AI

[1490] The server uses a voice generation AI to generate the character's voice. This voice generation AI can customize the character's voice quality and speaking style according to the user's request. For example, if the user requests a "soft voice," the voice generation AI will generate a voice that meets that request.

[1491] 4. Emotion Prediction Methods

[1492] The server uses an emotion prediction means to predict emotions from the user's input and actions. The emotion prediction means analyzes the user's text input and voice input and estimates the user's emotional state. For example, if the user inputs "I'm tired today," the emotion prediction means predicts that the user is tired.

[1493] 5. Content Recommendation Methods

[1494] The server recommends appropriate content based on the user's emotion predicted by the emotion prediction means. This content recommendation means suggests relaxing music, videos, news, entertainment content, etc. according to the user's emotional state. For example, if the server predicts that the user is tired, it recommends a relaxing music playlist.

[1495] 6. Means of interaction with users

[1496] The server uses a user interaction means to allow the character to have a natural conversation with the user. This interaction means generates an appropriate response according to the user's input, allowing the character to interact with the user. For example, if the user inputs "I'm tired today," the character responds, "I recommend a relaxing music playlist."

[1497] Examples of concrete examples and prompts

[1498] For example, if a user types, "I feel tired today," the character will respond, "I recommend a relaxing music playlist." In this way, predicting the user's emotions and providing appropriate content improves the user experience.

[1499] Prompt Sentence Examples

[1500] User: I'm tired today.

[1501] Character: I recommend a relaxing music playlist.

[1502] This invention makes it possible to predict user emotions and provide appropriate content, improving the user experience. It also makes it possible to realize a system with features such as character customization, global compatibility, continuous operation, and compatibility with a variety of media.

[1503] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1504] Step 1:

[1505] The user accesses the character generation system using a terminal and inputs their character customization requests. The input data includes customization information such as the character's appearance, personality, and voice. The server receives this input data and passes it on to the image generation AI, text generation AI, and voice generation AI.

[1506] Step 2:

[1507] The server uses image generation AI to generate a character's appearance based on the user's requests. The server receives the user's customization requests as input data, and the image generation AI generates the character's appearance based on those requests. Image data of the generated character is output.

[1508] Step 3:

[1509] The server uses a text generation AI to generate text that expresses the character's ego and personality. It receives the user's customization requests as input data, and the text generation AI generates the character's personality and background story based on those requests. The generated text data is output.

[1510] Step 4:

[1511] The server uses a voice generation AI to generate the character's voice. It receives the user's customization requests as input data, and the AI ​​generates the character's voice quality and speaking style based on those requests. The generated voice data is output.

[1512] Step 5:

[1513] The user starts a conversation with the character using the terminal. The server receives the user's input (text or voice) and passes it to the emotion prediction means. The content of the user's conversation is used as input data.

[1514] Step 6:

[1515] The server predicts the user's emotions using emotion prediction means. The server receives the user's dialogue content as input data, analyzes the content, and estimates the user's emotional state. The server outputs predicted emotion data.

[1516] Step 7:

[1517] The server recommends appropriate content based on the user's emotion predicted using the content recommendation means. The server receives the predicted emotion data as input data, and the content recommendation means selects content according to the emotion. The recommended content data is output.

[1518] Step 8:

[1519] The server uses a means for interacting with the user to have the character present recommended content to the user. The server receives the recommended content data as input data, and the interactive means conveys the content to the user. The user receives a response from the character through the terminal.

[1520] Step 9:

[1521] The user confirms the recommended content and initiates further dialogue if necessary. The server receives the user's input again and repeats the process from step 5.

[1522] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1523] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1524] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1525] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1526] [Third embodiment]

[1527] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1528] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1529] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1530] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1531] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1532] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1533] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1534] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1535] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1536] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1537] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1538] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1539] "Example 1"

[1540] One aspect of the present invention is a system that uses image generation AI, text generation AI, and voice generation AI to generate characters that can be used on social media. The image generation AI generates the character's appearance. The text generation AI generates the character's ego, that is, its personality and thoughts. The voice generation AI generates the character's voice. These AIs generate characters with consistent character settings that can be used on social media.

[1541] "Example 2"

[1542] In one embodiment of the present invention, the generated characters are used to advertise apparel, food, furniture, and home appliances. Specifically, the products are promoted by posting on social media a scene of the character wearing a particular apparel item or eating a particular food. This reduces filming costs and avoids the risk of scandals.

[1543] "Example 3"

[1544] In one embodiment of the present invention, the generated character has characteristics such as customizability, global compatibility, continuous operation, and compatibility with various media, and can appear in online events and have its own unique story and worldview. Specifically, the character's appearance, personality, voice, etc. can be customized according to the user's requests. Furthermore, the character can be operated 24 hours a day and can be accessed from anywhere in the world. Furthermore, the character can be active in various media such as television, radio, and the Internet.

[1545] The processing flow of each embodiment will be described below.

[1546] "Example 1"

[1547] Step 1: Generate the character's appearance using image generation AI. Specifically, set the character's features such as hair color, eye color, and body type.

[1548] Step 2: The text generation AI generates the character's ego, specifically setting the character's personality, hobbies, likes, dislikes, and other characteristics.

[1549] Step 3: The voice generation AI generates the character's voice. Specifically, the pitch, tone, speaking style, and other characteristics of the character's voice are set.

[1550] "Example 2"

[1551] Step 1: Create an advertisement for an apparel product using the generated character. Specifically, generate an image of the character wearing a specific apparel product.

[1552] Step 2: Post the generated advertising image to social media. Specifically, post an image of a character wearing the product along with detailed product information and how to purchase it.

[1553] "Example 3"

[1554] Step 1: Customize the generated character's appearance, personality, voice, etc. according to the user's requests. Specifically, change the character's hair color, personality, etc., based on the user's requests.

[1555] Step 2: The customized character will appear in online events. Specifically, the character will participate in online events, interact with users, promote products, etc.

[1556] Example 1

[1557] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1558] With conventional character generation systems, it was difficult to consistently generate a character's appearance, personality, and voice, making it impossible to efficiently create characters suitable for social media management. Furthermore, in the case of advertisements and online events, the high cost of filming and the risk of scandals meant that customization and global support were required.

[1559] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1560] In this invention, the server includes a means for receiving a prompt text entered by a user, a means for generating a character's appearance using an image generation AI based on the prompt text, a means for generating a character's ego using a sentence generation AI based on the prompt text, a means for generating a character's voice using a voice generation AI based on the prompt text, a means for integrating the generated appearance, personality, and voice to create a consistent character profile, and a means for providing the character profile to the user. This allows users to efficiently generate characters with consistent profile settings suitable for use on social media, reducing filming costs and avoiding the risk of scandals in advertisements and online events. Furthermore, characters with characteristics such as customizability, global compatibility, continuous operation, and compatibility with various media can be generated.

[1561] A "prompt sentence" is text containing instructions or requests that a user inputs to a generative AI model.

[1562] "Image generation AI" is an artificial intelligence model that generates images based on input prompt text.

[1563] "Sentence generation AI" is an artificial intelligence model that generates sentences based on input prompts.

[1564] "Speech generation AI" is an artificial intelligence model that generates speech based on an input prompt.

[1565] "Character appearance" refers to a character's appearance and physical characteristics.

[1566] "Character ego" refers to a character's personality, thoughts, and behavior patterns.

[1567] "Character voice" refers to the sounds and speaking characteristics of a character.

[1568] "Consistent character settings" refers to the overall image of a character with consistent characteristics in appearance, personality, and voice.

[1569] "SNS-operable character" refers to a character that can be active on a social networking service.

[1570] "Reducing filming costs" refers to reducing the costs of actual filming by using characters.

[1571] "Scandal risk avoidance" refers to the avoidance of the risk of scandals related to real people by using characters.

[1572] "Customizability" refers to the ability to change a character's characteristics according to the user's needs.

[1573] "Global compatibility" refers to the characteristics of a character that can adapt to different languages ​​and cultures.

[1574] "Always-on" refers to the ability of a character to be active 24 hours a day, 365 days a year.

[1575] "Multimedia compatible" refers to the ability of a character to operate in different media formats (e.g., images, video, audio).

[1576] "Appearing at an online event" refers to a character's participation in an event held via the Internet.

[1577] "Unique story and worldview" refers to the unique background, story, and setting that each character has.

[1578] This invention is a system that generates a character with consistent character settings that can be used in SNS management using image generation AI, text generation AI, and voice generation AI based on a prompt sentence entered by a user. A specific embodiment of this system is described below.

[1579] Hardware and software used

[1580] The server uses the following hardware and software:

[1581] Hardware: High-performance server computer

[1582] Software: Image generation AI (e.g., DALL-E, Stable Diffusion), text generation AI (e.g., GPT-4), speech generation AI (e.g., Tacotron 2, WaveNet)

[1583] Data processing and calculation

[1584] The server receives the prompt text entered by the user from the terminal and passes it to each generative AI model in an appropriate format. Specifically, it receives the following prompt text:

[1585] Appearance: "Young female character, blue hair, casual clothing"

[1586] Personality: "This character is curious and loves adventure."

[1587] Voice: "This character's voice has a bright, cheerful tone."

[1588] The server analyzes the received prompt sentence and performs the following process.

[1589] Appearance generation using image generation AI

[1590] The server passes a prompt about appearance to the image generation AI. The image generation AI generates an image of a character with the specified characteristics and returns it to the server. For example, based on the prompt "young female character, blue hair, casual clothing," it generates an image of the corresponding character.

[1591] Personality generation using sentence generation AI

[1592] The server passes a prompt about the character's personality to the sentence generation AI. The sentence generation AI generates a sentence about the character's personality and thoughts with the specified characteristics and returns it to the server. For example, based on the prompt "This character is curious and loves adventure," it generates a sentence that describes the character's personality in detail.

[1593] Voice generation using AI

[1594] The server passes a voice prompt to the voice generation AI. The voice generation AI generates a character's voice with the specified characteristics and returns it to the server. For example, based on the prompt, "This character's voice has a bright, cheerful tone," it generates a voice file for the corresponding character.

[1595] Character Settings Integration

[1596] The server integrates the generated images, text, and audio to create a consistent character profile, which is then saved in a format suitable for SNS management.

[1597] Character provision

[1598] The server sends the generated character settings to the device. The user can then check the character through the device and start using it on the SNS. For example, the user can post on the SNS using the generated character.

[1599] In this way, users can efficiently create characters with consistent character settings that can be used on social media, reducing filming costs and avoiding the risk of scandals when it comes to advertising and online events. Characters can also be created with features such as customizability, global compatibility, continuous operation, and compatibility with various media.

[1600] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1601] Step 1:

[1602] The user inputs a prompt into the input form on the terminal. For example, the following prompt is input:

[1603] Appearance: "Young female character, blue hair, casual clothing"

[1604] Personality: "This character is curious and loves adventure."

[1605] Voice: "This character's voice has a bright, cheerful tone."

[1606] The entered prompt sentence is sent from the terminal to the server.

[1607] Step 2:

[1608] The server analyzes the prompts received from the device. The analyzed prompts are passed to each generation AI model in an appropriate format. For example, prompts about appearance are passed to the image generation AI, prompts about personality are passed to the sentence generation AI, and prompts about voice are passed to the voice generation AI.

[1609] Step 3:

[1610] The server passes a prompt about appearance to the image generation AI. The image generation AI generates an image of a character with the specified characteristics. For example, based on the prompt "young female character, blue hair, casual clothing," it generates an image of the corresponding character. The generated image is returned to the server.

[1611] Step 4:

[1612] The server passes a prompt about the character's personality to the sentence generation AI. The sentence generation AI generates a sentence about the character's personality and thoughts with the specified characteristics. For example, based on the prompt "This character is curious and loves adventure," it generates a sentence that describes the character's personality in detail. The generated sentence is returned to the server.

[1613] Step 5:

[1614] The server passes a voice prompt to the voice generation AI. The voice generation AI generates a character's voice with the specified characteristics. For example, based on the prompt, "This character's voice has a bright, cheerful tone," it generates a voice file for the corresponding character. The generated voice file is returned to the server.

[1615] Step 6:

[1616] The server integrates the generated images, text, and audio to create a consistent character profile. For example, the server combines the generated images, text about the character's personality, and audio files to create a unified character profile. This character profile is stored in a format suitable for SNS management.

[1617] Step 7:

[1618] The server sends the generated character settings to the device. The user can then check the character through the device and start using it on the SNS. For example, the user can post on the SNS using the generated character.

[1619] (Application example 1)

[1620] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1621] With conventional character generation systems, it was difficult to generate virtual influencers with consistent character settings for social media management and content distribution services. Furthermore, the cost and time required to individually set the character's appearance, personality, and voice made efficient management difficult. Furthermore, filming costs and the risk of scandals were also issues for advertising and online events.

[1622] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1623] In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a text generation AI, a voice generation means using a voice generation AI, and a means for managing a virtual influencer in a content distribution service using the generated character. This allows for efficient generation of virtual influencers with consistent character settings and their use in social media management and content distribution services. Furthermore, it also enables reductions in filming costs and avoidance of scandal risks in advertising and online events.

[1624] "Image generation AI" is an artificial intelligence technology that generates images based on input text or prompts.

[1625] "Sentence generation AI" is an artificial intelligence technology that generates natural-sounding sentences based on input text and prompts.

[1626] "Speech generation AI" is an artificial intelligence technology that generates speech based on input text.

[1627] "Appearance generation means" refers to a means of generating a character's appearance or figure using image generation AI.

[1628] "Self-generation means" is a means of generating a character's personality and thoughts using text generation AI.

[1629] "Voice generation means" refers to a means of generating a character's voice using voice generation AI.

[1630] A "virtual influencer" is a virtual influencer who uses a generated character to be active on social media and content distribution services.

[1631] A "content distribution service" is a service that distributes content such as video, audio, and text via the Internet.

[1632] "Filming costs" are the expenses required to shoot characters and advertising materials.

[1633] "Scandal risk" refers to the risk of scandal or problematic behavior that a character or influencer may cause.

[1634] An "online event" is an event held over the internet.

[1635] "Customizability" refers to the ability of users to freely change a character's appearance, personality, voice, etc.

[1636] "Global compatibility" refers to the ability of characters and systems to support multiple languages ​​and cultures.

[1637] "Always-on operation" refers to the characteristic of characters and systems operating 24 hours a day, 365 days a year.

[1638] "Multimedia compatibility" refers to the ability of characters and systems to support multiple media formats such as video, audio, and text.

[1639] A "unique story" is a character's unique background or story.

[1640] A "worldview" is the virtual environment or setting in which a character operates.

[1641] The following system configuration and processing procedure will be described as an embodiment of the present invention.

[1642] System Configuration

[1643] This system uses the following hardware and software:

[1644] Hardware: Servers, smartphones, PCs

[1645] Software: Python, OpenAI API, PIL (Python Imaging Library), pyttsx3 (Python Text-to-Speech)

[1646] Processing steps

[1647] The server includes the following means:

[1648] 1. Image generation method: Using image generation AI, generate the character's appearance based on the input prompt.

[1649] 2. Self-generation method: Using text generation AI, generate a character's personality and thoughts based on the input prompts.

[1650] 3. Voice generation method: Using voice generation AI, a character's voice is generated based on the input text.

[1651] 4. Virtual influencer management means: Using the generated characters, virtual influencers are managed in content distribution services.

[1652] Data processing and calculation

[1653] The server processes and calculates the data as follows.

[1654] 1. Image Generation: The server uses the OpenAI API to generate an image of the character based on the prompt, and then converts the image to the appropriate format using PIL.

[1655] 2. Sentence Generation: The server uses the OpenAI API to generate the character's personality and thoughts based on the prompts. The generated sentences are saved as the character's ego.

[1656] 3. Voice generation: The server uses pyttsx3 to convert the generated text into an audio file, which generates the character's voice.

[1657] 4. Virtual Influencer Management: The server uses the generated character to automate activities on social media and content distribution services, including scheduling posts and interacting with fans.

[1658] Specific examples

[1659] Below are some examples of prompt sentences.

[1660] Image prompt: "A friendly and charismatic virtual influencer"

[1661] Writing prompt: "Describe the personality and daily activities of a virtual influencer who loves fashion and technology."

[1662] Based on this prompt, the server generates the character's appearance, personality, and voice, and allows the character to act as a virtual influencer on social media and content distribution services.

[1663] In this way, this invention makes it possible to efficiently generate virtual influencers with consistent character settings and utilize them in social media management and content distribution services. Furthermore, it also makes it possible to reduce filming costs and avoid the risk of scandals in advertising and online events.

[1664] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1665] Step 1:

[1666] The user inputs prompts about the character's appearance, personality, and voice using a smartphone or PC, which are then sent to the server.

[1667] Input: Prompts about the character's appearance, personality, and voice

[1668] Output: The prompt sent to the server

[1669] Step 2:

[1670] The server uses image generation AI to generate the character's appearance based on the input prompt, and the generated image is converted to the appropriate format using PIL.

[1671] Input: Prompt for character appearance

[1672] Output: An image of the generated character

[1673] Step 3:

[1674] The server uses a sentence generation AI to generate the character's personality and thoughts based on the input prompt sentence, and the generated sentence is saved as the character's ego.

[1675] Input: Character personality prompt

[1676] Output: Text about the generated character's personality and thoughts

[1677] Step 4:

[1678] The server uses a speech generation AI to convert the generated text into an audio file, which then generates the character's voice.

[1679] Input: Text about the character's personality and thoughts

[1680] Output: The generated character's voice

[1681] Step 5:

[1682] The server uses the generated character to automate activities on social media and content distribution services, including scheduling posts and interacting with fans.

[1683] Input: Image of the generated character, text about its personality and thoughts, and voice

[1684] Output: Character activities on social media and content distribution services

[1685] In this way, users can easily create virtual influencers and have them active on social media and content distribution services.

[1686] Example 2

[1687] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1688] Traditional advertising methods require the use of real models and locations, which not only increases filming costs but also poses the risk of scandal. While there is a demand for systems that offer features such as character customization, global compatibility, continuous operation, and compatibility with a variety of media, there is a lack of efficient methods to achieve this.

[1689] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1690] In this invention, the server includes an image generation means, a sentence generation means, a voice generation means, a means for inputting a prompt sentence, a means for analyzing the prompt sentence, a means for converting the generated data into a format for posting to an SNS, and a means for posting the formatted data to the SNS. This reduces filming costs, avoids the risk of scandals, and enables character customization, global compatibility, continuous operation, and compatibility with a variety of media.

[1691] "Image generation means" refers to a technique for generating an image based on a prompt sentence.

[1692] A "sentence generator" is a technique for generating text based on a prompt sentence.

[1693] The "voice generation means" is a technique for generating voice based on a prompt sentence.

[1694] The "means for inputting a prompt sentence" is an interface for the user to input the content they wish to generate in text format.

[1695] "Means for analyzing prompt sentences" refers to technology for analyzing input prompt sentences and converting them into a format suitable for a generative AI model.

[1696] "Means for converting generated data into a format for posting on SNS" refers to technology for converting generated images and text into a format suitable for SNS.

[1697] "Means for posting formatted data to a social networking site" refers to a technology for posting formatted data to a specified social networking site account.

[1698] This invention relates to a system for generating characters using a generative AI model and posting the characters to a social networking service. Specific embodiments of this system are described below.

[1699] System configuration

[1700] Hardware

[1701] Server: Runs the generative AI model, analyzes prompts, and converts data formats.

[1702] Terminal: A device on which a user enters a prompt, such as a computer, smartphone, or tablet.

[1703] software

[1704] Generative AI model: An AI model that generates images, text, and speech. Specifically, OpenAI's GPT-4 and DALL-E are used.

[1705] Social media posting formatter: Software that converts the generated data into a format suitable for social media.

[1706] Program processing

[1707] Entering a prompt statement

[1708] Using a dedicated application on a device or a web interface, a user enters a prompt that describes the scene they want to create for the character, for example, "Create a scene where the character is wearing a red dress."

[1709] Sending a prompt

[1710] The terminal sends the prompt text entered by the user to the server, where it is converted into an appropriate format and sent to the server.

[1711] Parsing the prompt statement

[1712] The server analyzes the received prompt and converts it into a format suitable for the generative AI model, for example, extracting keywords such as "character wearing a red dress."

[1713] Requests to generative AI models

[1714] The server sends a request to the generative AI model based on the analysis results, which includes the content of the prompt sentence.

[1715] Character image generation

[1716] The generative AI model receives requests from the server and generates a character image based on the specified scene, for example, an image of a character wearing a red dress.

[1717] Receiving the generated image

[1718] The server receives the character image returned by the generative AI model, which is generated based on the prompt text.

[1719] Image format conversion

[1720] The server then converts the received image into a format suitable for posting to the social networking site, for example by adjusting the size of the image and adding a caption.

[1721] Posting to social media

[1722] The server posts the formatted image to a specified social media account, which then publishes a scene of the generated character wearing a specific apparel item or eating a specific food item on the social media.

[1723] Specific examples

[1724] For example, if the user enters the prompt "Generate a scene where the character is wearing a red dress," the following specific actions will occur:

[1725] 1. The user types into the device application, "Generate a scene where the character is wearing a red dress."

[1726] 2. The terminal sends this prompt to the server.

[1727] 3. The server parses the prompt and extracts the keyword "character wearing a red dress."

[1728] 4. The server sends a request to the generation AI model saying, "Generate a character wearing a red dress."

[1729] 5. The generative AI model generates an image of a character wearing a red dress.

[1730] 6. The server receives the generated image and adjusts the size of the image for posting on social media.

[1731] 7. The server adds the caption "Check out our new red dress!" to the adjusted image.

[1732] 8. The server posts this image to the company's social media accounts.

[1733] This system allows companies to advertise efficiently, reduce filming costs, and avoid the risk of scandals.

[1734] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1735] Step 1:

[1736] The user enters a prompt statement.

[1737] Using a dedicated application or web interface on the device, the user inputs a prompt that describes the scene they want to create for the character. For example, they might input "Create a scene where the character is wearing a red dress." The input prompt is then stored in the device's memory.

[1738] Step 2:

[1739] The terminal sends the prompt to the server.

[1740] The terminal sends the prompt text entered by the user to the server. At this time, the prompt text is converted into an appropriate format and sent to the server over the network. The input is the prompt text, and the output is the prompt text sent to the server.

[1741] Step 3:

[1742] The server parses the prompt.

[1743] The server parses the received prompt and converts it into a format suitable for the generative AI model. For example, it extracts keywords such as "character wearing a red dress." The input is the prompt, and the output is the parsed keywords.

[1744] Step 4:

[1745] The server sends a request to the generative AI model.

[1746] The server sends a request to the generative AI model based on the analysis results. This request includes the contents of the prompt sentence. The input is the analyzed keyword, and the output is the request to the generative AI model.

[1747] Step 5:

[1748] A generative AI model generates character images.

[1749] The generative AI model receives a request from the server and generates a character image based on the specified scene. For example, generate an image of a character wearing a red dress. The input is the request to the generative AI model, and the output is the generated character image.

[1750] Step 6:

[1751] The server receives the generated image.

[1752] The server receives the character image returned by the generative AI model. This image is generated based on the prompt. The input is the generated character image, and the output is the image stored on the server.

[1753] Step 7:

[1754] The server formats the image for posting to social media.

[1755] The server converts the received image into a format suitable for posting to a social networking site, for example by adjusting the size of the image or adding a caption. The input is the generated character image, and the output is the formatted image.

[1756] Step 8:

[1757] The server posts the formatted image to a social networking site.

[1758] The server posts the formatted image to the specified social media account. This allows a scene of the generated character wearing a specific apparel item or eating a specific food to be published on the social media. The input is the formatted image, and the output is the image posted to the social media.

[1759] (Application example 2)

[1760] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1761] Traditional advertising production involves high filming costs and the risk of scandals. Furthermore, character-based advertising requires character customization and compatibility with a variety of media, but the technology required to achieve this is lacking. Furthermore, there are limited ways to automate posting to social media, making efficient advertising management difficult.

[1762] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a sentence generation AI, a voice generation means using a voice generation AI, and a means for automatically creating a scene in which the generated character uses a specific product and posting it to a social networking site. This enables reduction in photography costs, avoidance of the risk of scandals, character customization, compatibility with various media, and automatic posting to a social networking site.

[1763] "Image generation AI" is a system that generates images using artificial intelligence technology.

[1764] "Appearance generation means" is a function that generates a character's appearance using image generation AI.

[1765] "Text generation AI" is a system that generates text using artificial intelligence technology.

[1766] "Self-generation means" is a function that uses text generation AI to generate character personalities and background stories.

[1767] "Voice generation AI" is a system that generates voice using artificial intelligence technology.

[1768] "Voice generation means" is a function that generates a character's voice using voice generation AI.

[1769] The "means for automatically creating a scene in which a specific product is used" is a function for automatically creating a scene in which a generated character uses a specific product.

[1770] "Means for posting to SNS" is a function that automatically posts the generated scene to SNS.

[1771] "Apparel" refers to clothing and fashion items.

[1772] "Food" means food and drink intended for human consumption.

[1773] "Furniture and home appliances" refers to furniture and electrical appliances used in the home.

[1774] "Photography costs" refer to the costs incurred when taking photos or videos for advertising or promotional purposes.

[1775] "Scandal risk" refers to the risk that people or companies involved in advertising may become involved in a scandal.

[1776] "Customizability" refers to the ability for users to freely change the appearance and personality of their characters.

[1777] "Global compatibility" refers to the ability to adapt to different languages ​​and cultures.

[1778] "Continuous operation" refers to the system's ability to operate 24 hours a day, 365 days a year.

[1779] "Multimedia compatibility" refers to the ability of a character to be compatible with multiple media formats (images, videos, audio, etc.).

[1780] "Appearing at an online event" refers to the generated character participating in an event held via the Internet.

[1781] "Unique story and worldview" refers to the unique background story and setting of each character.

[1782] This invention is a system that generates a character using a generative AI model, automatically creates a scene of the character using a specific product, and posts the scene to a social networking site. Specific embodiments of this system are described below.

[1783] System configuration

[1784] The system consists of the following main components:

[1785] 1. Image generation AI: This is an artificial intelligence technology for generating character appearances. Specifically, it uses image generation models such as OpenAI's DALL-E.

[1786] 2. Text generation AI: This is an artificial intelligence technology for generating character personalities and background stories. Specifically, it uses text generation models such as OpenAI's GPT-4.

[1787] 3. Voice Generation AI: This is an artificial intelligence technology for generating character voices. Specifically, it uses voice generation models such as Google's WaveNet.

[1788] 4. SNS posting module: This module automatically posts scenes of generated characters using specific products to SNS.

[1789] System Operation

[1790] 1. Character Generation:

[1791] The server generates the character's appearance using image generation AI.

[1792] Next, a text generation AI is used to generate the character's ego and background story.

[1793] Finally, a voice generation AI is used to generate the character's voice.

[1794] 2. Create a scene:

[1795] The server automatically creates a scene in which a generated character uses a specific product, with the user providing a prompt such as:

[1796] Apparel: "Generate an image of a character wearing a new t-shirt."

[1797] Food: "Generate an image of a character eating a new snack."

[1798] Furniture Appliances: "Generate an image of a character sitting on a new sofa."

[1799] 3. Posting to social media:

[1800] The SNS posting module automatically posts the generated scenes to SNS, saving users the trouble of manually posting.

[1801] Hardware and software used

[1802] Hardware: The server uses a computer equipped with a high-performance GPU, which enables high-speed processing of generative AI models.

[1803] Software: OpenAI's DALL-E, GPT-4, and Google's WaveNet are used as generative AI models. The SNS posting module also uses the APIs of each SNS.

[1804] Specific examples

[1805] For example, if a clothing brand wants to advertise a new t-shirt, the user might enter the following prompt text:

[1806] Generate an image of the character wearing the new T-shirt.

[1807] Based on this prompt, the server uses image generation AI to generate a character image, text generation AI to create a background story for the character, and voice generation AI to generate a voice for the character. It then automatically creates a scene of the generated character wearing the new T-shirt and posts it to social media via the social media posting module.

[1808] In this way, it is possible to reduce filming costs, avoid the risk of scandals, customize characters, support a variety of media, and automatically post to social media.

[1809] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1810] Step 1:

[1811] The user uses the terminal to input a prompt, for example, "Please generate an image of a character wearing the new T-shirt." This prompt is then sent to the server.

[1812] Step 2:

[1813] The server analyzes the received prompt text and inputs it into the image generation AI. At this time, data processing is performed to generate the character's appearance based on the prompt text. Specifically, the character image is generated using OpenAI's DALL-E model. The input is the prompt text, and the output is an image of the generated character.

[1814] Step 3:

[1815] The server saves the generated character image and then inputs the prompt text into the text generation AI. During this process, data processing is performed to generate the character's ego and background story. Specifically, the character's background story is generated using OpenAI's GPT-4 model. The input is the prompt text, and the output is the generated background story.

[1816] Step 4:

[1817] The server saves the generated background story and then inputs the prompt text into the voice generation AI. At this time, data processing is performed to generate the character's voice. Specifically, the character's voice is generated using Google's WaveNet model. The input is the prompt text, and the output is the generated voice data.

[1818] Step 5:

[1819] The server integrates the generated character's image, background story, and voice data to automatically create a scene using a specific product. The scene is set based on the prompt entered by the user. The input is the generated character data, and the output is the scene data using a specific product.

[1820] Step 6:

[1821] The server inputs the generated scene data into the SNS posting module and automatically posts it to the SNS. The posting is done using the API of each SNS. The input is the generated scene data, and the output is the result of posting to the SNS.

[1822] Step 7:

[1823] Users can use their devices to check scenes posted to SNS. This saves users the trouble of manually posting. The input is the SNS posting result, and the output is the user's confirmation result.

[1824] Example 3

[1825] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1826] Conventional character generation systems lacked sufficient character customization and compatibility with diverse media, making it difficult to meet the diverse needs of users. Furthermore, it was difficult for users to reflect their specific requests when generating characters, and the generated characters often did not meet users' expectations. Furthermore, there were limitations on the character's operating hours and access regions, making global use difficult.

[1827] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1828] In this invention, the server includes a means for a user to input a prompt sentence, a means for receiving and analyzing the prompt sentence, a means for generating a character profile using a generative artificial intelligence model, a means for generating a voice for the character using speech synthesis software, and a means for providing the generated character to the user, thereby enabling character customization, global compatibility, continuous operation, and compatibility with a variety of media.

[1829] "Image generation artificial intelligence" is an artificial intelligence technology that generates images based on input data.

[1830] "Text generation artificial intelligence" is an artificial intelligence technology that generates text based on input data.

[1831] "Speech generation artificial intelligence" is an artificial intelligence technology that generates speech based on input data.

[1832] A "prompt sentence" is a sentence that describes instructions or requests that a user inputs to a system.

[1833] A "server" is a computer system that processes data and provides services over a network.

[1834] "Means of analysis" refers to the techniques and methods used to understand the input data and extract the necessary information.

[1835] A "generative artificial intelligence model" is a model for generating data using artificial intelligence technology.

[1836] "Speech synthesis software" is software for generating speech based on text data.

[1837] A "character profile" is information that describes a character's detailed settings and characteristics.

[1838] "Customizability" refers to the ability to change and adjust systems and characters according to the user's needs.

[1839] "Global compatibility" refers to the ability to access and use the system from anywhere in the world.

[1840] "Continuous operation" refers to the ability of a system to operate continuously 24 hours a day, 365 days a year.

[1841] "Multimedia compatibility" refers to the ability of a system or character to operate across multiple media, such as television, radio, and the Internet.

[1842] This invention is a system in which a user inputs a prompt sentence, and a server analyzes the prompt sentence and generates a character. A specific embodiment of this system will be described below.

[1843] First, the user inputs a prompt through the system interface, which includes detailed requests for the character's appearance, personality, voice, etc. For example, the following prompt can be input:

[1844] "Please create a character to appear at the online event. The character should be a woman in her 20s with a bright and lively personality, long blonde hair, and blue eyes. Please use a high-pitched, friendly voice."

[1845] Next, the server receives the prompt text sent by the user. The received prompt text is passed to an analysis module within the server. The server uses natural language processing technology to analyze the prompt text and extract each element of the character (appearance, personality, voice, etc.). Specifically, it uses a natural language processing library (e.g., spaCy).

[1846] Based on the analysis results, the server uses a generative AI model (e.g., OpenAI GPT-4) to generate a text-based profile for the character, which includes detailed settings for the character. The generated profile details the character's appearance, personality, background story, etc.

[1847] The server then uses speech synthesis software (e.g., Google Text-to-Speech) to generate a character voice based on the generated text-based profile, reflecting the tone and characteristics specified in the prompt.

[1848] Finally, the server provides the generated character's profile and voice to the user, who can then download the character and use it in online events and other media. This system enables character customization, global compatibility, continuous operation, and compatibility with a variety of media.

[1849] It is recommended to use a server equipped with a high-performance GPU (e.g., NVIDIA Tesla V100) to implement this system, as this will enable fast and efficient processing of the generative AI model and speech synthesis software.

[1850] As described above, the present invention provides a specific means for generating a customizable character in response to various user needs. The flow of the specification process in the third embodiment will be described with reference to FIG.

[1851] Step 1:

[1852] The user enters a prompt statement

[1853] The user inputs a prompt statement describing the character's requirements through the system interface. For example, the user can input a prompt statement such as, "Please create a character to appear in an online event. The character should be a woman in her twenties with a bright and lively personality, long blonde hair, and blue eyes. The voice should be high-pitched and friendly." The input data is text data containing detailed requirements such as the character's appearance, personality, and voice.

[1854] Step 2:

[1855] The server receives the prompt.

[1856] The server receives the prompt text sent by the user as an HTTP request. The received prompt text is passed to the analysis module in the server. The input data is the prompt text entered by the user, and the output data is the text data passed to the analysis module.

[1857] Step 3:

[1858] The server parses the prompt.

[1859] The server analyzes the prompt using natural language processing technology. Specifically, it uses a natural language processing library (e.g., spaCy) to understand the content of the prompt and extract each element of the character (appearance, personality, voice, etc.). The input data is the text data of the prompt, and the output data is data containing each element of the character as the analysis result.

[1860] Step 4:

[1861] The server generates a character profile using a generative AI model.

[1862] The server uses a generative AI model (e.g., OpenAI GPT-4) based on the analysis results to generate a text-based profile for the character. The generated profile contains detailed settings for the character. The input data is data containing each element of the character as a result of the analysis, and the output data is the generated character profile.

[1863] Step 5:

[1864] The server generates the character's voice using speech synthesis software.

[1865] The server generates a character voice using speech synthesis software (e.g., Google Text-to-Speech) based on the generated text-based profile. The generated voice reflects the tone and characteristics specified in the prompt. The input data is the generated character profile, and the output data is the generated character voice.

[1866] Step 6:

[1867] The server provides the generated character to the user.

[1868] The server provides the generated character's profile and voice to the user. The user can download the provided character and use it in online events or other media. The input data is the generated character's profile and voice, and the output data is the character data provided to the user.

[1869] (Application example 3)

[1870] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1871] Conventional character generation systems have limited user customization and interactive dialogue capabilities, and are not yet fully compatible with global platforms, 24 / 7 operation, or diverse media. Furthermore, there is a lack of systems that allow users to interact with characters in real time and advance the story. This limits the ability to improve the user experience and apply them to a variety of purposes.

[1872] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1873] In this invention, the server includes an appearance generation means using an image generation AI, an ego generation means using a text generation AI, a voice generation means using a voice generation AI, an interactive dialogue system means, a multilingual support means, a server infrastructure means that can operate 24 hours a day, a means for a user-customized character to tell a unique story, and a means for updating the story in real time according to user selections. As a result, the user-customized character can interactively tell a story and be updated in real time, improving the user experience and enabling applications in a variety of uses.

[1874] "Image generation AI" is an artificial intelligence technology that generates a character's appearance based on user input.

[1875] "Text generation AI" is an artificial intelligence technology for generating character egos and stories.

[1876] "Voice generation AI" is an artificial intelligence technology for generating character voices.

[1877] An "interactive dialogue system" is a system that allows users and characters to interact in real time.

[1878] "Multilingual support" is a function that supports multiple languages ​​and translates what characters say according to the user's language.

[1879] "24-hour server infrastructure" refers to server infrastructure that can be kept running at all times.

[1880] "A means for user-customized characters to tell their own stories" refers to a means for users to create and tell their own stories with characters they have created.

[1881] "Means for updating a story in real time according to user selection" means means for changing and updating a story in real time based on user selection.

[1882] The following system configuration and processing procedure will be described as an embodiment of the present invention.

[1883] System Configuration

[1884] This system uses the following hardware and software:

[1885] Hardware

[1886] Smartphone (iOS, Android)

[1887] Head-mounted displays (Oculus Quest, HTC Vive, etc.)

[1888] Server infrastructure (AWS, etc.)

[1889] software

[1890] Unity (game engine)

[1891] TensorFlow (generative AI model)

[1892] Google Translate API (multilingual...

Claims

1. A means for acquiring a prompt sentence including an instruction or request to be input to the generative AI model; A means for generating a character's appearance using an image generation AI based on the prompt sentence; A means for generating an ego of the character using a sentence generation AI based on the prompt sentence; A means for generating a voice of the character using a voice generation AI based on the prompt sentence; means for integrating the appearance, ego and voice of the generated characters to create a consistent character setting; A system including:

2. A means for acquiring a prompt statement including a request including a character's appearance, personality, and voice; A means for extracting elements including the character's appearance, personality, and voice from the acquired prompt sentence; A means for generating a text-based profile including detailed settings of the character using a generative AI model based on each of the extracted elements of the character; means for generating a voice for said character using voice synthesis software based on the generated profile; A system including:

3. A means to accept settings for the character's appearance, personality, and voice within the application; means for generating a 3D model and a voice for the character based on the received settings; A means for generating a story using a generative AI model based on a theme selected by a user; a means for updating the progress of the story in real time in response to user selections; means for enabling a user to interact with said character by voice; A system including:

Images