system

Abstract

Provide a system. A means for receiving audio data and image data of a specific character or celebrity; means for analyzing received voice data and performing voice synthesis; A means for analyzing image data and generating a character avatar; A way to analyze character scripts and incorporate unique phrases and tones into the AI ​​model, A means to receive a company's product information, advertising messages, and FAQ data and use them to train an AI model; A way to preview and adjust the generated AI chatbot, The means to deploy and integrate the final AI chatbot into a specific platform, A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Companies are searching for effective advertising methods to promote their products and services to end users, but traditional advertising models are one-way information transmission and lack interactive communication with users. This makes it difficult to maintain consumer interest and limits the effectiveness of communicating product information. Furthermore, advertising methods that utilize specific characters or celebrities are expensive and time-consuming, placing a heavy burden on companies. To address these challenges, new advertising methods are needed that can effectively communicate the appeal of products through dialogue with users. [Means for solving the problem]

[0005] This invention proposes a system that provides businesses with AI chatbots modeled after specific characters or celebrities. Specifically, it receives voice and image data, analyzes them, synthesizes speech, and generates a character avatar. It then analyzes the character's script to incorporate its unique phrasing and tone into the AI ​​model, which then integrates the company's product information and advertising messages. The generated AI chatbot can be adjusted using a preview function and deployed to the company's website, app, social media, etc. This allows end users to obtain product information through interactions with familiar characters, resulting in effective advertising.

[0006] "Voice data" refers to digital data containing recorded voices of characters or celebrities, and is used for voice synthesis and analysis.

[0007] "Image data" is digital data that includes visual information of characters and celebrities, and is data used for avatar generation and visual expression.

[0008] "Speech synthesis" is a technology in which a computer generates a human voice based on voice data.

[0009] A "character avatar" is a digital recreation of the visual representation of a particular character or celebrity, used as a visual element for a chatbot.

[0010] A "character script" is text data that includes lines and unique expressions that characters or celebrities speak in specific situations.

[0011] An "AI model" is a mathematical model trained to perform a specific task using artificial intelligence techniques, and in this invention is responsible for natural language processing and speech synthesis.

[0012] "Company product information" is detailed information about products and services provided by a company, including data such as price, features, and usage.

[0013] An "advertising message" is text data that includes specific promotional content and branding information that a company wants to convey to consumers.

[0014] "FAQ data" is data that compiles frequently asked questions about products and services and their answers.

[0015] "Natural language processing" is a technology that allows computers to understand and process human language.

[0016] "Training" is the process of training an AI model using historical and test data to optimize it for a specific task.

[0017] "Preview" is a function that allows you to check the behavior and responses of the generated AI chatbot in advance and make any necessary adjustments.

[0018] "Deployment" is the process of placing the completed AI chatbot on a designated platform and actually starting operation.

[0019] "Specific platform" refers to the digital environment, such as a website, app, or social media, where the AI ​​chatbot actually operates.

[0020] "UI" stands for user interface, a graphical interface that allows users to interact with AI chatbots.

[0021] "Interaction data" refers to data that includes the content of the conversations exchanged between users and AI chatbots, as well as operation history.

[0022] A "server" is a computer system that processes and stores data, and in this invention is responsible for generating and training chatbots.

[0023] A "terminal" is a computer graphics device used by a user and is the hardware used to interact with a chatbot.

[0024] "Users" are individuals or organizations that use this system and receive product and service information via corporate advertising tools. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0026] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0027] First, the terms used in the following description will be explained.

[0028] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0029] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0030] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0031] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0032] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0033] [First embodiment]

[0034] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0035] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0036] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0037] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0038] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0039] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0040] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0041] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0042] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0043] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0044] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0045] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0046] The present invention is a system that provides businesses with AI chatbots modeled after specific characters or celebrities, and aims to effectively communicate product information and advertising messages through dialogue with users. Specific embodiments of the system are described below.

[0047] Program processing overview

[0048] Terminal

[0049] 1. A corporate user accesses the management console and logs into the system.

[0050] 2. After logging in, the corporate user uploads data of the selected talent, celebrity, anime character, etc. to the system, including voice data, image data, and character scripts.

[0051] 3. The uploaded data is sent to the system's server.

[0052] server

[0053] 1. Receive the voice data and analyze it using natural language processing technology.

[0054] 2. Based on the analyzed voice data, voice synthesis is performed to generate the character's voice.

[0055] 3. Image data is analyzed to generate a character avatar, using facial recognition and animation generation technology.

[0056] 4. Analyze character scripts and incorporate unique phrases and tones into the AI ​​model.

[0057] 5. Receive product information, advertising messages, and FAQ data provided by companies, and use this information to further customize and train the AI ​​model.

[0058] 6. Allow enterprise users to preview the generated AI chatbot and make any necessary adjustments.

[0059] 7. Deploy the AI ​​chatbot once it has been adjusted to the specified platform (website, app, social media, etc.).

[0060] User

[0061] 1. An end user accesses a company's platform (website or app) and initiates a conversation with an AI chatbot.

[0062] 2. AI chatbots respond to user inquiries and convey product information and advertising messages.

[0063] 3. User feedback and interaction history are sent to the server and used for the next model update and performance improvement.

[0064] Specific examples

[0065] For example, suppose a car manufacturer wants to create an AI chatbot modeled after a popular anime character to promote a new product. In this case, the process would involve the following steps:

[0066] Corporate User Operations

[0067] 1. The automaker's representative logs in to the management console.

[0068] 2. Upload voice data, image data, and scripts (character lines and distinctive expressions) of popular anime characters.

[0069] 3. Enter detailed information and advertising messages about the company's new products.

[0070] Server Processing

[0071] 1. Analyze the uploaded audio data and generate the character's voice.

[0072] 2. Analyze the image data and create a character avatar.

[0073] 3. Analyze the character script and incorporate the character's unique tone of voice into the AI ​​model.

[0074] 4. Integrate new product information and advertising messages into the AI ​​model to train the chatbot.

[0075] 5. After training is complete, a preview of the chatbot is provided, allowing corporate users to check and adjust its operation.

[0076] 6. Once the AI ​​chatbot is fully tuned, it will be deployed on the car manufacturer's official website.

[0077] End User Experience

[0078] 1. An end user visits the car manufacturer's website.

[0079] 2. An AI chatbot based on a popular anime character responds to user inquiries and provides information about new products.

[0080] 3. The user's interaction history and feedback are sent to the server and used to improve the model next time.

[0081] This allows users to efficiently obtain product information through friendly characters, and companies can carry out effective promotions. As a new form of advertising, this system will likely play an important role in companies' marketing strategies.

[0082] The processing flow will be explained below.

[0083] Step 1:

[0084] Terminal: Corporate users log in to the management console by entering their username and password, and once authentication is successful, they are able to access the system's main screen.

[0085] Step 2:

[0086] On your device, select "Character Selection" from the Admin Console menu, select a talent, celebrity, or anime character to use, and upload the voice data, image data, and character script of the selected character to the system.

[0087] Step 3:

[0088] Server: Receives the uploaded voice data and sends it to the voice analysis engine, which converts the voice data into text (Speech-to-Text) and synthesizes speech based on that text (Text-to-Speech).

[0089] Step 4:

[0090] Server: Receives image data and sends it to the image analysis engine, which uses deep learning technology to generate an avatar of the character, which is used as a visual representation.

[0091] Step 5:

[0092] Server: Receives character scripts and sends them to a natural language processing (NLP) engine, which analyzes the scripts, extracts the character's unique phrasing and tone of voice, and trains the AI ​​model.

[0093] Step 6:

[0094] Terminal: Corporate users enter product information, advertising messages, and FAQ data into the management console and send them to the server.

[0095] Step 7:

[0096] Server: Analyzes received product information and advertising messages and integrates them into the AI ​​model, allowing the chatbot to provide appropriate information to the user.

[0097] Step 8:

[0098] Server: Integrates character voices, avatars, phrases, and product information to generate AI chatbots, which are then provided as previews for corporate users to check.

[0099] Step 9:

[0100] Device: The company user can check the chatbot's behavior on the preview screen and adjust the settings and scripts as necessary. The adjustments are sent back to the server and reflected.

[0101] Step 10:

[0102] Server: Deploy the AI ​​chatbot once it has been tuned to a specific platform (e.g., company website, app, social media), set up API endpoints using a deployment tool, and verify the integration.

[0103] Step 11:

[0104] User: An end user accesses a company's website or app and starts a conversation with an AI chatbot. The chatbot analyzes the user's inquiry and generates an appropriate response.

[0105] Step 12:

[0106] Server: Collects user interaction history and feedback and stores it in a database. The collected data is used for the next model update and performance improvement.

[0107] Through the above processing steps, businesses can utilize AI chatbots as an effective advertising tool to efficiently convey product information and advertising messages to end users.

[0108] Example 1

[0109] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0110] Companies need a user-friendly interface to effectively communicate product information and advertising messages to customers. However, with existing systems, it is difficult to create an AI chatbot modeled after a specific character or celebrity, and reproducing their unique phrasing and tone of voice requires advanced technology and a great deal of time. It is also not easy to collect user interaction data and use it to improve the model for the next time. As a result, companies' marketing strategies may not be implemented effectively.

[0111] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0112] In this invention, the server includes a means for corporate users to log in, a means for receiving voice data and image data of specific characters or celebrities, a means for analyzing the received voice data and performing speech synthesis using natural language processing technology, a means for analyzing image data and generating a character avatar using facial recognition and animation generation technology, a means for analyzing the character script and incorporating the character's unique phrasing and tone of voice into the AI ​​model, a means for receiving the company's product information, advertising messages, and FAQ data and training the AI ​​model based on them, a means for corporate users to preview and adjust the generated AI chatbot, and a means for deploying and integrating the final AI chatbot into a specific platform. This allows companies to easily create friendly AI chatbots using specific characters or celebrities, enabling them to implement effective marketing strategies. Furthermore, user interaction data can be collected and used to improve the model next time, thereby continuously improving the performance of the AI ​​chatbot.

[0113] The "means for corporate users to log in" refers to a function that allows a user to access the system's management console, enter authentication information, log in to the system, and start operations.

[0114] "Means for receiving audio data and image data of specific characters or celebrities" refers to a function for uploading audio and image files of specific characters or celebrities to the system and transmitting them to the server.

[0115] "Means for analyzing received voice data and synthesizing voice using natural language processing technology" refers to the function in which the server analyzes the voice data, extracts text, and generates the character's voice using a voice synthesis tool.

[0116] "Means for analyzing image data and generating a character avatar using facial recognition and animation generation technology" refers to a function in which the server analyzes image data, extracts the facial features of a character, and creates a moving avatar using facial recognition and animation generation technology.

[0117] "Means for analyzing character scripts and incorporating unique phrases and tones into the AI ​​model" refers to a function in which the server analyzes the text data of a character script and incorporates the character's unique phrases and tones into the AI ​​model.

[0118] "Means for receiving product information, advertising messages, and FAQ data from a company and training an AI model based on the same" refers to a function in which the server receives product information, advertising messages, and FAQ data provided by a company and uses that data to train an AI model.

[0119] "Means for corporate users to preview and adjust the generated AI chatbot" refers to a function that allows corporate users to preview the generated AI chatbot on the management console and adjust it as necessary.

[0120] "Means for deploying and integrating the final AI chatbot into a specific platform" refers to the function by which the server deploys the finalized AI chatbot into a specified platform such as a website, app, or social networking site, and begins operation.

[0121] The present invention is a system that provides businesses with AI chatbots modeled after specific characters or celebrities, and aims to effectively communicate product information and advertising messages through dialogue with users. Specific embodiments are described below.

[0122] Hardware and software used

[0123] Device: PC or tablet used by a corporate user

[0124] Software: Management console, web browser

[0125] Server: High-performance cloud server (e.g., AWS (registered trademark) EC2, Google (registered trademark) Cloud)

[0126] Software: Natural Language Processing (NLP) tools (e.g., Intel OpenVINO), facial recognition tools (e.g., dlib), speech synthesis tools (e.g., Google Text-to-Speech API)

[0127] Program processing overview

[0128] The system begins when a corporate user accesses the management console and logs in. The corporate user then uploads data about the talent, celebrity, or anime character they have selected to the system, including voice data, image data, and character scripts.

[0129] The device sends this data to a server, which uses NLP tools to analyze the voice data and synthesizes it. The server also analyzes the image data with facial recognition tools to generate a character avatar. The character script is analyzed to incorporate specific phrasing and tone of voice into the AI ​​model.

[0130] Product information, advertising messages, and FAQ data provided by companies are received by the server and used to train the AI ​​model. After the AI ​​model is trained, the generated AI chatbot is provided for company users to preview and make any necessary adjustments. The final AI chatbot is then deployed to the specified platform (website, app, social media, etc.).

[0131] End users access the company's platform and begin a conversation with the AI ​​chatbot. The AI ​​chatbot responds to the user's inquiries and provides product information and advertising messages. User feedback and conversation history are sent to the server and used to improve the model for the next time.

[0132] Specific examples

[0133] For example, suppose an automobile manufacturer creates an AI chatbot modeled after a popular anime character to promote a new product. In this case, the automobile manufacturer's representative operates it as follows:

[0134] 1. The automaker's representative logs in to the management console.

[0135] 2. Upload voice data, image data, and scripts (character lines and distinctive expressions) of popular anime characters.

[0136] 3. Enter detailed information and advertising messages about the company's new products.

[0137] The server performs the following process:

[0138] 1. Analyze the uploaded audio data and generate the character's voice.

[0139] 2. Analyze the image data and create a character avatar.

[0140] 3. Analyze the character script and incorporate the character's unique tone of voice into the AI ​​model.

[0141] 4. Integrate new product information and advertising messages into the AI ​​model to train the chatbot.

[0142] 5. After training is complete, a preview of the chatbot is provided, allowing corporate users to check and adjust its operation.

[0143] 6. Once the AI ​​chatbot is fully tuned, it will be deployed on the car manufacturer's official website.

[0144] The end user experience is as follows:

[0145] 1. An end user visits the car manufacturer's website.

[0146] 2. An AI chatbot based on a popular anime character responds to user inquiries and provides information about new products.

[0147] 3. The user's interaction history and feedback are sent to the server and used to improve the model next time.

[0148] Prompt Sentence Examples

[0149] "Generate an AI chatbot using popular anime characters to provide information about a company's new products."

[0150] "Enter your character script and product information to create a customized AI chatbot."

[0151] Deploy the generated AI chatbot on your website and collect user feedback.

[0152] This system allows companies to carry out effective promotions and allows users to efficiently obtain product information through friendly characters.

[0153] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0154] Step 1:

[0155] A corporate user accesses the Admin Console and logs in.

[0156] Input: Username, Password

[0157] How it works:

[0158] The terminal displays the management console screen, and the corporate user enters their username and password. The server receives the entered authentication information and authenticates it by checking it against the database. Once the user is authenticated, the server generates a dashboard page, sends it to the terminal, and displays it.

[0159] Output: Dashboard page

[0160] Step 2:

[0161] Corporate users upload voice data, image data, and character scripts of characters and celebrities.

[0162] Input: Audio data, image data, character script

[0163] How it works:

[0164] Corporate users click the "Create a new chatbot" button on the management console, select and upload files for voice data (e.g., "character voice.wav"), image data (e.g., "character image.png"), and character script (e.g., "character lines.txt"). The device sends these files to the server, which saves the received data in a specified folder and prepares it for the next process.

[0165] Output: Saved audio data, image data, character scripts

[0166] Step 3:

[0167] The server analyzes the voice data and performs voice synthesis.

[0168] Input: Saved audio data

[0169] How it works:

[0170] The server analyzes the stored voice data "character voice.wav" using an NLP tool (e.g., Intel's OpenVINO). This analysis extracts text from the voice and obtains voice characteristics. The server then uses the Google Text-to-Speech API to synthesize a character's voice based on the extracted text. The synthesized voice data is saved and used for further processing.

[0171] Output: Synthesized voice data

[0172] Step 4:

[0173] The server analyzes the image data and generates a character avatar.

[0174] Input: Saved image data

[0175] How it works:

[0176] The server analyzes the stored image data "characterimage.png" using a facial recognition tool (e.g., dlib) to extract the character's facial features (e.g., large eyes, distinctive smile). It then uses animation generation technology to generate an avatar with those facial features. The generated avatar is then used for the next process.

[0177] Output: Generated character avatar

[0178] Step 5:

[0179] The server analyzes the character script and incorporates the character's unique phrases and tone of voice into the AI ​​model.

[0180] Input: Saved character script

[0181] How it works:

[0182] The server analyzes the saved character script "character dialogue.txt." Through this analysis, the character's unique expressions and tone of voice are extracted and incorporated into the AI ​​model. This allows dialogue that reflects the character's individuality. The processed content is saved in the AI ​​model and used for subsequent processing.

[0183] Output: AI model incorporating the character's unique phrasing and tone of voice

[0184] Step 6:

[0185] The server trains the AI ​​model.

[0186] Input: Company product information, advertising messages, FAQ data, processed AI model

[0187] How it works:

[0188] The server receives product information, advertising messages, and FAQ data provided by companies and integrates this data into the AI ​​model. The server then uses NLP and machine learning algorithms to train the AI ​​model based on this data. This creates an AI chatbot whose characters are capable of appropriately conveying product information and advertising messages.

[0189] Output: A trained AI model

[0190] Step 7:

[0191] Corporate users can preview and adjust the generated AI chatbot.

[0192] Input: A trained AI model

[0193] How it works:

[0194] The server provides the generated AI chatbot for preview on the management console. Corporate users can actually try to interact with the AI ​​chatbot on the preview screen and check its behavior and what it says. If necessary, corporate users can adjust the character's specific speech content and interface and send the results to the server.

[0195] Output: The final tuned AI chatbot

[0196] Step 8:

[0197] The server deploys and integrates the final AI chatbot into a specific platform.

[0198] Input: The final tuned AI chatbot

[0199] How it works:

[0200] The server deploys the finalized AI chatbot to the designated platform, such as a website, app, or social networking site, and begins operation. The server integrates the chatbot with the platform through web hosting and APIs, making it immediately available for use.

[0201] Output: Deployed AI chatbot

[0202] Step 9:

[0203] An end user accesses a company's platform and begins interacting with an AI chatbot.

[0204] Input: End user inquiry

[0205] How it works:

[0206] An end user accesses a company's website or app and clicks the "Ask a character!" button. The AI ​​chatbot responds with "Hello, how is it?" and provides product information or advertising messages in response to the user's inquiry. The user's interaction history and feedback are sent to the server and used to improve the model for the next time.

[0207] Output: User interaction history and feedback

[0208] This system allows companies to carry out effective promotions and allows users to efficiently obtain product information through friendly characters.

[0209] (Application example 1)

[0210] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0211] The purpose of this invention is to provide a means for effectively communicating corporate product information and advertising messages. However, in conventional systems, when AI chatbots modeled after characters or celebrities are provided to companies, there is no mechanism for direct interaction with customers in physical stores. As a result, it is difficult to provide customers visiting physical stores with an experience equivalent to that of online information provision. Furthermore, there is a lack of technological means for realizing interactive dialogue that reflects the atmosphere of the physical store and customer behavior data.

[0212] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0213] In this invention, the server includes: means for receiving voice data and image data of a specific character or celebrity; means for analyzing the voice data and performing voice synthesis; means for analyzing the image data and generating a character avatar; means for analyzing the character script and incorporating the character's unique phrasing and tone of voice into an AI model; means for receiving a company's product information, advertising messages, and FAQ data and training the AI ​​model based thereon; means for previewing and adjusting the generated AI chatbot; means for deploying and integrating the final AI chatbot into a specific platform; and means for interacting with customers via terminals installed in physical stores and providing product information. This enables information provision in physical stores to be as effective as online, and provides an interactive customer experience that matches the atmosphere of the physical store.

[0214] "Audio data" refers to digital data containing the recorded voice of a specific character or celebrity.

[0215] "Image data" refers to data that stores images of specific characters or celebrities in digital format.

[0216] "Analysis" is the process of understanding the content of received audio or image data and extracting its features.

[0217] "Speech synthesis" is a technology that generates new voices based on analyzed voice data.

[0218] A "character avatar" is a digital graphic that recreates the appearance of a particular character or celebrity.

[0219] A "character script" is a collection of lines that contain the distinctive phrasing and tone of a character or celebrity.

[0220] An "AI model" is an algorithm that uses artificial intelligence, a digital model that learns and performs specific tasks.

[0221] "Training" is the process by which an AI model learns using specific data.

[0222] "Preview" is a function that allows you to check the behavior of the generated AI chatbot in advance.

[0223] "Deployment" refers to placing the generated AI chatbot so that it can operate on a specific platform.

[0224] "Platform" is a general term for the environment and services in which AI chatbots operate.

[0225] "In-store terminals" are digital devices that enable customer interaction in physical stores.

[0226] "Customers" refers to consumers and users who visit physical stores.

[0227] "Product information" is detailed information about the features and benefits of products offered by a company.

[0228] An "advertising message" is a promotional message that conveys the appeal of a product offered by a company.

[0229] "FAQ data" is a collection of frequently asked questions and their answers.

[0230] "Interaction data" is data generated when a user interacts with an AI chatbot.

[0231] "Ambient sounds" are background sounds used to recreate the atmosphere of a physical store.

[0232] "Video" refers to videos and images that visually convey the atmosphere of a physical store.

[0233] "Behavioral data" is data that records customer behavior and movements in physical stores.

[0234] MODE FOR CARRYING OUT THE INVENTION

[0235] This invention is a system that realizes an AI chatbot modeled after a specific character or celebrity, and aims to effectively convey product information and advertising messages through dialogue with customers in a physical store environment.

[0236] System Configuration and Operation

[0237] Hardware

[0238] This system uses the following hardware:

[0239] Brick-and-mortar devices: Includes devices such as tablets, kiosks, and smart glasses.

[0240] Server: A server with high-performance computing resources for training and deploying AI models.

[0241] software

[0242] The software used includes the following:

[0243] Natural Language Processing Library: Uses high-performance AI models such as GPT-3 (registered trademark).

[0244] Speech synthesis engine: Provides technology for synthesizing voice data.

[0245] Image analysis library: Generate character avatars using OpenCV etc.

[0246] Database system: A system for storing customer interaction and behavior data.

[0247] System Operation

[0248] The server performs the following process.

[0249] 1. Receive audio and image data of specific characters and celebrities provided by companies.

[0250] 2. Analyze the voice data and perform voice synthesis. Generate a new voice based on the analyzed voice data.

[0251] 3. Analyze the image data and generate a character avatar.

[0252] 4. Analyze character scripts and incorporate unique phrases and tones into the AI ​​model.

[0253] 5. Receives company product information, advertising messages, and FAQ data and uses them to train an AI model.

[0254] 6. Preview the generated AI chatbot and make any necessary adjustments.

[0255] 7. The final AI chatbot is deployed to a specific platform, i.e., a terminal installed in a physical store, and begins interacting with customers.

[0256] Specific examples

[0257] For example, consider a car dealership that installs an AI chatbot modeled after a popular anime character in its physical store to promote a new product.

[0258] Dealers log in to the management console and upload voice data, image data, and scripts (character lines and distinctive expressions) for specific animated characters.

[0259] The server analyzes this data and generates an AI chatbot.

[0260] The generated chatbot is deployed on tablet devices installed in the car dealership's physical store.

[0261] When customers visit the store, animated characters will provide information about cars and related products and introduce promotions.

[0262] Prompt Sentence Examples

[0263] "Please tell us the character's name and details of the new car model for visitors."

[0264] "Please tell me the character's name and the latest promotional information."

[0265] In this way, the system of the present invention enables interactive dialogue through friendly characters even in brick-and-mortar stores, and supports the provision of effective product information and promotional activities.

[0266] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0267] Step 1:

[0268] Corporate users access the management console and log in to the system. They then upload voice data, image data, and character scripts of specific characters or celebrities to the system. The input data is voice data, image data, and script data, which is then sent to the server.

[0269] Step 2:

[0270] The server analyzes the received voice data, extracts features from the voice data using natural language processing technology, and then generates the character's unique voice using a voice synthesis engine. The input data here is the voice data, and the output is a synthesized voice file.

[0271] Step 3:

[0272] The server analyzes the image data and generates a character avatar. It uses an image analysis library to extract features from the received image data and create an avatar. The input data is the image data, and the output is the generated character avatar.

[0273] Step 4:

[0274] The server analyzes the character script and incorporates the specific phrasing and tone of voice into the AI ​​model. A natural language processing model is used to train the phrasing and tone of voice of the script data. The input data here is the script data, and the output is the trained AI model.

[0275] Step 5:

[0276] The server receives product information, advertising messages, and FAQ data provided by the company and further trains the AI ​​model based on this information. The input data are product information, advertising messages, and FAQ data, and the output is an AI model that integrates product information.

[0277] Step 6:

[0278] Corporate users can preview the generated AI chatbot and make any necessary adjustments. They can check its operation on a device installed in a physical store and check the voice and avatar behavior. The input data is a trained AI model, a synthesized voice file, and a character avatar, and the output is a previewed AI chatbot.

[0279] Step 7:

[0280] The final AI chatbot is deployed to a terminal in a physical store and begins interacting with customers. A deployment tool is used to place the AI ​​chatbot on the terminal and prepare a system for interacting with customers. The input data is the previewed AI chatbot, and the output is an AI chatbot that interacts with customers in a physical store.

[0281] Step 8:

[0282] A user (customer) visits a physical store and initiates a conversation with an AI chatbot via a terminal device. The terminal device receives the customer's input and sends it to the server. The input data is the customer's question, and the output is the AI ​​chatbot's response.

[0283] Step 9:

[0284] The server collects customer interaction history and feedback and uses it for the next model update and performance improvement. This data is stored in a database and machine learning algorithms are used to improve the model. The input data is the customer interaction history and feedback, and the output is an updated AI model.

[0285] These steps enable the server and terminal to work together to provide effective information and customer experiences even in physical stores.

[0286] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0287] This invention provides a system that combines an emotion engine with an AI chatbot modeled after a specific character or celebrity, with the aim of effectively communicating product information and advertising messages through dialogue with users. It also recognizes the user's emotions and dynamically adjusts responses based on them, enabling a more natural and engaging conversational experience.

[0288] Program processing overview

[0289] Terminal

[0290] 1. A corporate user logs in to the management console. After entering their username and password and successful authentication, they are able to access the system's main screen.

[0291] 2. After logging in, the corporate user uploads the voice data, image data, and character script of the selected talent, celebrity, or anime character to the system.

[0292] 3. Perform the necessary initial setup for the emotion engine and adjust the emotion model parameters.

[0293] server

[0294] 1. The voice data is received and analyzed using natural language processing technology. The analysis engine converts the voice data into text and synthesizes speech based on that text data.

[0295] 2. Image data is received and a character avatar is generated using an image analysis engine.

[0296] 3. The character script is received and analyzed by a natural language processing (NLP) engine to extract specific phrases and tone of voice, which are then trained into an AI model.

[0297] 4. Receive product information, advertising messages, and FAQ data provided by companies, and use them to customize and train AI models.

[0298] 5. Use the emotion engine to analyze and recognize emotions from user input data.

[0299] 6. Select and set scripts for the AI ​​chatbot to dynamically adjust responses and tone based on emotion recognition results.

[0300] 7. Allow enterprise users to preview the generated AI chatbot and make any necessary adjustments.

[0301] 8. Deploy the AI ​​chatbot once it has been tuned to a specific platform.

[0302] User

[0303] 1. An end user accesses a company's platform and initiates a conversation with an AI chatbot.

[0304] 2. The emotion engine analyzes the emotions in the user's input data (text and voice) and adjusts the chatbot's responses accordingly.

[0305] 3. The AI ​​chatbot provides appropriate product information and advertising messages in response to user inquiries.

[0306] 4. The dialogue history and feedback are sent to the server and used to improve the model and performance next time.

[0307] Specific examples

[0308] Corporate User Operations

[0309] Let's take the example of a fashion brand creating an AI chatbot modeled after a popular character to launch a new advertising campaign.

[0310] 1. A brand marketer logs in to the Admin console.

[0311] 2. Upload the voice data, image data, and character scripts of popular characters and set the parameters of the emotion engine.

[0312] 3. Enter product information and advertising messages into the system.

[0313] Server Processing

[0314] 1. Analyze the uploaded voice data and perform voice synthesis.

[0315] 2. Analyze the image data and generate a character avatar.

[0316] 3. Analyze the character script and train the AI ​​model to recognize specific phrases and tones.

[0317] 4. Incorporate received product information and advertising messages into the model to train the chatbot.

[0318] 5. Use an emotion engine to analyze user emotions and dynamically adjust responses based on the results.

[0319] 6. Check the behavior of your AI chatbot with the preview function and make any necessary adjustments.

[0320] 7. After tuning, deploy the AI ​​chatbot on the brand's official website.

[0321] End User Experience

[0322] 1. An end user visits a brand's website and initiates a conversation with an AI chatbot.

[0323] 2. The emotion engine recognizes emotions from the user's input data, and the chatbot responds according to those emotions.

[0324] 3. Users can effectively obtain information about new products and promotions through friendly characters.

[0325] 4. User feedback and interaction history are sent to the server and used to improve the model next time.

[0326] In this way, AI chatbots that utilize an emotion engine allow users to enjoy a more natural and engaging conversational experience, and businesses can realize effective advertising tools. This system will provide new value in advertising and marketing.

[0327] The processing flow will be explained below.

[0328] Step 1:

[0329] Terminal: Corporate users log in to the management console by entering their username and password, and once authentication is successful, they are able to access the system's main screen.

[0330] Step 2:

[0331] On your device, select "Character Selection" from the Admin Console menu, select a talent, celebrity, or anime character to use, and upload the voice data, image data, and character script of the selected character to the system.

[0332] Step 3:

[0333] Device: Perform the necessary initial setup for the emotion engine and adjust the parameters of the emotion model, for example, setting which emotions (joy, sadness, anger, etc.) to recognize.

[0334] Step 4:

[0335] Server: Receives the uploaded voice data and sends it to the voice analysis engine. The analysis engine converts the voice data into text (Speech-to-Text). Text-to-Speech synthesis is performed based on the converted text data to generate the character's voice.

[0336] Step 5:

[0337] Server: Receives image data and sends it to the image analysis engine, which uses deep learning technology to generate a character avatar. The generated avatar is used as the visual representation of the chatbot.

[0338] Step 6:

[0339] Server: Receives character scripts and sends them to a natural language processing (NLP) engine, which analyzes the scripts, extracts the character's unique phrasing and tone of voice, and trains the AI ​​model.

[0340] Step 7:

[0341] Terminal: Corporate users enter product information, advertising messages, and FAQ data into the management console and send them to the server.

[0342] Step 8:

[0343] Server: Analyzes received product information and advertising messages and integrates them into the AI ​​model, allowing the chatbot to provide appropriate information to the user.

[0344] Step 9:

[0345] Server: Analyzes and recognizes emotions from user input data (text and voice) using an emotion engine. For example, it uses natural language processing technology to classify the user's emotional state into categories such as Joy, Anger, and Sadness.

[0346] Step 10:

[0347] Server: Dynamically adjusts the AI ​​chatbot's response and tone based on the results of emotion recognition. For example, if the user is angry, respond with a calm tone; if the user is happy, respond with a cheerful tone.

[0348] Step 11:

[0349] Server: Integrates character voices, avatars, phrases, emotion recognition, and product information to generate an AI chatbot. The generated chatbot is provided as a preview for corporate users to check.

[0350] Step 12:

[0351] Device: The company user can check the chatbot's behavior on the preview screen and adjust the settings and scripts as necessary. The adjustments are sent back to the server and reflected.

[0352] Step 13:

[0353] Server: Deploy the finalized AI chatbot to a specific platform (e.g., company website, app, social media), set up API endpoints using a deployment tool, and verify the integration.

[0354] Step 14:

[0355] User: An end user visits a company's website or app and initiates a conversation with an AI chatbot. When the user types a question or comment, the emotion engine analyzes their emotional state and the chatbot generates an appropriate response.

[0356] Step 15:

[0357] Server: Collects user interaction history and feedback and stores it in a database. The collected data is used for the next model update and performance improvement.

[0358] Through these processing steps, businesses can utilize AI chatbots combined with emotion engines to provide end users with a more natural and engaging conversational experience, while also serving as an effective advertising tool.

[0359] Example 2

[0360] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0361] Conventional chatbot systems lack the ability to recognize emotions in user interactions and dynamically adjust responses based on those emotions. As a result, users do not enjoy a natural conversational experience, making it difficult for companies to convey effective advertising messages. Furthermore, when modeling characters or celebrities, it is difficult to reproduce their unique expressions and tone of voice, which often results in low user engagement.

[0362] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0363] In this invention, the server includes means for receiving voice data and image data of a specific character or celebrity, means for analyzing the received voice data and performing voice synthesis, means for analyzing the image data and generating a character avatar, means for analyzing the character script and incorporating the character's unique expressions and tone of voice into an AI model, means for receiving a company's product information, advertising messages, and FAQ data and training the AI ​​model based thereon, means for recognizing emotions from user input data and dynamically adjusting responses using an emotion engine, means for previewing and adjusting the generated AI chatbot, and means for deploying and integrating the final AI chatbot into a specific platform. This allows users to enjoy a more natural and engaging conversational experience and enables companies to deliver effective advertising messages.

[0364] "Audio data" refers to data in which audio information is recorded in digital format, and includes the voice of a specific character or celebrity.

[0365] "Image data" refers to visual information recorded in digital format, and includes images of specific characters or celebrities.

[0366] A "character script" is text data that records the lines, expressions, and phrases used by a specific character.

[0367] "Speech synthesis" is a technology that generates speech based on text data, and is performed using natural language processing technology.

[0368] A "character avatar" is a digital representation of a particular character that is generated based on analyzed image data.

[0369] "Distinctive expressions and tone of voice" refers to the unique ways of expression and speaking used by particular characters or celebrities.

[0370] An "AI model" is a mathematical model for realizing artificial intelligence, and specifically includes algorithms that utilize machine learning and deep learning.

[0371] An "emotion engine" is the part of the system that recognizes emotions from user input data and dynamically adjusts responses based on the results.

[0372] "Platform" refers to the digital environment, such as a particular website or application, in which an AI chatbot is deployed.

[0373] "Deployment" means placing the developed system or software in a production environment and making it actually function.

[0374] The present invention provides a system that combines an emotion engine with an AI chatbot modeled after a specific character or celebrity. The purpose is to effectively communicate product information and advertising messages through dialogue with users. Specific embodiments of this system are described in detail below.

[0375] Hardware and Software Configuration

[0376] Terminal

[0377] The terminal is used by corporate users and is a device for accessing the management console. This management console includes functions for uploading data, setting parameters, and previewing. Specifically, since it is accessed via a web browser, the terminal can be used with a wide range of operating systems, including Windows, macOS (registered trademark), and Linux (registered trademark).

[0378] server

[0379] The server analyzes the data and generates the AI ​​chatbot. The server is equipped with the following main software and engines:

[0380] Natural Language Processing (NLP) engines (e.g., spaCy, NLTK)

[0381] Image analysis engine (e.g. OpenCV, TENSORFLOW (registered trademark))

[0382] Speech synthesis engine (e.g., Google Text-to-Speech API, Amazon Polly)

[0383] Emotion engine (e.g., IBM Watson® Emotion Analysis)

[0384] Data processing and calculation

[0385] Voice data analysis and voice synthesis

[0386] The server receives voice data uploaded by corporate users from their devices and analyzes it using a natural language processing (NLP) engine. The analyzed text data is then converted into speech by a speech synthesis engine.

[0387] Image data analysis and avatar generation

[0388] The server receives the uploaded image data and uses an image analysis engine to generate a character avatar, which is used as the visual for the chatbot.

[0389] Character script analysis and model training

[0390] The server analyzes the character scripts received from corporate users, extracts specific phrases and tones using a natural language processing (NLP) engine, and trains the AI ​​model. This trained model is reflected in the chatbot's dialogue.

[0391] Integrating product information and advertising messages

[0392] Product information, advertising messages, and FAQ data provided by corporate users are received by the server, and the AI ​​model is customized based on this data, allowing marketing information to be naturally incorporated as part of the conversation.

[0393] Emotion recognition and response regulation

[0394] The emotion engine recognizes emotions from user input data and dynamically adjusts the tone of responses based on the results. For example, if a user expresses positive emotions, the chatbot will respond with a positive response.

[0395] Specific examples

[0396] Let's take the example of a fashion brand creating an AI chatbot modeled after a popular character to launch a new advertising campaign. The brand's marketer logs into the management console and uploads the popular character's voice data, image data, and character script to the system. This allows the server to perform voice, image, and NLP analysis to set the character avatar and its unique phrasing and tone of voice. Next, they adjust the emotion engine parameters and enter product information and advertising messages into the system. The fully customized AI chatbot can be checked through the preview function, and after any necessary adjustments are made, it is deployed to the brand's official website.

[0397] A user accesses a brand's website and begins a conversation with an AI chatbot. The emotion engine recognizes emotions from the user's input data and returns appropriate responses, providing a more natural and friendly conversational experience. For example, if a user inputs, "I want a new jacket," the chatbot will provide specific information such as, "The new jacket is from this year's fall / winter collection and is excellent for keeping you warm. It also comes in a wide variety of colors."

[0398] Prompt Sentence Examples

[0399] Here are some examples of prompts to input to a generative AI model:

[0400] 1. "I want to create an AI chatbot that uses an emotion engine. First, please tell me the steps to upload the character's voice and image data."

[0401] 2. "Give me an example of how a chatbot can tailor its response to a user's emotion when they ask a question."

[0402] 3. "How can a fashion brand effectively communicate new product information through friendly characters?"

[0403] By using these prompts, users can obtain more detailed instructions and explanations about specific tasks and settings, providing a more natural and engaging interaction experience for users and enabling companies to effectively communicate their advertising messages.

[0404] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0405] Step 1:

[0406] Terminal

[0407] A corporate user logs in to the management console. As input, they enter their username and password and click the "Login" button. The system receives this and performs authentication. If successful, they are redirected to the main screen. The output is access to the system's main screen.

[0408] Step 2:

[0409] Terminal

[0410] A company user uploads the voice data, image data, and character script of a character. As input, they specify these files in the respective upload forms and click the "Upload" button. The files are sent to the server. The output is the data correctly saved on the server.

[0411] Step 3:

[0412] Terminal

[0413] Corporate users configure the emotion engine initially and adjust the emotion model parameters. As input, they set emotion recognition accuracy and emotion category parameters on the settings screen. The system receives this and saves the emotion engine configuration data. The output is the adjusted emotion model.

[0414] Step 4:

[0415] server

[0416] The server analyzes the voice data and synthesizes the voice. The uploaded voice data is used as input. The server converts the voice to text using a natural language processing (NLP) engine, and then generates a new voice using a speech synthesis engine based on the text. The output is the generated voice data.

[0417] Step 5:

[0418] server

[0419] The server analyzes the image data and generates a character avatar. The uploaded image data is used as input. The server uses an image analysis engine to generate a 3D avatar from the image. The output is the generated character avatar.

[0420] Step 6:

[0421] server

[0422] The server analyzes the character script and trains the AI ​​model to recognize specific phrases and tones. The uploaded character script is used as input. A natural language processing (NLP) engine analyzes the script, extracts specific patterns, and trains the AI ​​model. The output is a trained AI model.

[0423] Step 7:

[0424] server

[0425] The server receives product information, advertising messages, and FAQ data provided by companies and customizes the AI ​​model. These data are used as input. The server analyzes them and uses them as necessary training data to strengthen the AI ​​model. The output is a customized AI model.

[0426] Step 8:

[0427] server

[0428] The server uses an emotion engine to recognize emotions from user input data and dynamically adjust responses. The input is text and voice data from the user's interaction with the chatbot. The server analyzes this using the emotion engine and adjusts the chatbot's response based on the results. The output is a response based on the emotion.

[0429] Step 9:

[0430] Terminal

[0431] Corporate users can preview the generated AI chatbot and make adjustments as needed. The input is a simulated interaction on the preview screen. Based on user feedback, the server adjusts the settings. The output is the adjusted chatbot.

[0432] Step 10:

[0433] server

[0434] The server deploys and integrates the final AI chatbot on a specific platform. The completed chatbot data is used as input. A deployment tool is used to integrate the chatbot into a specified website or application. The output is a chatbot running in a production environment.

[0435] Step 11:

[0436] User

[0437] An end user accesses a company's platform and starts a conversation with an AI chatbot. As input, they type a message into the chat window of a website or application. The system receives this and starts a conversation. The output is the content of the conversation with the chatbot.

[0438] Step 12:

[0439] server

[0440] The server dynamically adjusts responses based on the user's emotions. The inputs are the user's speech and the results of emotion analysis. Based on this, the server selects an appropriate response tone, and the chatbot responds to the user. The output is a response adjusted according to the user's emotions.

[0441] Step 13:

[0442] server

[0443] The server collects the interaction history and feedback and uses it to improve the model. The user's interaction history and feedback are used as input. These data are used for the next model training. The output is an improved AI model.

[0444] (Application example 2)

[0445] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0446] In recent years, AI chatbots have been used on various platforms, but to improve the user experience, more natural and emotionally adaptive responses are required. Conventional systems lack the ability to recognize and dynamically adjust to user emotions, and responses are often mechanical. Furthermore, content distribution services lack mechanisms for providing personalized content based on user emotions. To address these issues, the present invention aims to provide an AI chatbot system combined with an emotion recognition engine to improve the user experience.

[0447] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving voice data and image data of a specific character or celebrity; means for analyzing the received voice data and performing voice synthesis; means for analyzing image data and generating a character avatar; means for receiving company product information, advertising messages, and FAQ data and training an AI model based on them; means for analyzing a user's emotions using an emotion recognition engine and adjusting responses based on the analyzed emotions; means for processing user input data in a content distribution service platform and providing personalized content; means for voice synthesizing a response based on the emotion of a specific character; and means for visualizing a character avatar and coordinating the visual display with audio playback. This makes it possible to provide appropriate responses and personalized content according to the user's emotions.

[0448] "Voice data" refers to information that digitally records the voice of a user or a specific character.

[0449] "Image data" refers to visual information of characters or celebrities recorded in digital format.

[0450] "Speech synthesis" is a technology that uses a computer program to generate a human voice based on text data.

[0451] A "character avatar" is a digital icon or persona that visually represents a particular character.

[0452] A "character script" is a collection of lines and responses that a character utters, each with a specific phrase and tone of voice.

[0453] An "AI model" is a collection of algorithms trained to perform a specific task using artificial intelligence techniques.

[0454] An "emotion recognition engine" is an algorithm or tool that analyzes emotions from user input data and identifies those emotions.

[0455] "Personalized content" is digital content that is individually tailored based on a user's interests and preferences.

[0456] "Deployment" is the process of implementing software or an application on a particular platform.

[0457] A "server" is a computer system that provides services to other computers over a network.

[0458] A "platform" is the underlying hardware or software environment on which an application runs.

[0459] A "natural language processing model" is an algorithm used by computers to understand and generate human language.

[0460] The system for realizing the present invention includes the following components and means: Specific components and their respective processes will be described in detail below.

[0461] System Configuration

[0462] This system uses the following hardware and software:

[0463] Hardware:

[0464] Smartphone

[0465] Head-mounted display (HMD)

[0466] server

[0467] software:

[0468] Google Cloud Natural Language API

[0469] Google Cloud Text-to-Speech API

[0470] Unity

[0471] Program processing overview

[0472] The server has the following features:

[0473] 1. Receiving and analyzing audio and image data:

[0474] It receives voice and image data of characters and celebrities, converts the voice data into text using the Google Cloud Natural Language API, and then synthesizes speech based on that text using the Google Cloud Text-to-Speech API.

[0475] Image data is used to generate and visualize character avatars using Unity.

[0476] 2. Dynamic emotion recognition and response adjustment:

[0477] The user's input data (text) is subjected to sentiment analysis using the Google Cloud Natural Language API, and the sentiment score and strength of the sentiment are calculated.

[0478] Based on this, the character's response is selected and speech synthesis is performed using the Google Cloud Text-to-Speech API.

[0479] The terminal has the following features:

[0480] 1. Providing interaction:

[0481] It provides a user interface (UI) that allows users to initiate a dialogue with the AI ​​chatbot via a smartphone or HMD.

[0482] Visualize a character avatar and play synthesized voice in sync.

[0483] The user interacts with:

[0484] 1. Start the conversation:

[0485] Initiate a conversation with the AI ​​chatbot using an application on your smartphone or HMD.

[0486] Based on user input (text or voice), the system performs sentiment analysis and provides responses and content recommendations based on that.

[0487] Specific examples

[0488] For example, when a user expresses their opinion about a particular movie on a movie streaming service, the service will suggest movies based on the sentiment analysis results. The prompt sentence is as follows:

[0489] Example prompt sentence:

[0490] Input Text: "The movie I saw recently was very moving. It made me cry."

[0491] Emotion score: high positive emotion, strong emotional intensity

[0492] AI chatbot response: "So you like inspiring movies! I recommend 'XYZ' as a tear-jerking movie. I'll play a preview for you."

[0493] In this way, by providing content that corresponds to the user's emotions, it is possible to build a system that can provide a more natural and attractive interactive experience.

[0494] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0495] Step 1:

[0496] Receiving and analyzing audio and image data

[0497] The server receives audio data and image data of specific characters and celebrities provided by the company.

[0498] The received audio data is sent to the Google Cloud Natural Language API and converted into text, which is later used for speech synthesis.

[0499] The image data is analyzed by Unity and a character avatar is generated.

[0500] Input: Voice and image data provided by the company

[0501] Output: Speech data converted to text and generated character avatar

[0502] Step 2:

[0503] Emotion recognition engine for analyzing user emotions

[0504] The user inputs text or voice using a smartphone or HMD.

[0505] The user's input data (text or voice) is sent to the server and analyzed for sentiment scores and sentiment strength using the Google Cloud Natural Language API.

[0506] The results of this sentiment analysis are used to dynamically tailor responses.

[0507] Input: User-entered data (text or voice)

[0508] Output: Sentiment score and intensity of the emotion

[0509] Step 3:

[0510] Response generation and speech synthesis

[0511] The server selects an appropriate response based on the emotion score and the intensity of the emotion.

[0512] The selected response text is synthesized using the Google Cloud Text-to-Speech API and output in the appropriate character's voice.

[0513] The synthesized voice file is sent to the user's terminal.

[0514] Input: Sentiment analysis results, response text

[0515] Output: A synthesized response

[0516] Step 4:

[0517] Character Avatar Visualization

[0518] The server transmits the character avatar along with the synthesized voice to the user's terminal.

[0519] The user's device uses Unity to synchronize the display of the character avatar and the playback of the audio.

[0520] This allows users to experience a natural interaction both visually and aurally.

[0521] Input: Character avatar, audio file

[0522] Output: Visualized avatar and audio playback

[0523] Step 5:

[0524] Collection of conversation history and feedback

[0525] The server collects user interaction history and feedback to help improve the model in future.

[0526] The collected data is used to train AI models.

[0527] Input: Dialogue history, user feedback

[0528] Output: New training data

[0529] Specific working example:

[0530] For example, if a user enters the text, "The movie I saw recently was very moving. It brought me to tears," the system performs sentiment analysis on the text and detects high positive and strong emotional intensity. Based on the results, the system generates a response to the user via synthesized voice: "I see you like moving movies! I recommend 'XYZ' as a tear-jerking movie. I'll play a preview." A character avatar is also displayed at the same time, enhancing the naturalness of the interaction visually and aurally.

[0531] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0532] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0533] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0534] [Second embodiment]

[0535] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0536] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0537] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0538] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0539] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0540] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0541] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0542] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0543] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0544] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0545] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0546] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0547] The present invention is a system that provides businesses with AI chatbots modeled after specific characters or celebrities, and aims to effectively communicate product information and advertising messages through dialogue with users. Specific embodiments of the system are described below.

[0548] Program processing overview

[0549] Terminal

[0550] 1. A corporate user accesses the management console and logs into the system.

[0551] 2. After logging in, the corporate user uploads data of the selected talent, celebrity, anime character, etc. to the system, including voice data, image data, and character scripts.

[0552] 3. The uploaded data is sent to the system's server.

[0553] server

[0554] 1. Receive the voice data and analyze it using natural language processing technology.

[0555] 2. Based on the analyzed voice data, voice synthesis is performed to generate the character's voice.

[0556] 3. Image data is analyzed to generate a character avatar, using facial recognition and animation generation technology.

[0557] 4. Analyze character scripts and incorporate unique phrases and tones into the AI ​​model.

[0558] 5. Receive product information, advertising messages, and FAQ data provided by companies, and use this information to further customize and train the AI ​​model.

[0559] 6. Allow enterprise users to preview the generated AI chatbot and make any necessary adjustments.

[0560] 7. Deploy the AI ​​chatbot once it has been adjusted to the specified platform (website, app, social media, etc.).

[0561] User

[0562] 1. An end user accesses a company's platform (website or app) and initiates a conversation with an AI chatbot.

[0563] 2. AI chatbots respond to user inquiries and convey product information and advertising messages.

[0564] 3. User feedback and interaction history are sent to the server and used for the next model update and performance improvement.

[0565] Specific examples

[0566] For example, suppose a car manufacturer wants to create an AI chatbot modeled after a popular anime character to promote a new product. In this case, the process would involve the following steps:

[0567] Corporate User Operations

[0568] 1. The automaker's representative logs in to the management console.

[0569] 2. Upload voice data, image data, and scripts (character lines and distinctive expressions) of popular anime characters.

[0570] 3. Enter detailed information and advertising messages about the company's new products.

[0571] Server Processing

[0572] 1. Analyze the uploaded audio data and generate the character's voice.

[0573] 2. Analyze the image data and create a character avatar.

[0574] 3. Analyze the character script and incorporate the character's unique tone of voice into the AI ​​model.

[0575] 4. Integrate new product information and advertising messages into the AI ​​model to train the chatbot.

[0576] 5. After training is complete, a preview of the chatbot is provided, allowing corporate users to check and adjust its operation.

[0577] 6. Once the AI ​​chatbot is fully tuned, it will be deployed on the car manufacturer's official website.

[0578] End User Experience

[0579] 1. An end user visits the car manufacturer's website.

[0580] 2. An AI chatbot based on a popular anime character responds to user inquiries and provides information about new products.

[0581] 3. The user's interaction history and feedback are sent to the server and used to improve the model next time.

[0582] This allows users to efficiently obtain product information through friendly characters, and companies can carry out effective promotions. As a new form of advertising, this system will likely play an important role in companies' marketing strategies.

[0583] The processing flow will be explained below.

[0584] Step 1:

[0585] Terminal: Corporate users log in to the management console by entering their username and password, and once authentication is successful, they are able to access the system's main screen.

[0586] Step 2:

[0587] On your device, select "Character Selection" from the Admin Console menu, select a talent, celebrity, or anime character to use, and upload the voice data, image data, and character script of the selected character to the system.

[0588] Step 3:

[0589] Server: Receives the uploaded voice data and sends it to the voice analysis engine, which converts the voice data into text (Speech-to-Text) and synthesizes speech based on that text (Text-to-Speech).

[0590] Step 4:

[0591] Server: Receives image data and sends it to the image analysis engine, which uses deep learning technology to generate an avatar of the character, which is used as a visual representation.

[0592] Step 5:

[0593] Server: Receives character scripts and sends them to a natural language processing (NLP) engine, which analyzes the scripts, extracts the character's unique phrasing and tone of voice, and trains the AI ​​model.

[0594] Step 6:

[0595] Terminal: Corporate users enter product information, advertising messages, and FAQ data into the management console and send them to the server.

[0596] Step 7:

[0597] Server: Analyzes received product information and advertising messages and integrates them into the AI ​​model, allowing the chatbot to provide appropriate information to the user.

[0598] Step 8:

[0599] Server: Integrates character voices, avatars, phrases, and product information to generate AI chatbots, which are then provided as previews for corporate users to check.

[0600] Step 9:

[0601] Device: The company user can check the chatbot's behavior on the preview screen and adjust the settings and scripts as necessary. The adjustments are sent back to the server and reflected.

[0602] Step 10:

[0603] Server: Deploy the AI ​​chatbot once it has been tuned to a specific platform (e.g., company website, app, social media), set up API endpoints using a deployment tool, and verify the integration.

[0604] Step 11:

[0605] User: An end user accesses a company's website or app and starts a conversation with an AI chatbot. The chatbot analyzes the user's inquiry and generates an appropriate response.

[0606] Step 12:

[0607] Server: Collects user interaction history and feedback and stores it in a database. The collected data is used for the next model update and performance improvement.

[0608] Through the above processing steps, businesses can utilize AI chatbots as an effective advertising tool to efficiently convey product information and advertising messages to end users.

[0609] Example 1

[0610] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0611] Companies need a user-friendly interface to effectively communicate product information and advertising messages to customers. However, with existing systems, it is difficult to create an AI chatbot modeled after a specific character or celebrity, and reproducing their unique phrasing and tone of voice requires advanced technology and a great deal of time. It is also not easy to collect user interaction data and use it to improve the model for the next time. As a result, companies' marketing strategies may not be implemented effectively.

[0612] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0613] In this invention, the server includes a means for corporate users to log in, a means for receiving voice data and image data of specific characters or celebrities, a means for analyzing the received voice data and performing speech synthesis using natural language processing technology, a means for analyzing image data and generating a character avatar using facial recognition and animation generation technology, a means for analyzing the character script and incorporating the character's unique phrasing and tone of voice into the AI ​​model, a means for receiving the company's product information, advertising messages, and FAQ data and training the AI ​​model based on them, a means for corporate users to preview and adjust the generated AI chatbot, and a means for deploying and integrating the final AI chatbot into a specific platform. This allows companies to easily create friendly AI chatbots using specific characters or celebrities, enabling them to implement effective marketing strategies. Furthermore, user interaction data can be collected and used to improve the model next time, thereby continuously improving the performance of the AI ​​chatbot.

[0614] The "means for corporate users to log in" refers to a function that allows a user to access the system's management console, enter authentication information, log in to the system, and start operations.

[0615] "Means for receiving audio data and image data of specific characters or celebrities" refers to a function for uploading audio and image files of specific characters or celebrities to the system and transmitting them to the server.

[0616] "Means for analyzing received voice data and synthesizing voice using natural language processing technology" refers to the function in which the server analyzes the voice data, extracts text, and generates the character's voice using a voice synthesis tool.

[0617] "Means for analyzing image data and generating a character avatar using facial recognition and animation generation technology" refers to a function in which the server analyzes image data, extracts the facial features of a character, and creates a moving avatar using facial recognition and animation generation technology.

[0618] "Means for analyzing character scripts and incorporating unique phrases and tones into the AI ​​model" refers to a function in which the server analyzes the text data of a character script and incorporates the character's unique phrases and tones into the AI ​​model.

[0619] "Means for receiving product information, advertising messages, and FAQ data from a company and training an AI model based on the same" refers to a function in which the server receives product information, advertising messages, and FAQ data provided by a company and uses that data to train an AI model.

[0620] "Means for corporate users to preview and adjust the generated AI chatbot" refers to a function that allows corporate users to preview the generated AI chatbot on the management console and adjust it as necessary.

[0621] "Means for deploying and integrating the final AI chatbot into a specific platform" refers to the function by which the server deploys the finalized AI chatbot into a specified platform such as a website, app, or social networking site, and begins operation.

[0622] The present invention is a system that provides businesses with AI chatbots modeled after specific characters or celebrities, and aims to effectively communicate product information and advertising messages through dialogue with users. Specific embodiments are described below.

[0623] Hardware and software used

[0624] Device: PC or tablet used by a corporate user

[0625] Software: Management console, web browser

[0626] Server: High-performance cloud server (e.g. AWS EC2, Google Cloud)

[0627] Software: Natural Language Processing (NLP) tools (e.g., Intel OpenVINO), facial recognition tools (e.g., dlib), speech synthesis tools (e.g., Google Text-to-Speech API)

[0628] Program processing overview

[0629] The system begins when a corporate user accesses the management console and logs in. The corporate user then uploads data about the talent, celebrity, or anime character they have selected to the system, including voice data, image data, and character scripts.

[0630] The device sends this data to a server, which uses NLP tools to analyze the voice data and synthesizes it. The server also analyzes the image data with facial recognition tools to generate a character avatar. The character script is analyzed to incorporate specific phrasing and tone of voice into the AI ​​model.

[0631] Product information, advertising messages, and FAQ data provided by companies are received by the server and used to train the AI ​​model. After the AI ​​model is trained, the generated AI chatbot is provided for company users to preview and make any necessary adjustments. The final AI chatbot is then deployed to the specified platform (website, app, social media, etc.).

[0632] End users access the company's platform and begin a conversation with the AI ​​chatbot. The AI ​​chatbot responds to the user's inquiries and provides product information and advertising messages. User feedback and conversation history are sent to the server and used to improve the model for the next time.

[0633] Specific examples

[0634] For example, suppose an automobile manufacturer creates an AI chatbot modeled after a popular anime character to promote a new product. In this case, the automobile manufacturer's representative operates it as follows:

[0635] 1. The automaker's representative logs in to the management console.

[0636] 2. Upload voice data, image data, and scripts (character lines and distinctive expressions) of popular anime characters.

[0637] 3. Enter detailed information and advertising messages about the company's new products.

[0638] The server performs the following process:

[0639] 1. Analyze the uploaded audio data and generate the character's voice.

[0640] 2. Analyze the image data and create a character avatar.

[0641] 3. Analyze the character script and incorporate the character's unique tone of voice into the AI ​​model.

[0642] 4. Integrate new product information and advertising messages into the AI ​​model to train the chatbot.

[0643] 5. After training is complete, a preview of the chatbot is provided, allowing corporate users to check and adjust its operation.

[0644] 6. Once the AI ​​chatbot is fully tuned, it will be deployed on the car manufacturer's official website.

[0645] The end user experience is as follows:

[0646] 1. An end user visits the car manufacturer's website.

[0647] 2. An AI chatbot based on a popular anime character responds to user inquiries and provides information about new products.

[0648] 3. The user's interaction history and feedback are sent to the server and used to improve the model next time.

[0649] Prompt Sentence Examples

[0650] "Generate an AI chatbot using popular anime characters to provide information about a company's new products."

[0651] "Enter your character script and product information to create a customized AI chatbot."

[0652] Deploy the generated AI chatbot on your website and collect user feedback.

[0653] This system allows companies to carry out effective promotions and allows users to efficiently obtain product information through friendly characters.

[0654] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0655] Step 1:

[0656] A corporate user accesses the Admin Console and logs in.

[0657] Input: Username, Password

[0658] How it works:

[0659] The terminal displays the management console screen, and the corporate user enters their username and password. The server receives the entered authentication information and authenticates it by checking it against the database. Once the user is authenticated, the server generates a dashboard page, sends it to the terminal, and displays it.

[0660] Output: Dashboard page

[0661] Step 2:

[0662] Corporate users upload voice data, image data, and character scripts of characters and celebrities.

[0663] Input: Audio data, image data, character script

[0664] How it works:

[0665] Corporate users click the "Create a new chatbot" button on the management console, select and upload files for voice data (e.g., "character voice.wav"), image data (e.g., "character image.png"), and character script (e.g., "character lines.txt"). The device sends these files to the server, which saves the received data in a specified folder and prepares it for the next process.

[0666] Output: Saved audio data, image data, character scripts

[0667] Step 3:

[0668] The server analyzes the voice data and performs voice synthesis.

[0669] Input: Saved audio data

[0670] How it works:

[0671] The server analyzes the stored voice data "character voice.wav" using an NLP tool (e.g., Intel's OpenVINO). This analysis extracts text from the voice and obtains voice characteristics. The server then uses the Google Text-to-Speech API to synthesize a character's voice based on the extracted text. The synthesized voice data is saved and used for further processing.

[0672] Output: Synthesized voice data

[0673] Step 4:

[0674] The server analyzes the image data and generates a character avatar.

[0675] Input: Saved image data

[0676] How it works:

[0677] The server analyzes the stored image data "characterimage.png" using a facial recognition tool (e.g., dlib) to extract the character's facial features (e.g., large eyes, distinctive smile). It then uses animation generation technology to generate an avatar with those facial features. The generated avatar is then used for the next process.

[0678] Output: Generated character avatar

[0679] Step 5:

[0680] The server analyzes the character script and incorporates the character's unique phrases and tone of voice into the AI ​​model.

[0681] Input: Saved character script

[0682] How it works:

[0683] The server analyzes the saved character script "character dialogue.txt." Through this analysis, the character's unique expressions and tone of voice are extracted and incorporated into the AI ​​model. This allows dialogue that reflects the character's individuality. The processed content is saved in the AI ​​model and used for subsequent processing.

[0684] Output: AI model incorporating the character's unique phrasing and tone of voice

[0685] Step 6:

[0686] The server trains the AI ​​model.

[0687] Input: Company product information, advertising messages, FAQ data, processed AI model

[0688] How it works:

[0689] The server receives product information, advertising messages, and FAQ data provided by companies and integrates this data into the AI ​​model. The server then uses NLP and machine learning algorithms to train the AI ​​model based on this data. This creates an AI chatbot whose characters are capable of appropriately conveying product information and advertising messages.

[0690] Output: A trained AI model

[0691] Step 7:

[0692] Corporate users can preview and adjust the generated AI chatbot.

[0693] Input: A trained AI model

[0694] How it works:

[0695] The server provides the generated AI chatbot for preview on the management console. Corporate users can actually try to interact with the AI ​​chatbot on the preview screen and check its behavior and what it says. If necessary, corporate users can adjust the character's specific speech content and interface and send the results to the server.

[0696] Output: The final tuned AI chatbot

[0697] Step 8:

[0698] The server deploys and integrates the final AI chatbot into a specific platform.

[0699] Input: The final tuned AI chatbot

[0700] How it works:

[0701] The server deploys the finalized AI chatbot to the designated platform, such as a website, app, or social networking site, and begins operation. The server integrates the chatbot with the platform through web hosting and APIs, making it immediately available for use.

[0702] Output: Deployed AI chatbot

[0703] Step 9:

[0704] An end user accesses a company's platform and begins interacting with an AI chatbot.

[0705] Input: End user inquiry

[0706] How it works:

[0707] An end user accesses a company's website or app and clicks the "Ask a character!" button. The AI ​​chatbot responds with "Hello, how is it?" and provides product information or advertising messages in response to the user's inquiry. The user's interaction history and feedback are sent to the server and used to improve the model for the next time.

[0708] Output: User interaction history and feedback

[0709] This system allows companies to carry out effective promotions and allows users to efficiently obtain product information through friendly characters.

[0710] (Application example 1)

[0711] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0712] The purpose of this invention is to provide a means for effectively communicating corporate product information and advertising messages. However, in conventional systems, when AI chatbots modeled after characters or celebrities are provided to companies, there is no mechanism for direct interaction with customers in physical stores. As a result, it is difficult to provide customers visiting physical stores with an experience equivalent to that of online information provision. Furthermore, there is a lack of technological means for realizing interactive dialogue that reflects the atmosphere of the physical store and customer behavior data.

[0713] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0714] In this invention, the server includes: means for receiving voice data and image data of a specific character or celebrity; means for analyzing the voice data and performing voice synthesis; means for analyzing the image data and generating a character avatar; means for analyzing the character script and incorporating the character's unique phrasing and tone of voice into an AI model; means for receiving a company's product information, advertising messages, and FAQ data and training the AI ​​model based thereon; means for previewing and adjusting the generated AI chatbot; means for deploying and integrating the final AI chatbot into a specific platform; and means for interacting with customers via terminals installed in physical stores and providing product information. This enables information provision in physical stores to be as effective as online, and provides an interactive customer experience that matches the atmosphere of the physical store.

[0715] "Audio data" refers to digital data containing the recorded voice of a specific character or celebrity.

[0716] "Image data" refers to data that stores images of specific characters or celebrities in digital format.

[0717] "Analysis" is the process of understanding the content of received audio or image data and extracting its features.

[0718] "Speech synthesis" is a technology that generates new voices based on analyzed voice data.

[0719] A "character avatar" is a digital graphic that recreates the appearance of a particular character or celebrity.

[0720] A "character script" is a collection of lines that contain the distinctive phrasing and tone of a character or celebrity.

[0721] An "AI model" is an algorithm that uses artificial intelligence, a digital model that learns and performs specific tasks.

[0722] "Training" is the process by which an AI model learns using specific data.

[0723] "Preview" is a function that allows you to check the behavior of the generated AI chatbot in advance.

[0724] "Deployment" refers to placing the generated AI chatbot so that it can operate on a specific platform.

[0725] "Platform" is a general term for the environment and services in which AI chatbots operate.

[0726] "In-store terminals" are digital devices that enable customer interaction in physical stores.

[0727] "Customers" refers to consumers and users who visit physical stores.

[0728] "Product information" is detailed information about the features and benefits of products offered by a company.

[0729] An "advertising message" is a promotional message that conveys the appeal of a product offered by a company.

[0730] "FAQ data" is a collection of frequently asked questions and their answers.

[0731] "Interaction data" is data generated when a user interacts with an AI chatbot.

[0732] "Ambient sounds" are background sounds used to recreate the atmosphere of a physical store.

[0733] "Video" refers to videos and images that visually convey the atmosphere of a physical store.

[0734] "Behavioral data" is data that records customer behavior and movements in physical stores.

[0735] MODE FOR CARRYING OUT THE INVENTION

[0736] This invention is a system that realizes an AI chatbot modeled after a specific character or celebrity, and aims to effectively convey product information and advertising messages through dialogue with customers in a physical store environment.

[0737] System Configuration and Operation

[0738] Hardware

[0739] This system uses the following hardware:

[0740] Brick-and-mortar devices: Includes devices such as tablets, kiosks, and smart glasses.

[0741] Server: A server with high-performance computing resources for training and deploying AI models.

[0742] software

[0743] The software used includes the following:

[0744] Natural Language Processing Libraries: Use high-performance AI models such as GPT-3.

[0745] Speech synthesis engine: Provides technology for synthesizing voice data.

[0746] Image analysis library: Generate character avatars using OpenCV etc.

[0747] Database system: A system for storing customer interaction and behavior data.

[0748] System Operation

[0749] The server performs the following process.

[0750] 1. Receive audio and image data of specific characters and celebrities provided by companies.

[0751] 2. Analyze the voice data and perform voice synthesis. Generate a new voice based on the analyzed voice data.

[0752] 3. Analyze the image data and generate a character avatar.

[0753] 4. Analyze character scripts and incorporate unique phrases and tones into the AI ​​model.

[0754] 5. Receives company product information, advertising messages, and FAQ data and uses them to train an AI model.

[0755] 6. Preview the generated AI chatbot and make any necessary adjustments.

[0756] 7. The final AI chatbot is deployed to a specific platform, i.e., a terminal installed in a physical store, and begins interacting with customers.

[0757] Specific examples

[0758] For example, consider a car dealership that installs an AI chatbot modeled after a popular anime character in its physical store to promote a new product.

[0759] Dealers log in to the management console and upload voice data, image data, and scripts (character lines and distinctive expressions) for specific animated characters.

[0760] The server analyzes this data and generates an AI chatbot.

[0761] The generated chatbot is deployed on tablet devices installed in the car dealership's physical store.

[0762] When customers visit the store, animated characters will provide information about cars and related products and introduce promotions.

[0763] Prompt Sentence Examples

[0764] "Please tell us the character's name and details of the new car model for visitors."

[0765] "Please tell me the character's name and the latest promotional information."

[0766] In this way, the system of the present invention enables interactive dialogue through friendly characters even in brick-and-mortar stores, and supports the provision of effective product information and promotional activities.

[0767] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0768] Step 1:

[0769] Corporate users access the management console and log in to the system. They then upload voice data, image data, and character scripts of specific characters or celebrities to the system. The input data is voice data, image data, and script data, which is then sent to the server.

[0770] Step 2:

[0771] The server analyzes the received voice data, extracts features from the voice data using natural language processing technology, and then generates the character's unique voice using a voice synthesis engine. The input data here is the voice data, and the output is a synthesized voice file.

[0772] Step 3:

[0773] The server analyzes the image data and generates a character avatar. It uses an image analysis library to extract features from the received image data and create an avatar. The input data is the image data, and the output is the generated character avatar.

[0774] Step 4:

[0775] The server analyzes the character script and incorporates the specific phrasing and tone of voice into the AI ​​model. A natural language processing model is used to train the phrasing and tone of voice of the script data. The input data here is the script data, and the output is the trained AI model.

[0776] Step 5:

[0777] The server receives product information, advertising messages, and FAQ data provided by the company and further trains the AI ​​model based on this information. The input data are product information, advertising messages, and FAQ data, and the output is an AI model that integrates product information.

[0778] Step 6:

[0779] Corporate users can preview the generated AI chatbot and make any necessary adjustments. They can check its operation on a device installed in a physical store and check the voice and avatar behavior. The input data is a trained AI model, a synthesized voice file, and a character avatar, and the output is a previewed AI chatbot.

[0780] Step 7:

[0781] The final AI chatbot is deployed to a terminal in a physical store and begins interacting with customers. A deployment tool is used to place the AI ​​chatbot on the terminal and prepare a system for interacting with customers. The input data is the previewed AI chatbot, and the output is an AI chatbot that interacts with customers in a physical store.

[0782] Step 8:

[0783] A user (customer) visits a physical store and initiates a conversation with an AI chatbot via a terminal device. The terminal device receives the customer's input and sends it to the server. The input data is the customer's question, and the output is the AI ​​chatbot's response.

[0784] Step 9:

[0785] The server collects customer interaction history and feedback and uses it for the next model update and performance improvement. This data is stored in a database and machine learning algorithms are used to improve the model. The input data is the customer interaction history and feedback, and the output is an updated AI model.

[0786] These steps enable the server and terminal to work together to provide effective information and customer experiences even in physical stores.

[0787] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0788] This invention provides a system that combines an emotion engine with an AI chatbot modeled after a specific character or celebrity, with the aim of effectively communicating product information and advertising messages through dialogue with users. It also recognizes the user's emotions and dynamically adjusts responses based on them, enabling a more natural and engaging conversational experience.

[0789] Program processing overview

[0790] Terminal

[0791] 1. A corporate user logs in to the management console. After entering their username and password and successful authentication, they are able to access the system's main screen.

[0792] 2. After logging in, the corporate user uploads the voice data, image data, and character script of the selected talent, celebrity, or anime character to the system.

[0793] 3. Perform the necessary initial setup for the emotion engine and adjust the emotion model parameters.

[0794] server

[0795] 1. The voice data is received and analyzed using natural language processing technology. The analysis engine converts the voice data into text and synthesizes speech based on that text data.

[0796] 2. Image data is received and a character avatar is generated using an image analysis engine.

[0797] 3. The character script is received and analyzed by a natural language processing (NLP) engine to extract specific phrases and tone of voice, which are then trained into an AI model.

[0798] 4. Receive product information, advertising messages, and FAQ data provided by companies, and use them to customize and train AI models.

[0799] 5. Use the emotion engine to analyze and recognize emotions from user input data.

[0800] 6. Select and set scripts for the AI ​​chatbot to dynamically adjust responses and tone based on emotion recognition results.

[0801] 7. Allow enterprise users to preview the generated AI chatbot and make any necessary adjustments.

[0802] 8. Deploy the AI ​​chatbot once it has been tuned to a specific platform.

[0803] User

[0804] 1. An end user accesses a company's platform and initiates a conversation with an AI chatbot.

[0805] 2. The emotion engine analyzes the emotions in the user's input data (text and voice) and adjusts the chatbot's responses accordingly.

[0806] 3. The AI ​​chatbot provides appropriate product information and advertising messages in response to user inquiries.

[0807] 4. The dialogue history and feedback are sent to the server and used to improve the model and performance next time.

[0808] Specific examples

[0809] Corporate User Operations

[0810] Let's take the example of a fashion brand creating an AI chatbot modeled after a popular character to launch a new advertising campaign.

[0811] 1. A brand marketer logs in to the Admin console.

[0812] 2. Upload the voice data, image data, and character scripts of popular characters and set the parameters of the emotion engine.

[0813] 3. Enter product information and advertising messages into the system.

[0814] Server Processing

[0815] 1. Analyze the uploaded voice data and perform voice synthesis.

[0816] 2. Analyze the image data and generate a character avatar.

[0817] 3. Analyze the character script and train the AI ​​model to recognize specific phrases and tones.

[0818] 4. Incorporate received product information and advertising messages into the model to train the chatbot.

[0819] 5. Use an emotion engine to analyze user emotions and dynamically adjust responses based on the results.

[0820] 6. Check the behavior of your AI chatbot with the preview function and make any necessary adjustments.

[0821] 7. After tuning, deploy the AI ​​chatbot on the brand's official website.

[0822] End User Experience

[0823] 1. An end user visits a brand's website and initiates a conversation with an AI chatbot.

[0824] 2. The emotion engine recognizes emotions from the user's input data, and the chatbot responds according to those emotions.

[0825] 3. Users can effectively obtain information about new products and promotions through friendly characters.

[0826] 4. User feedback and interaction history are sent to the server and used to improve the model next time.

[0827] In this way, AI chatbots that utilize an emotion engine allow users to enjoy a more natural and engaging conversational experience, and businesses can realize effective advertising tools. This system will provide new value in advertising and marketing.

[0828] The processing flow will be explained below.

[0829] Step 1:

[0830] Terminal: Corporate users log in to the management console by entering their username and password, and once authentication is successful, they are able to access the system's main screen.

[0831] Step 2:

[0832] On your device, select "Character Selection" from the Admin Console menu, select a talent, celebrity, or anime character to use, and upload the voice data, image data, and character script of the selected character to the system.

[0833] Step 3:

[0834] Device: Perform the necessary initial setup for the emotion engine and adjust the parameters of the emotion model, for example, setting which emotions (joy, sadness, anger, etc.) to recognize.

[0835] Step 4:

[0836] Server: Receives the uploaded voice data and sends it to the voice analysis engine. The analysis engine converts the voice data into text (Speech-to-Text). Text-to-Speech synthesis is performed based on the converted text data to generate the character's voice.

[0837] Step 5:

[0838] Server: Receives image data and sends it to the image analysis engine, which uses deep learning technology to generate a character avatar. The generated avatar is used as the visual representation of the chatbot.

[0839] Step 6:

[0840] Server: Receives character scripts and sends them to a natural language processing (NLP) engine, which analyzes the scripts, extracts the character's unique phrasing and tone of voice, and trains the AI ​​model.

[0841] Step 7:

[0842] Terminal: Corporate users enter product information, advertising messages, and FAQ data into the management console and send them to the server.

[0843] Step 8:

[0844] Server: Analyzes received product information and advertising messages and integrates them into the AI ​​model, allowing the chatbot to provide appropriate information to the user.

[0845] Step 9:

[0846] Server: Analyzes and recognizes emotions from user input data (text and voice) using an emotion engine. For example, it uses natural language processing technology to classify the user's emotional state into categories such as Joy, Anger, and Sadness.

[0847] Step 10:

[0848] Server: Dynamically adjusts the AI ​​chatbot's response and tone based on the results of emotion recognition. For example, if the user is angry, respond with a calm tone; if the user is happy, respond with a cheerful tone.

[0849] Step 11:

[0850] Server: Integrates character voices, avatars, phrases, emotion recognition, and product information to generate an AI chatbot. The generated chatbot is provided as a preview for corporate users to check.

[0851] Step 12:

[0852] Device: The company user can check the chatbot's behavior on the preview screen and adjust the settings and scripts as necessary. The adjustments are sent back to the server and reflected.

[0853] Step 13:

[0854] Server: Deploy the finalized AI chatbot to a specific platform (e.g., company website, app, social media), set up API endpoints using a deployment tool, and verify the integration.

[0855] Step 14:

[0856] User: An end user visits a company's website or app and initiates a conversation with an AI chatbot. When the user types a question or comment, the emotion engine analyzes their emotional state and the chatbot generates an appropriate response.

[0857] Step 15:

[0858] Server: Collects user interaction history and feedback and stores it in a database. The collected data is used for the next model update and performance improvement.

[0859] Through these processing steps, businesses can utilize AI chatbots combined with emotion engines to provide end users with a more natural and engaging conversational experience, while also serving as an effective advertising tool.

[0860] Example 2

[0861] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0862] Conventional chatbot systems lack the ability to recognize emotions in user interactions and dynamically adjust responses based on those emotions. As a result, users do not enjoy a natural conversational experience, making it difficult for companies to convey effective advertising messages. Furthermore, when modeling characters or celebrities, it is difficult to reproduce their unique expressions and tone of voice, which often results in low user engagement.

[0863] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0864] In this invention, the server includes means for receiving voice data and image data of a specific character or celebrity, means for analyzing the received voice data and performing voice synthesis, means for analyzing the image data and generating a character avatar, means for analyzing the character script and incorporating the character's unique expressions and tone of voice into an AI model, means for receiving a company's product information, advertising messages, and FAQ data and training the AI ​​model based thereon, means for recognizing emotions from user input data and dynamically adjusting responses using an emotion engine, means for previewing and adjusting the generated AI chatbot, and means for deploying and integrating the final AI chatbot into a specific platform. This allows users to enjoy a more natural and engaging conversational experience and enables companies to deliver effective advertising messages.

[0865] "Audio data" refers to data in which audio information is recorded in digital format, and includes the voice of a specific character or celebrity.

[0866] "Image data" refers to visual information recorded in digital format, and includes images of specific characters or celebrities.

[0867] A "character script" is text data that records the lines, expressions, and phrases used by a specific character.

[0868] "Speech synthesis" is a technology that generates speech based on text data, and is performed using natural language processing technology.

[0869] A "character avatar" is a digital representation of a particular character that is generated based on analyzed image data.

[0870] "Distinctive expressions and tone of voice" refers to the unique ways of expression and speaking used by particular characters or celebrities.

[0871] An "AI model" is a mathematical model for realizing artificial intelligence, and specifically includes algorithms that utilize machine learning and deep learning.

[0872] An "emotion engine" is the part of the system that recognizes emotions from user input data and dynamically adjusts responses based on the results.

[0873] "Platform" refers to the digital environment, such as a particular website or application, in which an AI chatbot is deployed.

[0874] "Deployment" means placing the developed system or software in a production environment and making it actually function.

[0875] The present invention provides a system that combines an emotion engine with an AI chatbot modeled after a specific character or celebrity. The purpose is to effectively communicate product information and advertising messages through dialogue with users. Specific embodiments of this system are described in detail below.

[0876] Hardware and Software Configuration

[0877] Terminal

[0878] The terminal is used by corporate users and is a device for accessing the management console. This management console includes functions for uploading data, setting parameters, and previewing. Specifically, since it is accessed via a web browser, the terminal can be used on a wide range of operating systems, including Windows, macOS, and Linux.

[0879] server

[0880] The server analyzes the data and generates the AI ​​chatbot. The server is equipped with the following main software and engines:

[0881] Natural Language Processing (NLP) engines (e.g., spaCy, NLTK)

[0882] Image analysis engine (e.g. OpenCV, TensorFlow)

[0883] Speech synthesis engine (e.g., Google Text-to-Speech API, Amazon Polly)

[0884] Emotion engine (e.g. IBM Watson Emotion Analysis)

[0885] Data processing and calculation

[0886] Voice data analysis and voice synthesis

[0887] The server receives voice data uploaded by corporate users from their devices and analyzes it using a natural language processing (NLP) engine. The analyzed text data is then converted into speech by a speech synthesis engine.

[0888] Image data analysis and avatar generation

[0889] The server receives the uploaded image data and uses an image analysis engine to generate a character avatar, which is used as the visual for the chatbot.

[0890] Character script analysis and model training

[0891] The server analyzes the character scripts received from corporate users, extracts specific phrases and tones using a natural language processing (NLP) engine, and trains the AI ​​model. This trained model is reflected in the chatbot's dialogue.

[0892] Integrating product information and advertising messages

[0893] Product information, advertising messages, and FAQ data provided by corporate users are received by the server, and the AI ​​model is customized based on this data, allowing marketing information to be naturally incorporated as part of the conversation.

[0894] Emotion recognition and response regulation

[0895] The emotion engine recognizes emotions from user input data and dynamically adjusts the tone of responses based on the results. For example, if a user expresses positive emotions, the chatbot will respond with a positive response.

[0896] Specific examples

[0897] Let's take the example of a fashion brand creating an AI chatbot modeled after a popular character to launch a new advertising campaign. The brand's marketer logs into the management console and uploads the popular character's voice data, image data, and character script to the system. This allows the server to perform voice, image, and NLP analysis to set the character avatar and its unique phrasing and tone of voice. Next, they adjust the emotion engine parameters and enter product information and advertising messages into the system. The fully customized AI chatbot can be checked through the preview function, and after any necessary adjustments are made, it is deployed to the brand's official website.

[0898] A user accesses a brand's website and begins a conversation with an AI chatbot. The emotion engine recognizes emotions from the user's input data and returns appropriate responses, providing a more natural and friendly conversational experience. For example, if a user inputs, "I want a new jacket," the chatbot will provide specific information such as, "The new jacket is from this year's fall / winter collection and is excellent for keeping you warm. It also comes in a wide variety of colors."

[0899] Prompt Sentence Examples

[0900] Here are some examples of prompts to input to a generative AI model:

[0901] 1. "I want to create an AI chatbot that uses an emotion engine. First, please tell me the steps to upload the character's voice and image data."

[0902] 2. "Give me an example of how a chatbot can tailor its response to a user's emotion when they ask a question."

[0903] 3. "How can a fashion brand effectively communicate new product information through friendly characters?"

[0904] By using these prompts, users can obtain more detailed instructions and explanations about specific tasks and settings, providing a more natural and engaging interaction experience for users and enabling companies to effectively communicate their advertising messages.

[0905] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0906] Step 1:

[0907] Terminal

[0908] A corporate user logs in to the management console. As input, they enter their username and password and click the "Login" button. The system receives this and performs authentication. If successful, they are redirected to the main screen. The output is access to the system's main screen.

[0909] Step 2:

[0910] Terminal

[0911] A company user uploads the voice data, image data, and character script of a character. As input, they specify these files in the respective upload forms and click the "Upload" button. The files are sent to the server. The output is the data correctly saved on the server.

[0912] Step 3:

[0913] Terminal

[0914] Corporate users configure the emotion engine initially and adjust the emotion model parameters. As input, they set emotion recognition accuracy and emotion category parameters on the settings screen. The system receives this and saves the emotion engine configuration data. The output is the adjusted emotion model.

[0915] Step 4:

[0916] server

[0917] The server analyzes the voice data and synthesizes the voice. The uploaded voice data is used as input. The server converts the voice to text using a natural language processing (NLP) engine, and then generates a new voice using a speech synthesis engine based on the text. The output is the generated voice data.

[0918] Step 5:

[0919] server

[0920] The server analyzes the image data and generates a character avatar. The uploaded image data is used as input. The server uses an image analysis engine to generate a 3D avatar from the image. The output is the generated character avatar.

[0921] Step 6:

[0922] server

[0923] The server analyzes the character script and trains the AI ​​model to recognize specific phrases and tones. The uploaded character script is used as input. A natural language processing (NLP) engine analyzes the script, extracts specific patterns, and trains the AI ​​model. The output is a trained AI model.

[0924] Step 7:

[0925] server

[0926] The server receives product information, advertising messages, and FAQ data provided by companies and customizes the AI ​​model. These data are used as input. The server analyzes them and uses them as necessary training data to strengthen the AI ​​model. The output is a customized AI model.

[0927] Step 8:

[0928] server

[0929] The server uses an emotion engine to recognize emotions from user input data and dynamically adjust responses. The input is text and voice data from the user's interaction with the chatbot. The server analyzes this using the emotion engine and adjusts the chatbot's response based on the results. The output is a response based on the emotion.

[0930] Step 9:

[0931] Terminal

[0932] Corporate users can preview the generated AI chatbot and make adjustments as needed. The input is a simulated interaction on the preview screen. Based on user feedback, the server adjusts the settings. The output is the adjusted chatbot.

[0933] Step 10:

[0934] server

[0935] The server deploys and integrates the final AI chatbot on a specific platform. The completed chatbot data is used as input. A deployment tool is used to integrate the chatbot into a specified website or application. The output is a chatbot running in a production environment.

[0936] Step 11:

[0937] User

[0938] An end user accesses a company's platform and starts a conversation with an AI chatbot. As input, they type a message into the chat window of a website or application. The system receives this and starts a conversation. The output is the content of the conversation with the chatbot.

[0939] Step 12:

[0940] server

[0941] The server dynamically adjusts responses based on the user's emotions. The inputs are the user's speech and the results of emotion analysis. Based on this, the server selects an appropriate response tone, and the chatbot responds to the user. The output is a response adjusted according to the user's emotions.

[0942] Step 13:

[0943] server

[0944] The server collects the interaction history and feedback and uses it to improve the model. The user's interaction history and feedback are used as input. These data are used for the next model training. The output is an improved AI model.

[0945] (Application example 2)

[0946] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0947] In recent years, AI chatbots have been used on various platforms, but to improve the user experience, more natural and emotionally adaptive responses are required. Conventional systems lack the ability to recognize and dynamically adjust to user emotions, and responses are often mechanical. Furthermore, content distribution services lack mechanisms for providing personalized content based on user emotions. To address these issues, the present invention aims to provide an AI chatbot system combined with an emotion recognition engine to improve the user experience.

[0948] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving voice data and image data of a specific character or celebrity; means for analyzing the received voice data and performing voice synthesis; means for analyzing image data and generating a character avatar; means for receiving company product information, advertising messages, and FAQ data and training an AI model based on them; means for analyzing a user's emotions using an emotion recognition engine and adjusting responses based on the analyzed emotions; means for processing user input data in a content distribution service platform and providing personalized content; means for voice synthesizing a response based on the emotion of a specific character; and means for visualizing a character avatar and coordinating the visual display with audio playback. This makes it possible to provide appropriate responses and personalized content according to the user's emotions.

[0949] "Voice data" refers to information that digitally records the voice of a user or a specific character.

[0950] "Image data" refers to visual information of characters or celebrities recorded in digital format.

[0951] "Speech synthesis" is a technology that uses a computer program to generate a human voice based on text data.

[0952] A "character avatar" is a digital icon or persona that visually represents a particular character.

[0953] A "character script" is a collection of lines and responses that a character utters, each with a specific phrase and tone of voice.

[0954] An "AI model" is a collection of algorithms trained to perform a specific task using artificial intelligence techniques.

[0955] An "emotion recognition engine" is an algorithm or tool that analyzes emotions from user input data and identifies those emotions.

[0956] "Personalized content" is digital content that is individually tailored based on a user's interests and preferences.

[0957] "Deployment" is the process of implementing software or an application on a particular platform.

[0958] A "server" is a computer system that provides services to other computers over a network.

[0959] A "platform" is the underlying hardware or software environment on which an application runs.

[0960] A "natural language processing model" is an algorithm used by computers to understand and generate human language.

[0961] The system for realizing the present invention includes the following components and means: Specific components and their respective processes will be described in detail below.

[0962] System Configuration

[0963] This system uses the following hardware and software:

[0964] Hardware:

[0965] Smartphone

[0966] Head-mounted display (HMD)

[0967] server

[0968] software:

[0969] Google Cloud Natural Language API

[0970] Google Cloud Text-to-Speech API

[0971] Unity

[0972] Program processing overview

[0973] The server has the following features:

[0974] 1. Receiving and analyzing audio and image data:

[0975] It receives voice and image data of characters and celebrities, converts the voice data into text using the Google Cloud Natural Language API, and then synthesizes speech based on that text using the Google Cloud Text-to-Speech API.

[0976] Image data is used to generate and visualize character avatars using Unity.

[0977] 2. Dynamic emotion recognition and response adjustment:

[0978] The user's input data (text) is subjected to sentiment analysis using the Google Cloud Natural Language API, and the sentiment score and strength of the sentiment are calculated.

[0979] Based on this, the character's response is selected and speech synthesis is performed using the Google Cloud Text-to-Speech API.

[0980] The terminal has the following features:

[0981] 1. Providing interaction:

[0982] It provides a user interface (UI) that allows users to initiate a dialogue with the AI ​​chatbot via a smartphone or HMD.

[0983] Visualize a character avatar and play synthesized voice in sync.

[0984] The user interacts with:

[0985] 1. Start the conversation:

[0986] Initiate a conversation with the AI ​​chatbot using an application on your smartphone or HMD.

[0987] Based on user input (text or voice), the system performs sentiment analysis and provides responses and content recommendations based on that.

[0988] Specific examples

[0989] For example, when a user expresses their opinion about a particular movie on a movie streaming service, the service will suggest movies based on the sentiment analysis results. The prompt sentence is as follows:

[0990] Example prompt sentence:

[0991] Input Text: "The movie I saw recently was very moving. It made me cry."

[0992] Emotion score: high positive emotion, strong emotional intensity

[0993] AI chatbot response: "So you like inspiring movies! I recommend 'XYZ' as a tear-jerking movie. I'll play a preview for you."

[0994] In this way, by providing content that corresponds to the user's emotions, it is possible to build a system that can provide a more natural and attractive interactive experience.

[0995] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0996] Step 1:

[0997] Receiving and analyzing audio and image data

[0998] The server receives audio data and image data of specific characters and celebrities provided by the company.

[0999] The received audio data is sent to the Google Cloud Natural Language API and converted into text, which is later used for speech synthesis.

[1000] The image data is analyzed by Unity and a character avatar is generated.

[1001] Input: Voice and image data provided by the company

[1002] Output: Speech data converted to text and generated character avatar

[1003] Step 2:

[1004] Emotion recognition engine for analyzing user emotions

[1005] The user inputs text or voice using a smartphone or HMD.

[1006] The user's input data (text or voice) is sent to the server and analyzed for sentiment scores and sentiment strength using the Google Cloud Natural Language API.

[1007] The results of this sentiment analysis are used to dynamically tailor responses.

[1008] Input: User-entered data (text or voice)

[1009] Output: Sentiment score and intensity of the emotion

[1010] Step 3:

[1011] Response generation and speech synthesis

[1012] The server selects an appropriate response based on the emotion score and the intensity of the emotion.

[1013] The selected response text is synthesized using the Google Cloud Text-to-Speech API and output in the appropriate character's voice.

[1014] The synthesized voice file is sent to the user's terminal.

[1015] Input: Sentiment analysis results, response text

[1016] Output: A synthesized response

[1017] Step 4:

[1018] Character Avatar Visualization

[1019] The server transmits the character avatar along with the synthesized voice to the user's terminal.

[1020] The user's device uses Unity to synchronize the display of the character avatar and the playback of the audio.

[1021] This allows users to experience a natural interaction both visually and aurally.

[1022] Input: Character avatar, audio file

[1023] Output: Visualized avatar and audio playback

[1024] Step 5:

[1025] Collection of conversation history and feedback

[1026] The server collects user interaction history and feedback to help improve the model in future.

[1027] The collected data is used to train AI models.

[1028] Input: Dialogue history, user feedback

[1029] Output: New training data

[1030] Specific working example:

[1031] For example, if a user enters the text, "The movie I saw recently was very moving. It brought me to tears," the system performs sentiment analysis on the text and detects high positive and strong emotional intensity. Based on the results, the system generates a response to the user via synthesized voice: "I see you like moving movies! I recommend 'XYZ' as a tear-jerking movie. I'll play a preview." A character avatar is also displayed at the same time, enhancing the naturalness of the interaction visually and aurally.

[1032] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1033] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1034] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1035] [Third embodiment]

[1036] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1037] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1038] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1039] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1040] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1041] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1042] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1043] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1044] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1045] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1046] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1047] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1048] The present invention is a system that provides businesses with AI chatbots modeled after specific characters or celebrities, and aims to effectively communicate product information and advertising messages through dialogue with users. Specific embodiments of the system are described below.

[1049] Program processing overview

[1050] Terminal

[1051] 1. A corporate user accesses the management console and logs into the system.

[1052] 2. After logging in, the corporate user uploads data of the selected talent, celebrity, anime character, etc. to the system, including voice data, image data, and character scripts.

[1053] 3. The uploaded data is sent to the system's server.

[1054] server

[1055] 1. Receive the voice data and analyze it using natural language processing technology.

[1056] 2. Based on the analyzed voice data, voice synthesis is performed to generate the character's voice.

[1057] 3. Image data is analyzed to generate a character avatar, using facial recognition and animation generation technology.

[1058] 4. Analyze character scripts and incorporate unique phrases and tones into the AI ​​model.

[1059] 5. Receive product information, advertising messages, and FAQ data provided by companies, and use this information to further customize and train the AI ​​model.

[1060] 6. Allow enterprise users to preview the generated AI chatbot and make any necessary adjustments.

[1061] 7. Deploy the AI ​​chatbot once it has been adjusted to the specified platform (website, app, social media, etc.).

[1062] User

[1063] 1. An end user accesses a company's platform (website or app) and initiates a conversation with an AI chatbot.

[1064] 2. AI chatbots respond to user inquiries and convey product information and advertising messages.

[1065] 3. User feedback and interaction history are sent to the server and used for the next model update and performance improvement.

[1066] Specific examples

[1067] For example, suppose a car manufacturer wants to create an AI chatbot modeled after a popular anime character to promote a new product. In this case, the process would involve the following steps:

[1068] Corporate User Operations

[1069] 1. The automaker's representative logs in to the management console.

[1070] 2. Upload voice data, image data, and scripts (character lines and distinctive expressions) of popular anime characters.

[1071] 3. Enter detailed information and advertising messages about the company's new products.

[1072] Server Processing

[1073] 1. Analyze the uploaded audio data and generate the character's voice.

[1074] 2. Analyze the image data and create a character avatar.

[1075] 3. Analyze the character script and incorporate the character's unique tone of voice into the AI ​​model.

[1076] 4. Integrate new product information and advertising messages into the AI ​​model to train the chatbot.

[1077] 5. After training is complete, a preview of the chatbot is provided, allowing corporate users to check and adjust its operation.

[1078] 6. Once the AI ​​chatbot is fully tuned, it will be deployed on the car manufacturer's official website.

[1079] End User Experience

[1080] 1. An end user visits the car manufacturer's website.

[1081] 2. An AI chatbot based on a popular anime character responds to user inquiries and provides information about new products.

[1082] 3. The user's interaction history and feedback are sent to the server and used to improve the model next time.

[1083] This allows users to efficiently obtain product information through friendly characters, and companies can carry out effective promotions. As a new form of advertising, this system will likely play an important role in companies' marketing strategies.

[1084] The processing flow will be explained below.

[1085] Step 1:

[1086] Terminal: Corporate users log in to the management console by entering their username and password, and once authentication is successful, they are able to access the system's main screen.

[1087] Step 2:

[1088] On your device, select "Character Selection" from the Admin Console menu, select a talent, celebrity, or anime character to use, and upload the voice data, image data, and character script of the selected character to the system.

[1089] Step 3:

[1090] Server: Receives the uploaded voice data and sends it to the voice analysis engine, which converts the voice data into text (Speech-to-Text) and synthesizes speech based on that text (Text-to-Speech).

[1091] Step 4:

[1092] Server: Receives image data and sends it to the image analysis engine, which uses deep learning technology to generate an avatar of the character, which is used as a visual representation.

[1093] Step 5:

[1094] Server: Receives character scripts and sends them to a natural language processing (NLP) engine, which analyzes the scripts, extracts the character's unique phrasing and tone of voice, and trains the AI ​​model.

[1095] Step 6:

[1096] Terminal: Corporate users enter product information, advertising messages, and FAQ data into the management console and send them to the server.

[1097] Step 7:

[1098] Server: Analyzes received product information and advertising messages and integrates them into the AI ​​model, allowing the chatbot to provide appropriate information to the user.

[1099] Step 8:

[1100] Server: Integrates character voices, avatars, phrases, and product information to generate AI chatbots, which are then provided as previews for corporate users to check.

[1101] Step 9:

[1102] Device: The company user can check the chatbot's behavior on the preview screen and adjust the settings and scripts as necessary. The adjustments are sent back to the server and reflected.

[1103] Step 10:

[1104] Server: Deploy the AI ​​chatbot once it has been tuned to a specific platform (e.g., company website, app, social media), set up API endpoints using a deployment tool, and verify the integration.

[1105] Step 11:

[1106] User: An end user accesses a company's website or app and starts a conversation with an AI chatbot. The chatbot analyzes the user's inquiry and generates an appropriate response.

[1107] Step 12:

[1108] Server: Collects user interaction history and feedback and stores it in a database. The collected data is used for the next model update and performance improvement.

[1109] Through the above processing steps, businesses can utilize AI chatbots as an effective advertising tool to efficiently convey product information and advertising messages to end users.

[1110] Example 1

[1111] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1112] Companies need a user-friendly interface to effectively communicate product information and advertising messages to customers. However, with existing systems, it is difficult to create an AI chatbot modeled after a specific character or celebrity, and reproducing their unique phrasing and tone of voice requires advanced technology and a great deal of time. It is also not easy to collect user interaction data and use it to improve the model for the next time. As a result, companies' marketing strategies may not be implemented effectively.

[1113] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1114] In this invention, the server includes a means for corporate users to log in, a means for receiving voice data and image data of specific characters or celebrities, a means for analyzing the received voice data and performing speech synthesis using natural language processing technology, a means for analyzing image data and generating a character avatar using facial recognition and animation generation technology, a means for analyzing the character script and incorporating the character's unique phrasing and tone of voice into the AI ​​model, a means for receiving the company's product information, advertising messages, and FAQ data and training the AI ​​model based on them, a means for corporate users to preview and adjust the generated AI chatbot, and a means for deploying and integrating the final AI chatbot into a specific platform. This allows companies to easily create friendly AI chatbots using specific characters or celebrities, enabling them to implement effective marketing strategies. Furthermore, user interaction data can be collected and used to improve the model next time, thereby continuously improving the performance of the AI ​​chatbot.

[1115] The "means for corporate users to log in" refers to a function that allows a user to access the system's management console, enter authentication information, log in to the system, and start operations.

[1116] "Means for receiving audio data and image data of specific characters or celebrities" refers to a function for uploading audio and image files of specific characters or celebrities to the system and transmitting them to the server.

[1117] "Means for analyzing received voice data and synthesizing voice using natural language processing technology" refers to the function in which the server analyzes the voice data, extracts text, and generates the character's voice using a voice synthesis tool.

[1118] "Means for analyzing image data and generating a character avatar using facial recognition and animation generation technology" refers to a function in which the server analyzes image data, extracts the facial features of a character, and creates a moving avatar using facial recognition and animation generation technology.

[1119] "Means for analyzing character scripts and incorporating unique phrases and tones into the AI ​​model" refers to a function in which the server analyzes the text data of a character script and incorporates the character's unique phrases and tones into the AI ​​model.

[1120] "Means for receiving product information, advertising messages, and FAQ data from a company and training an AI model based on the same" refers to a function in which the server receives product information, advertising messages, and FAQ data provided by a company and uses that data to train an AI model.

[1121] "Means for corporate users to preview and adjust the generated AI chatbot" refers to a function that allows corporate users to preview the generated AI chatbot on the management console and adjust it as necessary.

[1122] "Means for deploying and integrating the final AI chatbot into a specific platform" refers to the function by which the server deploys the finalized AI chatbot into a specified platform such as a website, app, or social networking site, and begins operation.

[1123] The present invention is a system that provides businesses with AI chatbots modeled after specific characters or celebrities, and aims to effectively communicate product information and advertising messages through dialogue with users. Specific embodiments are described below.

[1124] Hardware and software used

[1125] Device: PC or tablet used by a corporate user

[1126] Software: Management console, web browser

[1127] Server: High-performance cloud server (e.g. AWS EC2, Google Cloud)

[1128] Software: Natural Language Processing (NLP) tools (e.g., Intel OpenVINO), facial recognition tools (e.g., dlib), speech synthesis tools (e.g., Google Text-to-Speech API)

[1129] Program processing overview

[1130] The system begins when a corporate user accesses the management console and logs in. The corporate user then uploads data about the talent, celebrity, or anime character they have selected to the system, including voice data, image data, and character scripts.

[1131] The device sends this data to a server, which uses NLP tools to analyze the voice data and synthesizes it. The server also analyzes the image data with facial recognition tools to generate a character avatar. The character script is analyzed to incorporate specific phrasing and tone of voice into the AI ​​model.

[1132] Product information, advertising messages, and FAQ data provided by companies are received by the server and used to train the AI ​​model. After the AI ​​model is trained, the generated AI chatbot is provided for company users to preview and make any necessary adjustments. The final AI chatbot is then deployed to the specified platform (website, app, social media, etc.).

[1133] End users access the company's platform and begin a conversation with the AI ​​chatbot. The AI ​​chatbot responds to the user's inquiries and provides product information and advertising messages. User feedback and conversation history are sent to the server and used to improve the model for the next time.

[1134] Specific examples

[1135] For example, suppose an automobile manufacturer creates an AI chatbot modeled after a popular anime character to promote a new product. In this case, the automobile manufacturer's representative operates it as follows:

[1136] 1. The automaker's representative logs in to the management console.

[1137] 2. Upload voice data, image data, and scripts (character lines and distinctive expressions) of popular anime characters.

[1138] 3. Enter detailed information and advertising messages about the company's new products.

[1139] The server performs the following process:

[1140] 1. Analyze the uploaded audio data and generate the character's voice.

[1141] 2. Analyze the image data and create a character avatar.

[1142] 3. Analyze the character script and incorporate the character's unique tone of voice into the AI ​​model.

[1143] 4. Integrate new product information and advertising messages into the AI ​​model to train the chatbot.

[1144] 5. After training is complete, a preview of the chatbot is provided, allowing corporate users to check and adjust its operation.

[1145] 6. Once the AI ​​chatbot is fully tuned, it will be deployed on the car manufacturer's official website.

[1146] The end user experience is as follows:

[1147] 1. An end user visits the car manufacturer's website.

[1148] 2. An AI chatbot based on a popular anime character responds to user inquiries and provides information about new products.

[1149] 3. The user's interaction history and feedback are sent to the server and used to improve the model next time.

[1150] Prompt Sentence Examples

[1151] "Generate an AI chatbot using popular anime characters to provide information about a company's new products."

[1152] "Enter your character script and product information to create a customized AI chatbot."

[1153] Deploy the generated AI chatbot on your website and collect user feedback.

[1154] This system allows companies to carry out effective promotions and allows users to efficiently obtain product information through friendly characters.

[1155] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1156] Step 1:

[1157] A corporate user accesses the Admin Console and logs in.

[1158] Input: Username, Password

[1159] How it works:

[1160] The terminal displays the management console screen, and the corporate user enters their username and password. The server receives the entered authentication information and authenticates it by checking it against the database. Once the user is authenticated, the server generates a dashboard page, sends it to the terminal, and displays it.

[1161] Output: Dashboard page

[1162] Step 2:

[1163] Corporate users upload voice data, image data, and character scripts of characters and celebrities.

[1164] Input: Audio data, image data, character script

[1165] How it works:

[1166] Corporate users click the "Create a new chatbot" button on the management console, select and upload files for voice data (e.g., "character voice.wav"), image data (e.g., "character image.png"), and character script (e.g., "character lines.txt"). The device sends these files to the server, which saves the received data in a specified folder and prepares it for the next process.

[1167] Output: Saved audio data, image data, character scripts

[1168] Step 3:

[1169] The server analyzes the voice data and performs voice synthesis.

[1170] Input: Saved audio data

[1171] How it works:

[1172] The server analyzes the stored voice data "character voice.wav" using an NLP tool (e.g., Intel's OpenVINO). This analysis extracts text from the voice and obtains voice characteristics. The server then uses the Google Text-to-Speech API to synthesize a character's voice based on the extracted text. The synthesized voice data is saved and used for further processing.

[1173] Output: Synthesized voice data

[1174] Step 4:

[1175] The server analyzes the image data and generates a character avatar.

[1176] Input: Saved image data

[1177] How it works:

[1178] The server analyzes the stored image data "characterimage.png" using a facial recognition tool (e.g., dlib) to extract the character's facial features (e.g., large eyes, distinctive smile). It then uses animation generation technology to generate an avatar with those facial features. The generated avatar is then used for the next process.

[1179] Output: Generated character avatar

[1180] Step 5:

[1181] The server analyzes the character script and incorporates the character's unique phrases and tone of voice into the AI ​​model.

[1182] Input: Saved character script

[1183] How it works:

[1184] The server analyzes the saved character script "character dialogue.txt." Through this analysis, the character's unique expressions and tone of voice are extracted and incorporated into the AI ​​model. This allows dialogue that reflects the character's individuality. The processed content is saved in the AI ​​model and used for subsequent processing.

[1185] Output: AI model incorporating the character's unique phrasing and tone of voice

[1186] Step 6:

[1187] The server trains the AI ​​model.

[1188] Input: Company product information, advertising messages, FAQ data, processed AI model

[1189] How it works:

[1190] The server receives product information, advertising messages, and FAQ data provided by companies and integrates this data into the AI ​​model. The server then uses NLP and machine learning algorithms to train the AI ​​model based on this data. This creates an AI chatbot whose characters are capable of appropriately conveying product information and advertising messages.

[1191] Output: A trained AI model

[1192] Step 7:

[1193] Corporate users can preview and adjust the generated AI chatbot.

[1194] Input: A trained AI model

[1195] How it works:

[1196] The server provides the generated AI chatbot for preview on the management console. Corporate users can actually try to interact with the AI ​​chatbot on the preview screen and check its behavior and what it says. If necessary, corporate users can adjust the character's specific speech content and interface and send the results to the server.

[1197] Output: The final tuned AI chatbot

[1198] Step 8:

[1199] The server deploys and integrates the final AI chatbot into a specific platform.

[1200] Input: The final tuned AI chatbot

[1201] How it works:

[1202] The server deploys the finalized AI chatbot to the designated platform, such as a website, app, or social networking site, and begins operation. The server integrates the chatbot with the platform through web hosting and APIs, making it immediately available for use.

[1203] Output: Deployed AI chatbot

[1204] Step 9:

[1205] An end user accesses a company's platform and begins interacting with an AI chatbot.

[1206] Input: End user inquiry

[1207] How it works:

[1208] An end user accesses a company's website or app and clicks the "Ask a character!" button. The AI ​​chatbot responds with "Hello, how is it?" and provides product information or advertising messages in response to the user's inquiry. The user's interaction history and feedback are sent to the server and used to improve the model for the next time.

[1209] Output: User interaction history and feedback

[1210] This system allows companies to carry out effective promotions and allows users to efficiently obtain product information through friendly characters.

[1211] (Application example 1)

[1212] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1213] The purpose of this invention is to provide a means for effectively communicating corporate product information and advertising messages. However, in conventional systems, when AI chatbots modeled after characters or celebrities are provided to companies, there is no mechanism for direct interaction with customers in physical stores. As a result, it is difficult to provide customers visiting physical stores with an experience equivalent to that of online information provision. Furthermore, there is a lack of technological means for realizing interactive dialogue that reflects the atmosphere of the physical store and customer behavior data.

[1214] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1215] In this invention, the server includes: means for receiving voice data and image data of a specific character or celebrity; means for analyzing the voice data and performing voice synthesis; means for analyzing the image data and generating a character avatar; means for analyzing the character script and incorporating the character's unique phrasing and tone of voice into an AI model; means for receiving a company's product information, advertising messages, and FAQ data and training the AI ​​model based thereon; means for previewing and adjusting the generated AI chatbot; means for deploying and integrating the final AI chatbot into a specific platform; and means for interacting with customers via terminals installed in physical stores and providing product information. This enables information provision in physical stores to be as effective as online, and provides an interactive customer experience that matches the atmosphere of the physical store.

[1216] "Audio data" refers to digital data containing the recorded voice of a specific character or celebrity.

[1217] "Image data" refers to data that stores images of specific characters or celebrities in digital format.

[1218] "Analysis" is the process of understanding the content of received audio or image data and extracting its features.

[1219] "Speech synthesis" is a technology that generates new voices based on analyzed voice data.

[1220] A "character avatar" is a digital graphic that recreates the appearance of a particular character or celebrity.

[1221] A "character script" is a collection of lines that contain the distinctive phrasing and tone of a character or celebrity.

[1222] An "AI model" is an algorithm that uses artificial intelligence, a digital model that learns and performs specific tasks.

[1223] "Training" is the process by which an AI model learns using specific data.

[1224] "Preview" is a function that allows you to check the behavior of the generated AI chatbot in advance.

[1225] "Deployment" refers to placing the generated AI chatbot so that it can operate on a specific platform.

[1226] "Platform" is a general term for the environment and services in which AI chatbots operate.

[1227] "In-store terminals" are digital devices that enable customer interaction in physical stores.

[1228] "Customers" refers to consumers and users who visit physical stores.

[1229] "Product information" is detailed information about the features and benefits of products offered by a company.

[1230] An "advertising message" is a promotional message that conveys the appeal of a product offered by a company.

[1231] "FAQ data" is a collection of frequently asked questions and their answers.

[1232] "Interaction data" is data generated when a user interacts with an AI chatbot.

[1233] "Ambient sounds" are background sounds used to recreate the atmosphere of a physical store.

[1234] "Video" refers to videos and images that visually convey the atmosphere of a physical store.

[1235] "Behavioral data" is data that records customer behavior and movements in physical stores.

[1236] MODE FOR CARRYING OUT THE INVENTION

[1237] This invention is a system that realizes an AI chatbot modeled after a specific character or celebrity, and aims to effectively convey product information and advertising messages through dialogue with customers in a physical store environment.

[1238] System Configuration and Operation

[1239] Hardware

[1240] This system uses the following hardware:

[1241] Brick-and-mortar devices: Includes devices such as tablets, kiosks, and smart glasses.

[1242] Server: A server with high-performance computing resources for training and deploying AI models.

[1243] software

[1244] The software used includes the following:

[1245] Natural Language Processing Libraries: Use high-performance AI models such as GPT-3.

[1246] Speech synthesis engine: Provides technology for synthesizing voice data.

[1247] Image analysis library: Generate character avatars using OpenCV etc.

[1248] Database system: A system for storing customer interaction and behavior data.

[1249] System Operation

[1250] The server performs the following process.

[1251] 1. Receive audio and image data of specific characters and celebrities provided by companies.

[1252] 2. Analyze the voice data and perform voice synthesis. Generate a new voice based on the analyzed voice data.

[1253] 3. Analyze the image data and generate a character avatar.

[1254] 4. Analyze character scripts and incorporate unique phrases and tones into the AI ​​model.

[1255] 5. Receives company product information, advertising messages, and FAQ data and uses them to train an AI model.

[1256] 6. Preview the generated AI chatbot and make any necessary adjustments.

[1257] 7. The final AI chatbot is deployed to a specific platform, i.e., a terminal installed in a physical store, and begins interacting with customers.

[1258] Specific examples

[1259] For example, consider a car dealership that installs an AI chatbot modeled after a popular anime character in its physical store to promote a new product.

[1260] Dealers log in to the management console and upload voice data, image data, and scripts (character lines and distinctive expressions) for specific animated characters.

[1261] The server analyzes this data and generates an AI chatbot.

[1262] The generated chatbot is deployed on tablet devices installed in the car dealership's physical store.

[1263] When customers visit the store, animated characters will provide information about cars and related products and introduce promotions.

[1264] Prompt Sentence Examples

[1265] "Please tell us the character's name and details of the new car model for visitors."

[1266] "Please tell me the character's name and the latest promotional information."

[1267] In this way, the system of the present invention enables interactive dialogue through friendly characters even in brick-and-mortar stores, and supports the provision of effective product information and promotional activities.

[1268] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1269] Step 1:

[1270] Corporate users access the management console and log in to the system. They then upload voice data, image data, and character scripts of specific characters or celebrities to the system. The input data is voice data, image data, and script data, which is then sent to the server.

[1271] Step 2:

[1272] The server analyzes the received voice data, extracts features from the voice data using natural language processing technology, and then generates the character's unique voice using a voice synthesis engine. The input data here is the voice data, and the output is a synthesized voice file.

[1273] Step 3:

[1274] The server analyzes the image data and generates a character avatar. It uses an image analysis library to extract features from the received image data and create an avatar. The input data is the image data, and the output is the generated character avatar.

[1275] Step 4:

[1276] The server analyzes the character script and incorporates the specific phrasing and tone of voice into the AI ​​model. A natural language processing model is used to train the phrasing and tone of voice of the script data. The input data here is the script data, and the output is the trained AI model.

[1277] Step 5:

[1278] The server receives product information, advertising messages, and FAQ data provided by the company and further trains the AI ​​model based on this information. The input data are product information, advertising messages, and FAQ data, and the output is an AI model that integrates product information.

[1279] Step 6:

[1280] Corporate users can preview the generated AI chatbot and make any necessary adjustments. They can check its operation on a device installed in a physical store and check the voice and avatar behavior. The input data is a trained AI model, a synthesized voice file, and a character avatar, and the output is a previewed AI chatbot.

[1281] Step 7:

[1282] The final AI chatbot is deployed to a terminal in a physical store and begins interacting with customers. A deployment tool is used to place the AI ​​chatbot on the terminal and prepare a system for interacting with customers. The input data is the previewed AI chatbot, and the output is an AI chatbot that interacts with customers in a physical store.

[1283] Step 8:

[1284] A user (customer) visits a physical store and initiates a conversation with an AI chatbot via a terminal device. The terminal device receives the customer's input and sends it to the server. The input data is the customer's question, and the output is the AI ​​chatbot's response.

[1285] Step 9:

[1286] The server collects customer interaction history and feedback and uses it for the next model update and performance improvement. This data is stored in a database and machine learning algorithms are used to improve the model. The input data is the customer interaction history and feedback, and the output is an updated AI model.

[1287] These steps enable the server and terminal to work together to provide effective information and customer experiences even in physical stores.

[1288] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1289] This invention provides a system that combines an emotion engine with an AI chatbot modeled after a specific character or celebrity, with the aim of effectively communicating product information and advertising messages through dialogue with users. It also recognizes the user's emotions and dynamically adjusts responses based on them, enabling a more natural and engaging conversational experience.

[1290] Program processing overview

[1291] Terminal

[1292] 1. A corporate user logs in to the management console. After entering their username and password and successful authentication, they are able to access the system's main screen.

[1293] 2. After logging in, the corporate user uploads the voice data, image data, and character script of the selected talent, celebrity, or anime character to the system.

[1294] 3. Perform the necessary initial setup for the emotion engine and adjust the emotion model parameters.

[1295] server

[1296] 1. The voice data is received and analyzed using natural language processing technology. The analysis engine converts the voice data into text and synthesizes speech based on that text data.

[1297] 2. Image data is received and a character avatar is generated using an image analysis engine.

[1298] 3. The character script is received and analyzed by a natural language processing (NLP) engine to extract specific phrases and tone of voice, which are then trained into an AI model.

[1299] 4. Receive product information, advertising messages, and FAQ data provided by companies, and use them to customize and train AI models.

[1300] 5. Use the emotion engine to analyze and recognize emotions from user input data.

[1301] 6. Select and set scripts for the AI ​​chatbot to dynamically adjust responses and tone based on emotion recognition results.

[1302] 7. Allow enterprise users to preview the generated AI chatbot and make any necessary adjustments.

[1303] 8. Deploy the AI ​​chatbot once it has been tuned to a specific platform.

[1304] User

[1305] 1. An end user accesses a company's platform and initiates a conversation with an AI chatbot.

[1306] 2. The emotion engine analyzes the emotions in the user's input data (text and voice) and adjusts the chatbot's responses accordingly.

[1307] 3. The AI ​​chatbot provides appropriate product information and advertising messages in response to user inquiries.

[1308] 4. The dialogue history and feedback are sent to the server and used to improve the model and performance next time.

[1309] Specific examples

[1310] Corporate User Operations

[1311] Let's take the example of a fashion brand creating an AI chatbot modeled after a popular character to launch a new advertising campaign.

[1312] 1. A brand marketer logs in to the Admin console.

[1313] 2. Upload the voice data, image data, and character scripts of popular characters and set the parameters of the emotion engine.

[1314] 3. Enter product information and advertising messages into the system.

[1315] Server Processing

[1316] 1. Analyze the uploaded voice data and perform voice synthesis.

[1317] 2. Analyze the image data and generate a character avatar.

[1318] 3. Analyze the character script and train the AI ​​model to recognize specific phrases and tones.

[1319] 4. Incorporate received product information and advertising messages into the model to train the chatbot.

[1320] 5. Use an emotion engine to analyze user emotions and dynamically adjust responses based on the results.

[1321] 6. Check the behavior of your AI chatbot with the preview function and make any necessary adjustments.

[1322] 7. After tuning, deploy the AI ​​chatbot on the brand's official website.

[1323] End User Experience

[1324] 1. An end user visits a brand's website and initiates a conversation with an AI chatbot.

[1325] 2. The emotion engine recognizes emotions from the user's input data, and the chatbot responds according to those emotions.

[1326] 3. Users can effectively obtain information about new products and promotions through friendly characters.

[1327] 4. User feedback and interaction history are sent to the server and used to improve the model next time.

[1328] In this way, AI chatbots that utilize an emotion engine allow users to enjoy a more natural and engaging conversational experience, and businesses can realize effective advertising tools. This system will provide new value in advertising and marketing.

[1329] The processing flow will be explained below.

[1330] Step 1:

[1331] Terminal: Corporate users log in to the management console by entering their username and password, and once authentication is successful, they are able to access the system's main screen.

[1332] Step 2:

[1333] On your device, select "Character Selection" from the Admin Console menu, select a talent, celebrity, or anime character to use, and upload the voice data, image data, and character script of the selected character to the system.

[1334] Step 3:

[1335] Device: Perform the necessary initial setup for the emotion engine and adjust the parameters of the emotion model, for example, setting which emotions (joy, sadness, anger, etc.) to recognize.

[1336] Step 4:

[1337] Server: Receives the uploaded voice data and sends it to the voice analysis engine. The analysis engine converts the voice data into text (Speech-to-Text). Text-to-Speech synthesis is performed based on the converted text data to generate the character's voice.

[1338] Step 5:

[1339] Server: Receives image data and sends it to the image analysis engine, which uses deep learning technology to generate a character avatar. The generated avatar is used as the visual representation of the chatbot.

[1340] Step 6:

[1341] Server: Receives character scripts and sends them to a natural language processing (NLP) engine, which analyzes the scripts, extracts the character's unique phrasing and tone of voice, and trains the AI ​​model.

[1342] Step 7:

[1343] Terminal: Corporate users enter product information, advertising messages, and FAQ data into the management console and send them to the server.

[1344] Step 8:

[1345] Server: Analyzes received product information and advertising messages and integrates them into the AI ​​model, allowing the chatbot to provide appropriate information to the user.

[1346] Step 9:

[1347] Server: Analyzes and recognizes emotions from user input data (text and voice) using an emotion engine. For example, it uses natural language processing technology to classify the user's emotional state into categories such as Joy, Anger, and Sadness.

[1348] Step 10:

[1349] Server: Dynamically adjusts the AI ​​chatbot's response and tone based on the results of emotion recognition. For example, if the user is angry, respond with a calm tone; if the user is happy, respond with a cheerful tone.

[1350] Step 11:

[1351] Server: Integrates character voices, avatars, phrases, emotion recognition, and product information to generate an AI chatbot. The generated chatbot is provided as a preview for corporate users to check.

[1352] Step 12:

[1353] Device: The company user can check the chatbot's behavior on the preview screen and adjust the settings and scripts as necessary. The adjustments are sent back to the server and reflected.

[1354] Step 13:

[1355] Server: Deploy the finalized AI chatbot to a specific platform (e.g., company website, app, social media), set up API endpoints using a deployment tool, and verify the integration.

[1356] Step 14:

[1357] User: An end user visits a company's website or app and initiates a conversation with an AI chatbot. When the user types a question or comment, the emotion engine analyzes their emotional state and the chatbot generates an appropriate response.

[1358] Step 15:

[1359] Server: Collects user interaction history and feedback and stores it in a database. The collected data is used for the next model update and performance improvement.

[1360] Through these processing steps, businesses can utilize AI chatbots combined with emotion engines to provide end users with a more natural and engaging conversational experience, while also serving as an effective advertising tool.

[1361] Example 2

[1362] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1363] Conventional chatbot systems lack the ability to recognize emotions in user interactions and dynamically adjust responses based on those emotions. As a result, users do not enjoy a natural conversational experience, making it difficult for companies to convey effective advertising messages. Furthermore, when modeling characters or celebrities, it is difficult to reproduce their unique expressions and tone of voice, which often results in low user engagement.

[1364] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1365] In this invention, the server includes means for receiving voice data and image data of a specific character or celebrity, means for analyzing the received voice data and performing voice synthesis, means for analyzing the image data and generating a character avatar, means for analyzing the character script and incorporating the character's unique expressions and tone of voice into an AI model, means for receiving a company's product information, advertising messages, and FAQ data and training the AI ​​model based thereon, means for recognizing emotions from user input data and dynamically adjusting responses using an emotion engine, means for previewing and adjusting the generated AI chatbot, and means for deploying and integrating the final AI chatbot into a specific platform. This allows users to enjoy a more natural and engaging conversational experience and enables companies to deliver effective advertising messages.

[1366] "Audio data" refers to data in which audio information is recorded in digital format, and includes the voice of a specific character or celebrity.

[1367] "Image data" refers to visual information recorded in digital format, and includes images of specific characters or celebrities.

[1368] A "character script" is text data that records the lines, expressions, and phrases used by a specific character.

[1369] "Speech synthesis" is a technology that generates speech based on text data, and is performed using natural language processing technology.

[1370] A "character avatar" is a digital representation of a particular character that is generated based on analyzed image data.

[1371] "Distinctive expressions and tone of voice" refers to the unique ways of expression and speaking used by particular characters or celebrities.

[1372] An "AI model" is a mathematical model for realizing artificial intelligence, and specifically includes algorithms that utilize machine learning and deep learning.

[1373] An "emotion engine" is the part of the system that recognizes emotions from user input data and dynamically adjusts responses based on the results.

[1374] "Platform" refers to the digital environment, such as a particular website or application, in which an AI chatbot is deployed.

[1375] "Deployment" means placing the developed system or software in a production environment and making it actually function.

[1376] The present invention provides a system that combines an emotion engine with an AI chatbot modeled after a specific character or celebrity. The purpose is to effectively communicate product information and advertising messages through dialogue with users. Specific embodiments of this system are described in detail below.

[1377] Hardware and Software Configuration

[1378] Terminal

[1379] The terminal is used by corporate users and is a device for accessing the management console. This management console includes functions for uploading data, setting parameters, and previewing. Specifically, since it is accessed via a web browser, the terminal can be used on a wide range of operating systems, including Windows, macOS, and Linux.

[1380] server

[1381] The server analyzes the data and generates the AI ​​chatbot. The server is equipped with the following main software and engines:

[1382] Natural Language Processing (NLP) engines (e.g., spaCy, NLTK)

[1383] Image analysis engine (e.g. OpenCV, TensorFlow)

[1384] Speech synthesis engine (e.g., Google Text-to-Speech API, Amazon Polly)

[1385] Emotion engine (e.g. IBM Watson Emotion Analysis)

[1386] Data processing and calculation

[1387] Voice data analysis and voice synthesis

[1388] The server receives voice data uploaded by corporate users from their devices and analyzes it using a natural language processing (NLP) engine. The analyzed text data is then converted into speech by a speech synthesis engine.

[1389] Image data analysis and avatar generation

[1390] The server receives the uploaded image data and uses an image analysis engine to generate a character avatar, which is used as the visual for the chatbot.

[1391] Character script analysis and model training

[1392] The server analyzes the character scripts received from corporate users, extracts specific phrases and tones using a natural language processing (NLP) engine, and trains the AI ​​model. This trained model is reflected in the chatbot's dialogue.

[1393] Integrating product information and advertising messages

[1394] Product information, advertising messages, and FAQ data provided by corporate users are received by the server, and the AI ​​model is customized based on this data, allowing marketing information to be naturally incorporated as part of the conversation.

[1395] Emotion recognition and response regulation

[1396] The emotion engine recognizes emotions from user input data and dynamically adjusts the tone of responses based on the results. For example, if a user expresses positive emotions, the chatbot will respond with a positive response.

[1397] Specific examples

[1398] Let's take the example of a fashion brand creating an AI chatbot modeled after a popular character to launch a new advertising campaign. The brand's marketer logs into the management console and uploads the popular character's voice data, image data, and character script to the system. This allows the server to perform voice, image, and NLP analysis to set the character avatar and its unique phrasing and tone of voice. Next, they adjust the emotion engine parameters and enter product information and advertising messages into the system. The fully customized AI chatbot can be checked through the preview function, and after any necessary adjustments are made, it is deployed to the brand's official website.

[1399] A user accesses a brand's website and begins a conversation with an AI chatbot. The emotion engine recognizes emotions from the user's input data and returns appropriate responses, providing a more natural and friendly conversational experience. For example, if a user inputs, "I want a new jacket," the chatbot will provide specific information such as, "The new jacket is from this year's fall / winter collection and is excellent for keeping you warm. It also comes in a wide variety of colors."

[1400] Prompt Sentence Examples

[1401] Here are some examples of prompts to input to a generative AI model:

[1402] 1. "I want to create an AI chatbot that uses an emotion engine. First, please tell me the steps to upload the character's voice and image data."

[1403] 2. "Give me an example of how a chatbot can tailor its response to a user's emotion when they ask a question."

[1404] 3. "How can a fashion brand effectively communicate new product information through friendly characters?"

[1405] By using these prompts, users can obtain more detailed instructions and explanations about specific tasks and settings, providing a more natural and engaging interaction experience for users and enabling companies to effectively communicate their advertising messages.

[1406] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1407] Step 1:

[1408] Terminal

[1409] A corporate user logs in to the management console. As input, they enter their username and password and click the "Login" button. The system receives this and performs authentication. If successful, they are redirected to the main screen. The output is access to the system's main screen.

[1410] Step 2:

[1411] Terminal

[1412] A company user uploads the voice data, image data, and character script of a character. As input, they specify these files in the respective upload forms and click the "Upload" button. The files are sent to the server. The output is the data correctly saved on the server.

[1413] Step 3:

[1414] Terminal

[1415] Corporate users configure the emotion engine initially and adjust the emotion model parameters. As input, they set emotion recognition accuracy and emotion category parameters on the settings screen. The system receives this and saves the emotion engine configuration data. The output is the adjusted emotion model.

[1416] Step 4:

[1417] server

[1418] The server analyzes the voice data and synthesizes the voice. The uploaded voice data is used as input. The server converts the voice to text using a natural language processing (NLP) engine, and then generates a new voice using a speech synthesis engine based on the text. The output is the generated voice data.

[1419] Step 5:

[1420] server

[1421] The server analyzes the image data and generates a character avatar. The uploaded image data is used as input. The server uses an image analysis engine to generate a 3D avatar from the image. The output is the generated character avatar.

[1422] Step 6:

[1423] server

[1424] The server analyzes the character script and trains the AI ​​model to recognize specific phrases and tones. The uploaded character script is used as input. A natural language processing (NLP) engine analyzes the script, extracts specific patterns, and trains the AI ​​model. The output is a trained AI model.

[1425] Step 7:

[1426] server

[1427] The server receives product information, advertising messages, and FAQ data provided by companies and customizes the AI ​​model. These data are used as input. The server analyzes them and uses them as necessary training data to strengthen the AI ​​model. The output is a customized AI model.

[1428] Step 8:

[1429] server

[1430] The server uses an emotion engine to recognize emotions from user input data and dynamically adjust responses. The input is text and voice data from the user's interaction with the chatbot. The server analyzes this using the emotion engine and adjusts the chatbot's response based on the results. The output is a response based on the emotion.

[1431] Step 9:

[1432] Terminal

[1433] Corporate users can preview the generated AI chatbot and make adjustments as needed. The input is a simulated interaction on the preview screen. Based on user feedback, the server adjusts the settings. The output is the adjusted chatbot.

[1434] Step 10:

[1435] server

[1436] The server deploys and integrates the final AI chatbot on a specific platform. The completed chatbot data is used as input. A deployment tool is used to integrate the chatbot into a specified website or application. The output is a chatbot running in a production environment.

[1437] Step 11:

[1438] User

[1439] An end user accesses a company's platform and starts a conversation with an AI chatbot. As input, they type a message into the chat window of a website or application. The system receives this and starts a conversation. The output is the content of the conversation with the chatbot.

[1440] Step 12:

[1441] server

[1442] The server dynamically adjusts responses based on the user's emotions. The inputs are the user's speech and the results of emotion analysis. Based on this, the server selects an appropriate response tone, and the chatbot responds to the user. The output is a response adjusted according to the user's emotions.

[1443] Step 13:

[1444] server

[1445] The server collects the interaction history and feedback and uses it to improve the model. The user's interaction history and feedback are used as input. These data are used for the next model training. The output is an improved AI model.

[1446] (Application example 2)

[1447] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1448] In recent years, AI chatbots have been used on various platforms, but to improve the user experience, more natural and emotionally adaptive responses are required. Conventional systems lack the ability to recognize and dynamically adjust to user emotions, and responses are often mechanical. Furthermore, content distribution services lack mechanisms for providing personalized content based on user emotions. To address these issues, the present invention aims to provide an AI chatbot system combined with an emotion recognition engine to improve the user experience.

[1449] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving voice data and image data of a specific character or celebrity; means for analyzing the received voice data and performing voice synthesis; means for analyzing image data and generating a character avatar; means for receiving company product information, advertising messages, and FAQ data and training an AI model based on them; means for analyzing a user's emotions using an emotion recognition engine and adjusting responses based on the analyzed emotions; means for processing user input data in a content distribution service platform and providing personalized content; means for voice synthesizing a response based on the emotion of a specific character; and means for visualizing a character avatar and coordinating the visual display with audio playback. This makes it possible to provide appropriate responses and personalized content according to the user's emotions.

[1450] "Voice data" refers to information that digitally records the voice of a user or a specific character.

[1451] "Image data" refers to visual information of characters or celebrities recorded in digital format.

[1452] "Speech synthesis" is a technology that uses a computer program to generate a human voice based on text data.

[1453] A "character avatar" is a digital icon or persona that visually represents a particular character.

[1454] A "character script" is a collection of lines and responses that a character utters, each with a specific phrase and tone of voice.

[1455] An "AI model" is a collection of algorithms trained to perform a specific task using artificial intelligence techniques.

[1456] An "emotion recognition engine" is an algorithm or tool that analyzes emotions from user input data and identifies those emotions.

[1457] "Personalized content" is digital content that is individually tailored based on a user's interests and preferences.

[1458] "Deployment" is the process of implementing software or an application on a particular platform.

[1459] A "server" is a computer system that provides services to other computers over a network.

[1460] A "platform" is the underlying hardware or software environment on which an application runs.

[1461] A "natural language processing model" is an algorithm used by computers to understand and generate human language.

[1462] The system for realizing the present invention includes the following components and means: Specific components and their respective processes will be described in detail below.

[1463] System Configuration

[1464] This system uses the following hardware and software:

[1465] Hardware:

[1466] Smartphone

[1467] Head-mounted display (HMD)

[1468] server

[1469] software:

[1470] Google Cloud Natural Language API

[1471] Google Cloud Text-to-Speech API

[1472] Unity

[1473] Program processing overview

[1474] The server has the following features:

[1475] 1. Receiving and analyzing audio and image data:

[1476] It receives voice and image data of characters and celebrities, converts the voice data into text using the Google Cloud Natural Language API, and then synthesizes speech based on that text using the Google Cloud Text-to-Speech API.

[1477] Image data is used to generate and visualize character avatars using Unity.

[1478] 2. Dynamic emotion recognition and response adjustment:

[1479] The user's input data (text) is subjected to sentiment analysis using the Google Cloud Natural Language API, and the sentiment score and strength of the sentiment are calculated.

[1480] Based on this, the character's response is selected and speech synthesis is performed using the Google Cloud Text-to-Speech API.

[1481] The terminal has the following features:

[1482] 1. Providing interaction:

[1483] It provides a user interface (UI) that allows users to initiate a dialogue with the AI ​​chatbot via a smartphone or HMD.

[1484] Visualize a character avatar and play synthesized voice in sync.

[1485] The user interacts with:

[1486] 1. Start the conversation:

[1487] Initiate a conversation with the AI ​​chatbot using an application on your smartphone or HMD.

[1488] Based on user input (text or voice), the system performs sentiment analysis and provides responses and content recommendations based on that.

[1489] Specific examples

[1490] For example, when a user expresses their opinion about a particular movie on a movie streaming service, the service will suggest movies based on the sentiment analysis results. The prompt sentence is as follows:

[1491] Example prompt sentence:

[1492] Input Text: "The movie I saw recently was very moving. It made me cry."

[1493] Emotion score: high positive emotion, strong emotional intensity

[1494] AI chatbot response: "So you like inspiring movies! I recommend 'XYZ' as a tear-jerking movie. I'll play a preview for you."

[1495] In this way, by providing content that corresponds to the user's emotions, it is possible to build a system that can provide a more natural and attractive interactive experience.

[1496] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1497] Step 1:

[1498] Receiving and analyzing audio and image data

[1499] The server receives audio data and image data of specific characters and celebrities provided by the company.

[1500] The received audio data is sent to the Google Cloud Natural Language API and converted into text, which is later used for speech synthesis.

[1501] The image data is analyzed by Unity and a character avatar is generated.

[1502] Input: Voice and image data provided by the company

[1503] Output: Speech data converted to text and generated character avatar

[1504] Step 2:

[1505] Emotion recognition engine for analyzing user emotions

[1506] The user inputs text or voice using a smartphone or HMD.

[1507] The user's input data (text or voice) is sent to the server and analyzed for sentiment scores and sentiment strength using the Google Cloud Natural Language API.

[1508] The results of this sentiment analysis are used to dynamically tailor responses.

[1509] Input: User-entered data (text or voice)

[1510] Output: Sentiment score and intensity of the emotion

[1511] Step 3:

[1512] Response generation and speech synthesis

[1513] The server selects an appropriate response based on the emotion score and the intensity of the emotion.

[1514] The selected response text is synthesized using the Google Cloud Text-to-Speech API and output in the appropriate character's voice.

[1515] The synthesized voice file is sent to the user's terminal.

[1516] Input: Sentiment analysis results, response text

[1517] Output: A synthesized response

[1518] Step 4:

[1519] Character Avatar Visualization

[1520] The server transmits the character avatar along with the synthesized voice to the user's terminal.

[1521] The user's device uses Unity to synchronize the display of the character avatar and the playback of the audio.

[1522] This allows users to experience a natural interaction both visually and aurally.

[1523] Input: Character avatar, audio file

[1524] Output: Visualized avatar and audio playback

[1525] Step 5:

[1526] Collection of conversation history and feedback

[1527] The server collects user interaction history and feedback to help improve the model in future.

[1528] The collected data is used to train AI models.

[1529] Input: Dialogue history, user feedback

[1530] Output: New training data

[1531] Specific working example:

[1532] For example, if a user enters the text, "The movie I saw recently was very moving. It brought me to tears," the system performs sentiment analysis on the text and detects high positive and strong emotional intensity. Based on the results, the system generates a response to the user via synthesized voice: "I see you like moving movies! I recommend 'XYZ' as a tear-jerking movie. I'll play a preview." A character avatar is also displayed at the same time, enhancing the naturalness of the interaction visually and aurally.

[1533] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1534] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1535] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1536] [Fourth embodiment]

[1537] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1538] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1539] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1540] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1541] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1542] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1543] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1544] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1545] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1546] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1547] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1548] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1549] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1550] The present invention is a system that provides businesses with AI chatbots modeled after specific characters or celebrities, and aims to effectively communicate product information and advertising messages through dialogue with users. Specific embodiments of the system are described below.

[1551] Program processing overview

[1552] Terminal

[1553] 1. A corporate user accesses the management console and logs into the system.

[1554] 2. After logging in, the corporate user uploads data of the selected talent, celebrity, anime character, etc. to the system, including voice data, image data, and character scripts.

[1555] 3. The uploaded data is sent to the system's server.

[1556] server

[1557] 1. Receive the voice data and analyze it using natural language processing technology.

[1558] 2. Based on the analyzed voice data, voice synthesis is performed to generate the character's voice.

[1559] 3. Image data is analyzed to generate a character avatar, using facial recognition and animation generation technology.

[1560] 4. Analyze character scripts and incorporate unique phrases and tones into the AI ​​model.

[1561] 5. Receive product information, advertising messages, and FAQ data provided by companies, and use this information to further customize and train the AI ​​model.

[1562] 6. Allow enterprise users to preview the generated AI chatbot and make any necessary adjustments.

[1563] 7. Deploy the AI ​​chatbot once it has been adjusted to the specified platform (website, app, social media, etc.).

[1564] User

[1565] 1. An end user accesses a company's platform (website or app) and initiates a conversation with an AI chatbot.

[1566] 2. AI chatbots respond to user inquiries and convey product information and advertising messages.

[1567] 3. User feedback and interaction history are sent to the server and used for the next model update and performance improvement.

[1568] Specific examples

[1569] For example, suppose a car manufacturer wants to create an AI chatbot modeled after a popular anime character to promote a new product. In this case, the process would involve the following steps:

[1570] Corporate User Operations

[1571] 1. The automaker's representative logs in to the management console.

[1572] 2. Upload voice data, image data, and scripts (character lines and distinctive expressions) of popular anime characters.

[1573] 3. Enter detailed information and advertising messages about the company's new products.

[1574] Server Processing

[1575] 1. Analyze the uploaded audio data and generate the character's voice.

[1576] 2. Analyze the image data and create a character avatar.

[1577] 3. Analyze the character script and incorporate the character's unique tone of voice into the AI ​​model.

[1578] 4. Integrate new product information and advertising messages into the AI ​​model to train the chatbot.

[1579] 5. After training is complete, a preview of the chatbot is provided, allowing corporate users to check and adjust its operation.

[1580] 6. Once the AI ​​chatbot is fully tuned, it will be deployed on the car manufacturer's official website.

[1581] End User Experience

[1582] 1. An end user visits the car manufacturer's website.

[1583] 2. An AI chatbot based on a popular anime character responds to user inquiries and provides information about new products.

[1584] 3. The user's interaction history and feedback are sent to the server and used to improve the model next time.

[1585] This allows users to efficiently obtain product information through friendly characters, and companies can carry out effective promotions. As a new form of advertising, this system will likely play an important role in companies' marketing strategies.

[1586] The processing flow will be explained below.

[1587] Step 1:

[1588] Terminal: Corporate users log in to the management console by entering their username and password, and once authentication is successful, they are able to access the system's main screen.

[1589] Step 2:

[1590] On your device, select "Character Selection" from the Admin Console menu, select a talent, celebrity, or anime character to use, and upload the voice data, image data, and character script of the selected character to the system.

[1591] Step 3:

[1592] Server: Receives the uploaded voice data and sends it to the voice analysis engine, which converts the voice data into text (Speech-to-Text) and synthesizes speech based on that text (Text-to-Speech).

[1593] Step 4:

[1594] Server: Receives image data and sends it to the image analysis engine, which uses deep learning technology to generate an avatar of the character, which is used as a visual representation.

[1595] Step 5:

[1596] Server: Receives character scripts and sends them to a natural language processing (NLP) engine, which analyzes the scripts, extracts the character's unique phrasing and tone of voice, and trains the AI ​​model.

[1597] Step 6:

[1598] Terminal: Corporate users enter product information, advertising messages, and FAQ data into the management console and send them to the server.

[1599] Step 7:

[1600] Server: Analyzes received product information and advertising messages and integrates them into the AI ​​model, allowing the chatbot to provide appropriate information to the user.

[1601] Step 8:

[1602] Server: Integrates character voices, avatars, phrases, and product information to generate AI chatbots, which are then provided as previews for corporate users to check.

[1603] Step 9:

[1604] Device: The company user can check the chatbot's behavior on the preview screen and adjust the settings and scripts as necessary. The adjustments are sent back to the server and reflected.

[1605] Step 10:

[1606] Server: Deploy the AI ​​chatbot once it has been tuned to a specific platform (e.g., company website, app, social media), set up API endpoints using a deployment tool, and verify the integration.

[1607] Step 11:

[1608] User: An end user accesses a company's website or app and starts a conversation with an AI chatbot. The chatbot analyzes the user's inquiry and generates an appropriate response.

[1609] Step 12:

[1610] Server: Collects user interaction history and feedback and stores it in a database. The collected data is used for the next model update and performance improvement.

[1611] Through the above processing steps, businesses can utilize AI chatbots as an effective advertising tool to efficiently convey product information and advertising messages to end users.

[1612] Example 1

[1613] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1614] Companies need a user-friendly interface to effectively communicate product information and advertising messages to customers. However, with existing systems, it is difficult to create an AI chatbot modeled after a specific character or celebrity, and reproducing their unique phrasing and tone of voice requires advanced technology and a great deal of time. It is also not easy to collect user interaction data and use it to improve the model for the next time. As a result, companies' marketing strategies may not be implemented effectively.

[1615] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1616] In this invention, the server includes a means for corporate users to log in, a means for receiving voice data and image data of specific characters or celebrities, a means for analyzing the received voice data and performing speech synthesis using natural language processing technology, a means for analyzing image data and generating a character avatar using facial recognition and animation generation technology, a means for analyzing the character script and incorporating the character's unique phrasing and tone of voice into the AI ​​model, a means for receiving the company's product information, advertising messages, and FAQ data and training the AI ​​model based on them, a means for corporate users to preview and adjust the generated AI chatbot, and a means for deploying and integrating the final AI chatbot into a specific platform. This allows companies to easily create friendly AI chatbots using specific characters or celebrities, enabling them to implement effective marketing strategies. Furthermore, user interaction data can be collected and used to improve the model next time, thereby continuously improving the performance of the AI ​​chatbot.

[1617] The "means for corporate users to log in" refers to a function that allows a user to access the system's management console, enter authentication information, log in to the system, and start operations.

[1618] "Means for receiving audio data and image data of specific characters or celebrities" refers to a function for uploading audio and image files of specific characters or celebrities to the system and transmitting them to the server.

[1619] "Means for analyzing received voice data and synthesizing voice using natural language processing technology" refers to the function in which the server analyzes the voice data, extracts text, and generates the character's voice using a voice synthesis tool.

[1620] "Means for analyzing image data and generating a character avatar using facial recognition and animation generation technology" refers to a function in which the server analyzes image data, extracts the facial features of a character, and creates a moving avatar using facial recognition and animation generation technology.

[1621] "Means for analyzing character scripts and incorporating unique phrases and tones into the AI ​​model" refers to a function in which the server analyzes the text data of a character script and incorporates the character's unique phrases and tones into the AI ​​model.

[1622] "Means for receiving product information, advertising messages, and FAQ data from a company and training an AI model based on the same" refers to a function in which the server receives product information, advertising messages, and FAQ data provided by a company and uses that data to train an AI model.

[1623] "Means for corporate users to preview and adjust the generated AI chatbot" refers to a function that allows corporate users to preview the generated AI chatbot on the management console and adjust it as necessary.

[1624] "Means for deploying and integrating the final AI chatbot into a specific platform" refers to the function by which the server deploys the finalized AI chatbot into a specified platform such as a website, app, or social networking site, and begins operation.

[1625] The present invention is a system that provides businesses with AI chatbots modeled after specific characters or celebrities, and aims to effectively communicate product information and advertising messages through dialogue with users. Specific embodiments are described below.

[1626] Hardware and software used

[1627] Device: PC or tablet used by a corporate user

[1628] Software: Management console, web browser

[1629] Server: High-performance cloud server (e.g. AWS EC2, Google Cloud)

[1630] Software: Natural Language Processing (NLP) tools (e.g., Intel OpenVINO), facial recognition tools (e.g., dlib), speech synthesis tools (e.g., Google Text-to-Speech API)

[1631] Program processing overview

[1632] The system begins when a corporate user accesses the management console and logs in. The corporate user then uploads data about the talent, celebrity, or anime character they have selected to the system, including voice data, image data, and character scripts.

[1633] The device sends this data to a server, which uses NLP tools to analyze the voice data and synthesizes it. The server also analyzes the image data with facial recognition tools to generate a character avatar. The character script is analyzed to incorporate specific phrasing and tone of voice into the AI ​​model.

[1634] Product information, advertising messages, and FAQ data provided by companies are received by the server and used to train the AI ​​model. After the AI ​​model is trained, the generated AI chatbot is provided for company users to preview and make any necessary adjustments. The final AI chatbot is then deployed to the specified platform (website, app, social media, etc.).

[1635] End users access the company's platform and begin a conversation with the AI ​​chatbot. The AI ​​chatbot responds to the user's inquiries and provides product information and advertising messages. User feedback and conversation history are sent to the server and used to improve the model for the next time.

[1636] Specific examples

[1637] For example, suppose an automobile manufacturer creates an AI chatbot modeled after a popular anime character to promote a new product. In this case, the automobile manufacturer's representative operates it as follows:

[1638] 1. The automaker's representative logs in to the management console.

[1639] 2. Upload voice data, image data, and scripts (character lines and distinctive expressions) of popular anime characters.

[1640] 3. Enter detailed information and advertising messages about the company's new products.

[1641] The server performs the following process:

[1642] 1. Analyze the uploaded audio data and generate the character's voice.

[1643] 2. Analyze the image data and create a character avatar.

[1644] 3. Analyze the character script and incorporate the character's unique tone of voice into the AI ​​model.

[1645] 4. Integrate new product information and advertising messages into the AI ​​model to train the chatbot.

[1646] 5. After training is complete, a preview of the chatbot is provided, allowing corporate users to check and adjust its operation.

[1647] 6. Once the AI ​​chatbot is fully tuned, it will be deployed on the car manufacturer's official website.

[1648] The end user experience is as follows:

[1649] 1. An end user visits the car manufacturer's website.

[1650] 2. An AI chatbot based on a popular anime character responds to user inquiries and provides information about new products.

[1651] 3. The user's interaction history and feedback are sent to the server and used to improve the model next time.

[1652] Prompt Sentence Examples

[1653] "Generate an AI chatbot using popular anime characters to provide information about a company's new products."

[1654] "Enter your character script and product information to create a customized AI chatbot."

[1655] Deploy the generated AI chatbot on your website and collect user feedback.

[1656] This system allows companies to carry out effective promotions and allows users to efficiently obtain product information through friendly characters.

[1657] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1658] Step 1:

[1659] A corporate user accesses the Admin Console and logs in.

[1660] Input: Username, Password

[1661] How it works:

[1662] The terminal displays the management console screen, and the corporate user enters their username and password. The server receives the entered authentication information and authenticates it by checking it against the database. Once the user is authenticated, the server generates a dashboard page, sends it to the terminal, and displays it.

[1663] Output: Dashboard page

[1664] Step 2:

[1665] Corporate users upload voice data, image data, and character scripts of characters and celebrities.

[1666] Input: Audio data, image data, character script

[1667] How it works:

[1668] Corporate users click the "Create a new chatbot" button on the management console, select and upload files for voice data (e.g., "character voice.wav"), image data (e.g., "character image.png"), and character script (e.g., "character lines.txt"). The device sends these files to the server, which saves the received data in a specified folder and prepares it for the next process.

[1669] Output: Saved audio data, image data, character scripts

[1670] Step 3:

[1671] The server analyzes the voice data and performs voice synthesis.

[1672] Input: Saved audio data

[1673] How it works:

[1674] The server analyzes the stored voice data "character voice.wav" using an NLP tool (e.g., Intel's OpenVINO). This analysis extracts text from the voice and obtains voice characteristics. The server then uses the Google Text-to-Speech API to synthesize a character's voice based on the extracted text. The synthesized voice data is saved and used for further processing.

[1675] Output: Synthesized voice data

[1676] Step 4:

[1677] The server analyzes the image data and generates a character avatar.

[1678] Input: Saved image data

[1679] How it works:

[1680] The server analyzes the stored image data "characterimage.png" using a facial recognition tool (e.g., dlib) to extract the character's facial features (e.g., large eyes, distinctive smile). It then uses animation generation technology to generate an avatar with those facial features. The generated avatar is then used for the next process.

[1681] Output: Generated character avatar

[1682] Step 5:

[1683] The server analyzes the character script and incorporates the character's unique phrases and tone of voice into the AI ​​model.

[1684] Input: Saved character script

[1685] How it works:

[1686] The server analyzes the saved character script "character dialogue.txt." Through this analysis, the character's unique expressions and tone of voice are extracted and incorporated into the AI ​​model. This allows dialogue that reflects the character's individuality. The processed content is saved in the AI ​​model and used for subsequent processing.

[1687] Output: AI model incorporating the character's unique phrasing and tone of voice

[1688] Step 6:

[1689] The server trains the AI ​​model.

[1690] Input: Company product information, advertising messages, FAQ data, processed AI model

[1691] How it works:

[1692] The server receives product information, advertising messages, and FAQ data provided by companies and integrates this data into the AI ​​model. The server then uses NLP and machine learning algorithms to train the AI ​​model based on this data. This creates an AI chatbot whose characters are capable of appropriately conveying product information and advertising messages.

[1693] Output: A trained AI model

[1694] Step 7:

[1695] Corporate users can preview and adjust the generated AI chatbot.

[1696] Input: A trained AI model

[1697] How it works:

[1698] The server provides the generated AI chatbot for preview on the management console. Corporate users can actually try to interact with the AI ​​chatbot on the preview screen and check its behavior and what it says. If necessary, corporate users can adjust the character's specific speech content and interface and send the results to the server.

[1699] Output: The final tuned AI chatbot

[1700] Step 8:

[1701] The server deploys and integrates the final AI chatbot into a specific platform.

[1702] Input: The final tuned AI chatbot

[1703] How it works:

[1704] The server deploys the finalized AI chatbot to the designated platform, such as a website, app, or social networking site, and begins operation. The server integrates the chatbot with the platform through web hosting and APIs, making it immediately available for use.

[1705] Output: Deployed AI chatbot

[1706] Step 9:

[1707] An end user accesses a company's platform and begins interacting with an AI chatbot.

[1708] Input: End user inquiry

[1709] How it works:

[1710] An end user accesses a company's website or app and clicks the "Ask a character!" button. The AI ​​chatbot responds with "Hello, how is it?" and provides product information or advertising messages in response to the user's inquiry. The user's interaction history and feedback are sent to the server and used to improve the model for the next time.

[1711] Output: User interaction history and feedback

[1712] This system allows companies to carry out effective promotions and allows users to efficiently obtain product information through friendly characters.

[1713] (Application example 1)

[1714] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1715] The purpose of this invention is to provide a means for effectively communicating corporate product information and advertising messages. However, in conventional systems, when AI chatbots modeled after characters or celebrities are provided to companies, there is no mechanism for direct interaction with customers in physical stores. As a result, it is difficult to provide customers visiting physical stores with an experience equivalent to that of online information provision. Furthermore, there is a lack of technological means for realizing interactive dialogue that reflects the atmosphere of the physical store and customer behavior data.

[1716] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1717] In this invention, the server includes: means for receiving voice data and image data of a specific character or celebrity; means for analyzing the voice data and performing voice synthesis; means for analyzing the image data and generating a character avatar; means for analyzing the character script and incorporating the character's unique phrasing and tone of voice into an AI model; means for receiving a company's product information, advertising messages, and FAQ data and training the AI ​​model based thereon; means for previewing and adjusting the generated AI chatbot; means for deploying and integrating the final AI chatbot into a specific platform; and means for interacting with customers via terminals installed in physical stores and providing product information. This enables information provision in physical stores to be as effective as online, and provides an interactive customer experience that matches the atmosphere of the physical store.

[1718] "Audio data" refers to digital data containing the recorded voice of a specific character or celebrity.

[1719] "Image data" refers to data that stores images of specific characters or celebrities in digital format.

[1720] "Analysis" is the process of understanding the content of received audio or image data and extracting its features.

[1721] "Speech synthesis" is a technology that generates new voices based on analyzed voice data.

[1722] A "character avatar" is a digital graphic that recreates the appearance of a particular character or celebrity.

[1723] A "character script" is a collection of lines that contain the distinctive phrasing and tone of a character or celebrity.

[1724] An "AI model" is an algorithm that uses artificial intelligence, a digital model that learns and performs specific tasks.

[1725] "Training" is the process by which an AI model learns using specific data.

[1726] "Preview" is a function that allows you to check the behavior of the generated AI chatbot in advance.

[1727] "Deployment" refers to placing the generated AI chatbot so that it can operate on a specific platform.

[1728] "Platform" is a general term for the environment and services in which AI chatbots operate.

[1729] "In-store terminals" are digital devices that enable customer interaction in physical stores.

[1730] "Customers" refers to consumers and users who visit physical stores.

[1731] "Product information" is detailed information about the features and benefits of products offered by a company.

[1732] An "advertising message" is a promotional message that conveys the appeal of a product offered by a company.

[1733] "FAQ data" is a collection of frequently asked questions and their answers.

[1734] "Interaction data" is data generated when a user interacts with an AI chatbot.

[1735] "Ambient sounds" are background sounds used to recreate the atmosphere of a physical store.

[1736] "Video" refers to videos and images that visually convey the atmosphere of a physical store.

[1737] "Behavioral data" is data that records customer behavior and movements in physical stores.

[1738] MODE FOR CARRYING OUT THE INVENTION

[1739] This invention is a system that realizes an AI chatbot modeled after a specific character or celebrity, and aims to effectively convey product information and advertising messages through dialogue with customers in a physical store environment.

[1740] System Configuration and Operation

[1741] Hardware

[1742] This system uses the following hardware:

[1743] Brick-and-mortar devices: Includes devices such as tablets, kiosks, and smart glasses.

[1744] Server: A server with high-performance computing resources for training and deploying AI models.

[1745] software

[1746] The software used includes the following:

[1747] Natural Language Processing Libraries: Use high-performance AI models such as GPT-3.

[1748] Speech synthesis engine: Provides technology for synthesizing voice data.

[1749] Image analysis library: Generate character avatars using OpenCV etc.

[1750] Database system: A system for storing customer interaction and behavior data.

[1751] System Operation

[1752] The server performs the following process.

[1753] 1. Receive audio and image data of specific characters and celebrities provided by companies.

[1754] 2. Analyze the voice data and perform voice synthesis. Generate a new voice based on the analyzed voice data.

[1755] 3. Analyze the image data and generate a character avatar.

[1756] 4. Analyze character scripts and incorporate unique phrases and tones into the AI ​​model.

[1757] 5. Receives company product information, advertising messages, and FAQ data and uses them to train an AI model.

[1758] 6. Preview the generated AI chatbot and make any necessary adjustments.

[1759] 7. The final AI chatbot is deployed to a specific platform, i.e., a terminal installed in a physical store, and begins interacting with customers.

[1760] Specific examples

[1761] For example, consider a car dealership that installs an AI chatbot modeled after a popular anime character in its physical store to promote a new product.

[1762] Dealers log in to the management console and upload voice data, image data, and scripts (character lines and distinctive expressions) for specific animated characters.

[1763] The server analyzes this data and generates an AI chatbot.

[1764] The generated chatbot is deployed on tablet devices installed in the car dealership's physical store.

[1765] When customers visit the store, animated characters will provide information about cars and related products and introduce promotions.

[1766] Prompt Sentence Examples

[1767] "Please tell us the character's name and details of the new car model for visitors."

[1768] "Please tell me the character's name and the latest promotional information."

[1769] In this way, the system of the present invention enables interactive dialogue through friendly characters even in brick-and-mortar stores, and supports the provision of effective product information and promotional activities.

[1770] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1771] Step 1:

[1772] Corporate users access the management console and log in to the system. They then upload voice data, image data, and character scripts of specific characters or celebrities to the system. The input data is voice data, image data, and script data, which is then sent to the server.

[1773] Step 2:

[1774] The server analyzes the received voice data, extracts features from the voice data using natural language processing technology, and then generates the character's unique voice using a voice synthesis engine. The input data here is the voice data, and the output is a synthesized voice file.

[1775] Step 3:

[1776] The server analyzes the image data and generates a character avatar. It uses an image analysis library to extract features from the received image data and create an avatar. The input data is the image data, and the output is the generated character avatar.

[1777] Step 4:

[1778] The server analyzes the character script and incorporates the specific phrasing and tone of voice into the AI ​​model. A natural language processing model is used to train the phrasing and tone of voice of the script data. The input data here is the script data, and the output is the trained AI model.

[1779] Step 5:

[1780] The server receives product information, advertising messages, and FAQ data provided by the company and further trains the AI ​​model based on this information. The input data are product information, advertising messages, and FAQ data, and the output is an AI model that integrates product information.

[1781] Step 6:

[1782] Corporate users can preview the generated AI chatbot and make any necessary adjustments. They can check its operation on a device installed in a physical store and check the voice and avatar behavior. The input data is a trained AI model, a synthesized voice file, and a character avatar, and the output is a previewed AI chatbot.

[1783] Step 7:

[1784] The final AI chatbot is deployed to a terminal in a physical store and begins interacting with customers. A deployment tool is used to place the AI ​​chatbot on the terminal and prepare a system for interacting with customers. The input data is the previewed AI chatbot, and the output is an AI chatbot that interacts with customers in a physical store.

[1785] Step 8:

[1786] A user (customer) visits a physical store and initiates a conversation with an AI chatbot via a terminal device. The terminal device receives the customer's input and sends it to the server. The input data is the customer's question, and the output is the AI ​​chatbot's response.

[1787] Step 9:

[1788] The server collects customer interaction history and feedback and uses it for the next model update and performance improvement. This data is stored in a database and machine learning algorithms are used to improve the model. The input data is the customer interaction history and feedback, and the output is an updated AI model.

[1789] These steps enable the server and terminal to work together to provide effective information and customer experiences even in physical stores.

[1790] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1791] This invention provides a system that combines an emotion engine with an AI chatbot modeled after a specific character or celebrity, with the aim of effectively communicating product information and advertising messages through dialogue with users. It also recognizes the user's emotions and dynamically adjusts responses based on them, enabling a more natural and engaging conversational experience.

[1792] Program processing overview

[1793] Terminal

[1794] 1. A corporate user logs in to the management console. After entering their username and password and successful authentication, they are able to access the system's main screen.

[1795] 2. After logging in, the corporate user uploads the voice data, image data, and character script of the selected talent, celebrity, or anime character to the system.

[1796] 3. Perform the necessary initial setup for the emotion engine and adjust the emotion model parameters.

[1797] server

[1798] 1. The voice data is received and analyzed using natural language processing technology. The analysis engine converts the voice data into text and synthesizes speech based on that text data.

[1799] 2. Image data is received and a character avatar is generated using an image analysis engine.

[1800] 3. The character script is received and analyzed by a natural language processing (NLP) engine to extract specific phrases and tone of voice, which are then trained into an AI model.

[1801] 4. Receive product information, advertising messages, and FAQ data provided by companies, and use them to customize and train AI models.

[1802] 5. Use the emotion engine to analyze and recognize emotions from user input data.

[1803] 6. Select and set scripts for the AI ​​chatbot to dynamically adjust responses and tone based on emotion recognition results.

[1804] 7. Allow enterprise users to preview the generated AI chatbot and make any necessary adjustments.

[1805] 8. Deploy the AI ​​chatbot once it has been tuned to a specific platform.

[1806] User

[1807] 1. An end user accesses a company's platform and initiates a conversation with an AI chatbot.

[1808] 2. The emotion engine analyzes the emotions in the user's input data (text and voice) and adjusts the chatbot's responses accordingly.

[1809] 3. The AI ​​chatbot provides appropriate product information and advertising messages in response to user inquiries.

[1810] 4. The dialogue history and feedback are sent to the server and used to improve the model and performance next time.

[1811] Specific examples

[1812] Corporate User Operations

[1813] Let's take the example of a fashion brand creating an AI chatbot modeled after a popular character to launch a new advertising campaign.

[1814] 1. A brand marketer logs in to the Admin console.

[1815] 2. Upload the voice data, image data, and character scripts of popular characters and set the parameters of the emotion engine.

[1816] 3. Enter product information and advertising messages into the system.

[1817] Server Processing

[1818] 1. Analyze the uploaded voice data and perform voice synthesis.

[1819] 2. Analyze the image data and generate a character avatar.

[1820] 3. Analyze the character script and train the AI ​​model to recognize specific phrases and tones.

[1821] 4. Incorporate received product information and advertising messages into the model to train the chatbot.

[1822] 5. Use an emotion engine to analyze user emotions and dynamically adjust responses based on the results.

[1823] 6. Check the behavior of your AI chatbot with the preview function and make any necessary adjustments.

[1824] 7. After tuning, deploy the AI ​​chatbot on the brand's official website.

[1825] End User Experience

[1826] 1. An end user visits a brand's website and initiates a conversation with an AI chatbot.

[1827] 2. The emotion engine recognizes emotions from the user's input data, and the chatbot responds according to those emotions.

[1828] 3. Users can effectively obtain information about new products and promotions through friendly characters.

[1829] 4. User feedback and interaction history are sent to the server and used to improve the model next time.

[1830] In this way, AI chatbots that utilize an emotion engine allow users to enjoy a more natural and engaging conversational experience, and businesses can realize effective advertising tools. This system will provide new value in advertising and marketing.

[1831] The processing flow will be explained below.

[1832] Step 1:

[1833] Terminal: Corporate users log in to the management console by entering their username and password, and once authentication is successful, they are able to access the system's main screen.

[1834] Step 2:

[1835] On your device, select "Character Selection" from the Admin Console menu, select a talent, celebrity, or anime character to use, and upload the voice data, image data, and character script of the selected character to the system.

[1836] Step 3:

[1837] Device: Perform the necessary initial setup for the emotion engine and adjust the parameters of the emotion model, for example, setting which emotions (joy, sadness, anger, etc.) to recognize.

[1838] Step 4:

[1839] Server: Receives the uploaded voice data and sends it to the voice analysis engine. The analysis engine converts the voice data into text (Speech-to-Text). Text-to-Speech synthesis is performed based on the converted text data to generate the character's voice.

[1840] Step 5:

[1841] Server: Receives image data and sends it to the image analysis engine, which uses deep learning technology to generate a character avatar. The generated avatar is used as the visual representation of the chatbot.

[1842] Step 6:

[1843] Server: Receives character scripts and sends them to a natural language processing (NLP) engine, which analyzes the scripts, extracts the character's unique phrasing and tone of voice, and trains the AI ​​model.

[1844] Step 7:

[1845] Terminal: Corporate users enter product information, advertising messages, and FAQ data into the management console and send them to the server.

[1846] Step 8:

[1847] Server: Analyzes received product information and advertising messages and integrates them into the AI ​​model, allowing the chatbot to provide appropriate information to the user.

[1848] Step 9:

[1849] Server: Analyzes and recognizes emotions from user input data (text and voice) using an emotion engine. For example, it uses natural language processing technology to classify the user's emotional state into categories such as Joy, Anger, and Sadness.

[1850] Step 10:

[1851] Server: Dynamically adjusts the AI ​​chatbot's response and tone based on the results of emotion recognition. For example, if the user is angry, respond with a calm tone; if the user is happy, respond with a cheerful tone.

[1852] Step 11:

[1853] Server: Integrates character voices, avatars, phrases, emotion recognition, and product information to generate an AI chatbot. The generated chatbot is provided as a preview for corporate users to check.

[1854] Step 12:

[1855] Device: The company user can check the chatbot's behavior on the preview screen and adjust the settings and scripts as necessary. The adjustments are sent back to the server and reflected.

[1856] Step 13:

[1857] Server: Deploy the finalized AI chatbot to a specific platform (e.g., company website, app, social media), set up API endpoints using a deployment tool, and verify the integration.

[1858] Step 14:

[1859] User: An end user visits a company's website or app and initiates a conversation with an AI chatbot. When the user types a question or comment, the emotion engine analyzes their emotional state and the chatbot generates an appropriate response.

[1860] Step 15:

[1861] Server: Collects user interaction history and feedback and stores it in a database. The collected data is used for the next model update and performance improvement.

[1862] Through these processing steps, businesses can utilize AI chatbots combined with emotion engines to provide end users with a more natural and engaging conversational experience, while also serving as an effective advertising tool.

[1863] Example 2

[1864] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1865] Conventional chatbot systems lack the ability to recognize emotions in user interactions and dynamically adjust responses based on those emotions. As a result, users do not enjoy a natural conversational experience, making it difficult for companies to convey effective advertising messages. Furthermore, when modeling characters or celebrities, it is difficult to reproduce their unique expressions and tone of voice, which often results in low user engagement.

[1866] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1867] In this invention, the server includes means for receiving voice data and image data of a specific character or celebrity, means for analyzing the received voice data and performing voice synthesis, means for analyzing the image data and generating a character avatar, means for analyzing the character script and incorporating the character's unique expressions and tone of voice into an AI model, means for receiving a company's product information, advertising messages, and FAQ data and training the AI ​​model based thereon, means for recognizing emotions from user input data and dynamically adjusting responses using an emotion engine, means for previewing and adjusting the generated AI chatbot, and means for deploying and integrating the final AI chatbot into a specific platform. This allows users to enjoy a more natural and engaging conversational experience and enables companies to deliver effective advertising messages.

[1868] "Audio data" refers to data in which audio information is recorded in digital format, and includes the voice of a specific character or celebrity.

[1869] "Image data" refers to visual information recorded in digital format, and includes images of specific characters or celebrities.

[1870] A "character script" is text data that records the lines, expressions, and phrases used by a specific character.

[1871] "Speech synthesis" is a technology that generates speech based on text data, and is performed using natural language processing technology.

[1872] A "character avatar" is a digital representation of a particular character that is generated based on analyzed image data.

[1873] "Distinctive expressions and tone of voice" refers to the unique ways of expression and speaking used by particular characters or celebrities.

[1874] An "AI model" is a mathematical model for realizing artificial intelligence, and specifically includes algorithms that utilize machine learning and deep learning.

[1875] An "emotion engine" is the part of the system that recognizes emotions from user input data and dynamically adjusts responses based on the results.

[1876] "Platform" refers to the digital environment, such as a particular website or application, in which an AI chatbot is deployed.

[1877] "Deployment" means placing the developed system or software in a production environment and making it actually function.

[1878] The present invention provides a system that combines an emotion engine with an AI chatbot modeled after a specific character or celebrity. The purpose is to effectively communicate product information and advertising messages through dialogue with users. Specific embodiments of this system are described in detail below.

[1879] Hardware and Software Configuration

[1880] Terminal

[1881] The terminal is used by corporate users and is a device for accessing the management console. This management console includes functions for uploading data, setting parameters, and previewing. Specifically, since it is accessed via a web browser, the terminal can be used on a wide range of operating systems, including Windows, macOS, and Linux.

[1882] server

[1883] The server analyzes the data and generates the AI ​​chatbot. The server is equipped with the following main software and engines:

[1884] Natural Language Processing (NLP) engines (e.g., spaCy, NLTK)

[1885] Image analysis engine (e.g. OpenCV, TensorFlow)

[1886] Speech synthesis engine (e.g., Google Text-to-Speech API, Amazon Polly)

[1887] Emotion engine (e.g. IBM Watson Emotion Analysis)

[1888] Data processing and calculation

[1889] Voice data analysis and voice synthesis

[1890] The server receives voice data uploaded by corporate users from their devices and analyzes it using a natural language processing (NLP) engine. The analyzed text data is then converted into speech by a speech synthesis engine.

[1891] Image data analysis and avatar generation

[1892] The server receives the uploaded image data and uses an image analysis engine to generate a character avatar, which is used as the visual for the chatbot.

[1893] Character script analysis and model training

[1894] The server analyzes the character scripts received from corporate users, extracts specific phrases and tones using a natural language processing (NLP) engine, and trains the AI ​​model. This trained model is reflected in the chatbot's dialogue.

[1895] Integrating product information and advertising messages

[1896] Product information, advertising messages, and FAQ data provided by corporate users are received by the server, and the AI ​​model is customized based on this data, allowing marketing information to be naturally incorporated as part of the conversation.

[1897] Emotion recognition and response regulation

[1898] The emotion engine recognizes emotions from user input data and dynamically adjusts the tone of responses based on the results. For example, if a user expresses positive emotions, the chatbot will respond with a positive response.

[1899] Specific examples

[1900] Let's take the example of a fashion brand creating an AI chatbot modeled after a popular character to launch a new advertising campaign. The brand's marketer logs into the management console and uploads the popular character's voice data, image data, and character script to the system. This allows the server to perform voice, image, and NLP analysis to set the character avatar and its unique phrasing and tone of voice. Next, they adjust the emotion engine parameters and enter product information and advertising messages into the system. The fully customized AI chatbot can be checked through the preview function, and after any necessary adjustments are made, it is deployed to the brand's official website.

[1901] A user accesses a brand's website and begins a conversation with an AI chatbot. The emotion engine recognizes emotions from the user's input data and returns appropriate responses, providing a more natural and friendly conversational experience. For example, if a user inputs, "I want a new jacket," the chatbot will provide specific information such as, "The new jacket is from this year's fall / winter collection and is excellent for keeping you warm. It also comes in a wide variety of colors."

[1902] Prompt Sentence Examples

[1903] Here are some examples of prompts to input to a generative AI model:

[1904] 1. "I want to create an AI chatbot that uses an emotion engine. First, please tell me the steps to upload the character's voice and image data."

[1905] 2. "Give me an example of how a chatbot can tailor its response to a user's emotion when they ask a question."

[1906] 3. "How can a fashion brand effectively communicate new product information through friendly characters?"

[1907] By using these prompts, users can obtain more detailed instructions and explanations about specific tasks and settings, providing a more natural and engaging interaction experience for users and enabling companies to effectively communicate their advertising messages.

[1908] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1909] Step 1:

[1910] Terminal

[1911] A corporate user logs in to the management console. As input, they enter their username and password and click the "Login" button. The system receives this and performs authentication. If successful, they are redirected to the main screen. The output is access to the system's main screen.

[1912] Step 2:

[1913] Terminal

[1914] A company user uploads the voice data, image data, and character script of a character. As input, they specify these files in the respective upload forms and click the "Upload" button. The files are sent to the server. The output is the data correctly saved on the server.

[1915] Step 3:

[1916] Terminal

[1917] Corporate users configure the emotion engine initially and adjust the emotion model parameters. As input, they set emotion recognition accuracy and emotion category parameters on the settings screen. The system receives this and saves the emotion engine configuration data. The output is the adjusted emotion model.

[1918] Step 4:

[1919] server

[1920] The server analyzes the voice data and synthesizes the voice. The uploaded voice data is used as input. The server converts the voice to text using a natural language processing (NLP) engine, and then generates a new voice using a speech synthesis engine based on the text. The output is the generated voice data.

[1921] Step 5:

[1922] server

[1923] The server analyzes the image data and generates a character avatar. The uploaded image data is used as input. The server uses an image analysis engine to generate a 3D avatar from the image. The output is the generated character avatar.

[1924] Step 6:

[1925] server

[1926] The server analyzes the character script and trains the AI ​​model to recognize specific phrases and tones. The uploaded character script is used as input. A natural language processing (NLP) engine analyzes the script, extracts specific patterns, and trains the AI ​​model. The output is a trained AI model.

[1927] Step 7:

[1928] server

[1929] The server receives product information, advertising messages, and FAQ data provided by companies and customizes the AI ​​model. These data are used as input. The server analyzes them and uses them as necessary training data to strengthen the AI ​​model. The output is a customized AI model.

[1930] Step 8:

[1931] server

[1932] The server uses an emotion engine to recognize emotions from user input data and dynamically adjust responses. The input is text and voice data from the user's interaction with the chatbot. The server analyzes this using the emotion engine and adjusts the chatbot's response based on the results. The output is a response based on the emotion.

[1933] Step 9:

[1934] Terminal

[1935] Corporate users can preview the generated AI chatbot and make adjustments as needed. The input is a simulated interaction on the preview screen. Based on user feedback, the server adjusts the settings. The output is the adjusted chatbot.

[1936] Step 10:

[1937] server

[1938] The server deploys and integrates the final AI chatbot on a specific platform. The completed chatbot data is used as input. A deployment tool is used to integrate the chatbot into a specified website or application. The output is a chatbot running in a production environment.

[1939] Step 11:

[1940] User

[1941] An end user accesses a company's platform and starts a conversation with an AI chatbot. As input, they type a message into the chat window of a website or application. The system receives this and starts a conversation. The output is the content of the conversation with the chatbot.

[1942] Step 12:

[1943] server

[1944] The server dynamically adjusts responses based on the user's emotions. The inputs are the user's speech and the results of emotion analysis. Based on this, the server selects an appropriate response tone, and the chatbot responds to the user. The output is a response adjusted according to the user's emotions.

[1945] Step 13:

[1946] server

[1947] The server collects the interaction history and feedback and uses it to improve the model. The user's interaction history and feedback are used as input. These data are used for the next model training. The output is an improved AI model.

[1948] (Application example 2)

[1949] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1950] In recent years, AI chatbots have been used on various platforms, but to improve the user experience, more natural and emotionally adaptive responses are required. Conventional systems lack the ability to recognize and dynamically adjust to user emotions, and responses are often mechanical. Furthermore, content distribution services lack mechanisms for providing personalized content based on user emotions. To address these issues, the present invention aims to provide an AI chatbot system combined with an emotion recognition engine to improve the user experience.

[1951] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving voice data and image data of a specific character or celebrity; means for analyzing the received voice data and performing voice synthesis; means for analyzing image data and generating a character avatar; means for receiving company product information, advertising messages, and FAQ data and training an AI model based on them; means for analyzing a user's emotions using an emotion recognition engine and adjusting responses based on the analyzed emotions; means for processing user input data in a content distribution service platform and providing personalized content; means for voice synthesizing a response based on the emotion of a specific character; and means for visualizing a character avatar and coordinating the visual display with audio playback. This makes it possible to provide appropriate responses and personalized content according to the user's emotions.

[1952] "Voice data" refers to information that digitally records the voice of a user or a specific character.

[1953] "Image data" refers to visual information of characters or celebrities recorded in digital format.

[1954] "Speech synthesis" is a technology that uses a computer program to generate a human voice based on text data.

[1955] A "character avatar" is a digital icon or persona that visually represents a particular character.

[1956] A "character script" is a collection of lines and responses that a character utters, each with a specific phrase and tone of voice.

[1957] An "AI model" is a collection of algorithms trained to perform a specific task using artificial intelligence techniques.

[1958] An "emotion recognition engine" is an algorithm or tool that analyzes emotions from user input data and identifies those emotions.

[1959] "Personalized content" is digital content that is individually tailored based on a user's interests and preferences.

[1960] "Deployment" is the process of implementing software or an application on a particular platform.

[1961] A "server" is a computer system that provides services to other computers over a network.

[1962] A "platform" is the underlying hardware or software environment on which an application runs.

[1963] A "natural language processing model" is an algorithm used by computers to understand and generate human language.

[1964] The system for realizing the present invention includes the following components and means: Specific components and their respective processes will be described in detail below.

[1965] System Configuration

[1966] This system uses the following hardware and software:

[1967] Hardware:

[1968] Smartphone

[1969] Head-mounted display (HMD)

[1970] server

[1971] software:

[1972] Google Cloud Natural Language API

[1973] Google Cloud Text-to-Speech API

[1974] Unity

[1975] Program processing overview

[1976] The server has the following features:

[1977] 1. Receiving and analyzing audio and image data:

[1978] It receives voice and image data of characters and celebrities, converts the voice data into text using the Google Cloud Natural Language API, and then synthesizes speech based on that text using the Google Cloud Text-to-Speech API.

[1979] Image data is used to generate and visualize character avatars using Unity.

[1980] 2. Dynamic emotion recognition and response adjustment:

[1981] The user's input data (text) is subjected to sentiment analysis using the Google Cloud Natural Language API, and the sentiment score and strength of the sentiment are calculated.

[1982] Based on this, the character's response is selected and speech synthesis is performed using the Google Cloud Text-to-Speech API.

[1983] The terminal has the following features:

[1984] 1. Providing interaction:

[1985] It provides a user interface (UI) that allows users to initiate a dialogue with the AI ​​chatbot via a smartphone or HMD.

[1986] Visualize a character avatar and play synthesized voice in sync.

[1987] The user interacts with:

[1988] 1. Start the conversation:

[1989] Initiate a conversation with the AI ​​chatbot using an application on your smartphone or HMD.

[1990] Based on user input (text or voice), the system performs sentiment analysis and provides responses and content recommendations based on that.

[1991] Specific examples

[1992] For example, when a user expresses their opinion about a particular movie on a movie streaming service, the service will suggest movies based on the sentiment analysis results. The prompt sentence is as follows:

[1993] Example prompt sentence:

[1994] Input Text: "The movie I saw recently was very moving. It made me cry."

[1995] Emotion score: high positive emotion, strong emotional intensity

[1996] AI chatbot response: "So you like inspiring movies! I recommend 'XYZ' as a tear-jerking movie. I'll play a preview for you."

[1997] In this way, by providing content that corresponds to the user's emotions, it is possible to build a system that can provide a more natural and attractive interactive experience.

[1998] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1999] Step 1:

[2000] Receiving and analyzing audio and image data

[2001] The server receives audio data and image data of specific characters and celebrities provided by the company.

[2002] The received audio data is sent to the Google Cloud Natural Language API and converted into text, which is later used for speech synthesis.

[2003] The image data is analyzed by Unity and a character avatar is generated.

[2004] Input: Voice and image data provided by the company

[2005] Output: Speech data converted to text and generated character avatar

[2006] Step 2:

[2007] Emotion recognition engine for analyzing user emotions

[2008] The user inputs text or voice using a smartphone or HMD.

[2009] The user's input data (text or voice) is sent to the server and analyzed for sentiment scores and sentiment strength using the Google Cloud Natural Language API.

[2010] The results of this sentiment analysis are used to dynamically tailor responses.

[2011] Input: User-entered data (text or voice)

[2012] Output: Sentiment score and intensity of the emotion

[2013] Step 3:

[2014] Response generation and speech synthesis

[2015] The server selects an appropriate response based on the emotion score and the intensity of the emotion.

[2016] The selected response text is synthesized using the Google Cloud Text-to-Speech API and output in the appropriate character's voice.

[2017] The synthesized voice file is sent to the user's terminal.

[2018] Input: Sentiment analysis results, response text

[2019] Output: A synthesized response

[2020] Step 4:

[2021] Character Avatar Visualization

[2022] The server transmits the character avatar along with the synthesized voice to the user's terminal.

[2023] The user's device uses Unity to synchronize the display of the character avatar and the playback of the audio.

[2024] This allows users to experience a natural interaction both visually and aurally.

[2025] Input: Character avatar, audio file

[2026] Output: Visualized avatar and audio playback

[2027] Step 5:

[2028] Collection of conversation history and feedback

[2029] The server collects user interaction history and feedback to help improve the model in future.

[2030] The collected data is used to train AI models.

[2031] Input: Dialogue history, user feedback

[2032] Output: New training data

[2033] Specific working example:

[2034] For example, if a user enters the text, "The movie I saw recently was very moving. It brought me to tears," the system performs sentiment analysis on the text and detects high positive and strong emotional intensity. Based on the results, the system generates a response to the user via synthesized voice: "I see you like moving movies! I recommend 'XYZ' as a tear-jerking movie. I'll play a preview." A character avatar is also displayed at the same time, enhancing the naturalness of the interaction visually and aurally.

[2035] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2036] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2037] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2038] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2039] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2040] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2041] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2042] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2043] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2044] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2045] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2046] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2047] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2048] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2049] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2050] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2051] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2052] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2053] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2054] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2055] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2056] The following is further disclosed regarding the above embodiment.

[2057] (Claim 1)

[2058] means for receiving audio data and image data of specific characters or celebrities;

[2059] means for analyzing received voice data and performing voice synthesis;

[2060] A means for analyzing image data and generating a character avatar;

[2061] A way to analyze character scripts and incorporate unique phrases and tones into the AI ​​model,

[2062] A means to receive a company's product information, advertising messages, and FAQ data and use them to train an AI model;

[2063] A way to preview and adjust the generated AI chatbot,

[2064] The means to deploy and integrate the final AI chatbot into a specific platform,

[2065] A system including:

[2066] (Claim 2)

[2067] A means to train a natural language processing model to reproduce the character's distinctive phrasing and tone of voice; and

[2068] A means for providing a UI for a user to initiate a dialogue with the AI ​​chatbot;

[2069] A means of collecting user interaction data and using it to improve the system;

[2070] The system of claim 1 further comprising:

[2071] (Claim 3)

[2072] A method to input company product information data into an AI model, extract relevant items, and integrate them into the character script.

[2073] A means to deploy AI chatbots using deployment tools that verify integration with the platform;

[2074] means for transmitting a dialogue history including the content of an inquiry from a user and feedback to a server;

[2075] The system of claim 1 further comprising:

[2076] "Example 1"

[2077] (Claim 1)

[2078] A means for corporate users to log in;

[2079] means for receiving audio data and image data of specific characters or celebrities;

[2080] A means for ...

Claims

1. means for receiving audio data and image data of specific characters or celebrities; means for analyzing received voice data and performing voice synthesis; A means for analyzing image data and generating a character avatar; A way to analyze character scripts and incorporate unique phrases and tones into the AI ​​model, A means to receive a company's product information, advertising messages, and FAQ data and use them to train an AI model; A way to preview and adjust the generated AI chatbot, The means to deploy and integrate the final AI chatbot into a specific platform, A system including:

2. A means to train a natural language processing model to reproduce the character's distinctive phrasing and tone of voice; and A means for providing a UI for a user to initiate a dialogue with the AI ​​chatbot; A means of collecting user interaction data and using it to improve the system; The system of claim 1 further comprising:

3. A method to input company product information data into an AI model, extract relevant items, and integrate them into the character script. A means to deploy AI chatbots using deployment tools that verify integration with the platform; means for transmitting a dialogue history including the content of an inquiry from a user and feedback to a server; The system of claim 1 further comprising:

Images