System

Abstract

To provide a system.SOLUTION: A system includes: means for preprocessing data on the basis of voice characteristics and personality characteristics set by a user; means for training an AI learning model by using the preprocessed data; means for disclosing the trained AI learning model on a disclosable platform; means for enabling another user to purchase the disclosed AI learning model; means for using an emotion engine to analyze an emotion state of the another user in real time; and means for adjusting an interactive response of the AI learning model on the basis of a result of the analysis of the another user's emotion state.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, there are AIs that mimic human vocal cords and personalities, but there is no platform where users can create their own ideal vocal cords and personality, make it public, and share it with other users. This means that users cannot create AI with their ideal vocal cords and personality and share it with other users. [Means for solving the problem]

[0005] The present invention provides a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with users using the AI ​​learning model, and a means for creating, purchasing, and selling the AI ​​learning model on a public platform, which allows users to create AI with their ideal vocal cords and personality and share it with other users. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15]FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16] FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] The present invention is a system including a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with a user using the AI ​​learning model, and a means for creating, purchasing, and selling the AI ​​learning model on a platform that can be made public. Specifically, a user sets his or her ideal vocal cords and personality on the platform, and an AI learning model is created based on the settings. The created AI learning model is used for interacting with the user and is made public to other users on the platform. The published AI learning model can be purchased and sold by other users.

[0029] "Example 2"

[0030] As a specific example, User A sets his / her ideal vocal range and personality on the platform, and an AI learning model is created based on that setting. The created AI learning model is used for dialogue with User A and is also made available to other users on the platform. The published AI learning model can be purchased by another User B and used for dialogue with User B.

[0031] The process flow of each embodiment will be described below.

[0032] "Example 1"

[0033] Step 1: The user sets their ideal vocal range and personality on the platform. This setting includes vocal characteristics such as pitch, speed, and intonation, and personality characteristics such as the range and intensity of emotional expression such as joy, anger, sadness, and happiness.

[0034] Step 2: An AI learning model is created based on the settings. This AI learning model has the ability to mimic the user's ideal vocal chords and personality.

[0035] Step 3: The created AI learning model is used to interact with the user, who can then interact with the AI ​​that has the vocal range and personality of their dreams.

[0036] Step 4: The created AI learning model is made publicly available to other users on the platform. The published AI learning model can be purchased or sold by other users.

[0037] "Example 2"

[0038] Step 1: User A sets his / her ideal vocal cords and personality on the platform. This setting includes vocal cord characteristics such as voice pitch, speed, intonation, and personality characteristics such as range and intensity of emotional expression such as joy, anger, sadness, and happiness.

[0039] Step 2: An AI learning model is created based on the settings. This AI learning model has the ability to mimic User A's ideal vocal cords and personality.

[0040] Step 3: The created AI learning model is used to interact with User A. User A can interact with an AI that has the vocal range and personality of their ideal user.

[0041] Step 4: The created AI learning model is made available to other users on the platform. The published AI learning model is purchased by other user B.

[0042] Step 5: The purchased AI learning model is used to interact with User B. User B can interact with an AI that has User A's ideal vocal range and personality.

[0043] Example 1

[0044] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0045] Conventional AI learning models have difficulty reflecting the user's ideal vocal cords and personality, and it is also difficult to share or sell the generated models with other users. For this reason, it has not been possible to create AI learning models that meet the diverse needs of users and to effectively use those models. Furthermore, there has been a lack of a platform for users to share and sell the models they have generated with other users.

[0046] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0047] In this invention, the server includes a means for a user to access the platform and set an ideal vocal cord and personality, a means for the server to receive the user's settings and generate an AI learning model using voice synthesis software and a personality model generation algorithm, and a means for the server to store the generated AI learning model on the platform and enable the user to interact with it. This allows a user to easily create an AI learning model that reflects their ideal vocal cord and personality, and to share or sell the model with other users.

[0048] "User" refers to an individual or organization that uses the system to set their ideal vocal range and personality, and to create, interact with, publish, purchase, and sell AI learning models.

[0049] The "Platform" refers to an online system that allows Users to access and set their ideal vocal range and personality, and to create, publish, purchase, and sell AI learning models.

[0050] "Server" means a computer system that receives user settings, generates AI learning models using voice synthesis software and personality model generation algorithms, and stores, publishes, and sells them on the Platform.

[0051] "Voice synthesis software" refers to software for generating voice data based on the characteristics of the vocal cords set by the user.

[0052] The "personality model generation algorithm" is an algorithm for generating personality data based on personality characteristics set by the user.

[0053] An "AI learning model" is an artificial intelligence model that mimics a user's vocal cords and personality, generated based on the user's settings.

[0054] "Dialogue" refers to the act of a user communicating with an AI learning model through text or voice.

[0055] "Publication" refers to displaying the AI ​​learning model created by a user on the platform so that other users can view and purchase it.

[0056] "Purchase" refers to the act of other users paying a fee to obtain a published AI learning model.

[0057] "Sales" refers to the act of a user providing an AI learning model they have created to other users and receiving payment in return.

[0058] "Right to Use" means the right of the Purchaser to use the AI ​​learning model acquired.

[0059] This invention is a system in which a user sets their ideal vocal cords and personality, generates an AI learning model based on that, and then interacts with, publishes, purchases, and sells the model. A specific embodiment of this system will be described below.

[0060] First, the user accesses the platform using a terminal (PC, smartphone, tablet, etc.). The platform provides an interface for setting the vocal range and personality. The user selects the pitch of the voice, tone, speaking speed, and personality traits (e.g. friendly, calm, humorous, etc.).

[0061] The server then receives the user-entered configuration data. The server uses voice synthesis software (e.g., Google® Text-to-Speech API) and personality model generation algorithms (e.g., OpenAI®'s GPT-3®) to generate voice data and personality data based on the user's configuration. These data are then integrated to create an AI learning model.

[0062] The server stores the generated AI learning model on the platform. The user can then interact with the AI ​​learning model through the device. When interacting, the text or voice entered by the user is sent to the server, which uses the AI ​​learning model to generate an appropriate response and returns it to the device.

[0063] For example, if a user types "What's the weather today?" into a chat window, the server will use the AI ​​learning model to generate a response such as "It's sunny today" and display it in the chat window.

[0064] Furthermore, if the user wishes, the server will publish the generated AI learning model on the platform. The published model will be available for other users to view. Other users can select a model they are interested in from the published list and click the "Purchase" button. Once the purchase procedure is complete, the server will grant the purchaser the right to use the AI ​​learning model. The purchaser will be able to use this model through their device.

[0065] As a specific example, User A wants to create an AI learning model with a "calm voice and humorous personality." User A sets the following through the interface: "Voice pitch: medium-low," "Tone: calm," and "Personality: humorous." The server receives these settings and uses voice synthesis software and a personality model generation algorithm to generate voice data and personality data based on the settings, creating an AI learning model. The created model can then be used by User A for conversations. User A can also publish this model on the platform, making it available for other users to purchase.

[0066] An example of a prompt is, "Please create an AI learning model with a calm voice and a humorous personality. Please use a medium-low pitch and a calm tone."

[0067] In this way, the system allows AI learning models to be created, interacted with, published, and sold, with servers, terminals, and users each playing their respective roles.

[0068] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0069] Step 1:

[0070] Users access the platform and set their ideal vocal range and personality.

[0071] Specifically, the user uses a terminal to access the platform's URL and log in. Then, they go to the settings screen and input their voice pitch, tone, speaking speed, and personality traits.

[0072] Input: User's vocal and personality settings (e.g. voice pitch, mid-low, tone, calm, personality, humorous)

[0073] Output: The configuration data is sent to the server.

[0074] Step 2:

[0075] The server receives the user's settings and generates an AI learning model.

[0076] Specifically, the server analyzes the received configuration data, calls voice synthesis software (e.g., Google Text-to-Speech API) to generate voice data, and simultaneously generates personality data using a personality model generation algorithm (e.g., OpenAI's GPT-3). These data are then integrated to create an AI learning model.

[0077] Input: User's configuration data

[0078] Output: The generated AI learning model

[0079] Step 3:

[0080] The server stores the generated AI learning model on the platform and allows users to interact with it.

[0081] Specifically, the server stores the generated AI learning model in a database. When the user starts a dialogue, the server calls the model and generates a response according to the user's input.

[0082] Input: Generated AI learning model

[0083] Output: AI learning model stored on the platform

[0084] Step 4:

[0085] The user interacts with the AI ​​learning model.

[0086] Specifically, a user inputs text into a chat window through a terminal. The server receives the input text, uses the AI ​​learning model to generate an appropriate response, and displays it in the chat window.

[0087] Input: User text input (e.g. "What's the weather like today?")

[0088] Output: Response from the AI ​​learning model (e.g. "It's sunny today")

[0089] Step 5:

[0090] The server publishes the user's AI learning model on the platform.

[0091] Specifically, when a user clicks the "Publish" button, the server adds the model to a published list, from which other users can view the model.

[0092] Input: User's publishing instructions

[0093] Output: AI learning model added to the public list

[0094] Step 6:

[0095] Other users can purchase published AI learning models.

[0096] Specifically, other users select a model they are interested in from the public list and click the "Purchase" button. Once the purchase procedure is completed, the server grants the purchaser the right to use the model.

[0097] Input: Purchase instructions from other users

[0098] Output: The model usage rights granted to the buyer.

[0099] Step 7:

[0100] The server grants the buyer the rights to use the model.

[0101] Specifically, the server adds the model usage rights to the buyer's account, allowing the buyer to use the model through their device.

[0102] Input: Purchase procedure completion notification

[0103] Output: Model usage rights added to buyer's account

[0104] (Application example 1)

[0105] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0106] The shopping experience in conventional virtual stores had the problem that users needed a lot of time and effort to obtain product information. In addition, it was difficult for users to find products that matched their preferences and needs, and the shopping experience was often unsatisfactory. Furthermore, the conversational assistant was not customized based on the user's vocal cords or personality, so the conversation with the user was sometimes unnatural. To solve these problems, it is necessary to provide a conversational assistant using an AI learning model that mimics the user's ideal vocal cords and personality.

[0107] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0108] In this invention, the server includes a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with a user using the AI ​​learning model, a means for creating, purchasing, and selling the AI ​​learning model on a platform that can be made public, an interactive assistant means for supporting the user's shopping in a virtual store, a means for installing the interactive assistant means in a smartphone or a head-mounted display, and a means for the interactive assistant means to provide product information based on the user's ideal vocal cords and personality. This allows the user to improve their shopping experience in the virtual store through an interactive assistant based on their own ideal vocal cords and personality.

[0109] An "AI learning model for mimicking human vocal cords and personality" is an artificial intelligence learning model designed to reproduce the characteristics of a human voice, speaking style, and personality.

[0110] "Means of interacting with a user" refers to methods and technologies for achieving communication with a user.

[0111] A "publishable platform" is an online system that enables users to share, buy, or sell user-created content or data with other users.

[0112] A "virtual store" is a virtual store operated on the Internet where users can browse and purchase products online.

[0113] An "interactive assistant means" is a technique or method for providing information or support to a user through dialogue.

[0114] A "smartphone" is a highly functional mobile phone that can connect to the Internet and use applications.

[0115] A "head-mounted display" is a display device that provides visual information to a user when worn on the head.

[0116] "Means for providing product information" refers to methods or techniques for conveying detailed product information and features to users.

[0117] In order to implement the present invention, the following system configuration and program are required.

[0118] First, the server creates an AI learning model to mimic human vocal cords and personality. This AI learning model is generated based on the ideal vocal cords and personality set by the user. Specifically, the user sets their ideal vocal cords and personality on the platform and sends the setting information to the server. The server generates an AI learning model based on this information. The generated AI learning model is used to interact with the user.

[0119] The server then provides a means for interacting with the user using the generated AI learning model. The user can do this through a smartphone or a head-mounted display. The interactive assistant generates appropriate responses based on the user's input and plays them aloud using a speech synthesis engine.

[0120] In addition, the server provides a means to create, buy and sell AI learning models on a public platform where users can publish their ideal vocal range and personality, and other users can buy and sell the AI ​​learning models.

[0121] In the virtual store, the interactive assistant means assists the user in shopping. Specifically, when the user asks a question about a product, the interactive assistant means uses an AI learning model to provide appropriate product information. For example, if the user asks, "I'm looking for new running shoes. Do you have any recommendations?", the interactive assistant means responds, "How about these running shoes? They're lightweight and highly cushioned, making them ideal for long runs."

[0122] The following hardware and software are used to realize this system:

[0123] Hardware: Servers, smartphones, head-mounted displays

[0124] Software: OpenAI API, speech synthesis engine

[0125] Examples of specific prompts are as follows:

[0126] User: "I'm looking for new running shoes. Any recommendations?"

[0127] Assistant: "How about these running shoes? They're lightweight and well-cushioned, perfect for long runs."

[0128] In this way, users can enhance their shopping experience in virtual stores through a conversational assistant based on their ideal vocal range and personality.

[0129] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0130] Step 1:

[0131] Users access the platform and set their ideal vocal range and personality.

[0132] Input: User's vocal and personality settings

[0133] Output: Configuration information sent to the server

[0134] How it works: Through the platform's interface, the user inputs the tone of voice, speaking style, personality traits, etc. This configuration information is sent to the server.

[0135] Step 2:

[0136] The server generates an AI learning model based on the configuration information received.

[0137] Input: User's vocal and personality settings

[0138] Output: AI learning model

[0139] Specific operation: The server analyzes the received configuration information and uses it as data to generate an AI learning model. It uses the OpenAI API to generate an AI learning model based on the configuration information.

[0140] Step 3:

[0141] The generated AI learning model is used to prepare for interaction with the user.

[0142] Input: AI learning model

[0143] Output: Interaction ready state

[0144] Specific operation: The server integrates the generated AI learning model into the dialogue system, making it possible to dialogue with the user.

[0145] Step 4:

[0146] Users access the conversational assistant through a smartphone or head-mounted display.

[0147] Input: User's access request

[0148] Output: Launch of interactive assistant

[0149] Specific operation: A user starts an application on a smartphone or head-mounted display to access the conversational assistant. The application connects to the server and starts the conversational assistant.

[0150] Step 5:

[0151] The user inputs a question to the interactive assistant.

[0152] Input: User question

[0153] Output: The question data is sent to the server.

[0154] Specific operation: The user inputs a question to the conversational assistant by text or voice, and the input question is sent to the server.

[0155] Step 6:

[0156] The server generates a response based on the user's question.

[0157] Input: User question data

[0158] Output: Response data

[0159] Specific operation: The server analyzes the received question data and generates appropriate responses using an AI learning model. It uses the OpenAI API to generate natural responses to questions.

[0160] Step 7:

[0161] The generated response is converted into speech by a speech synthesis engine.

[0162] Input: Response data

[0163] Output: Audio data

[0164] Specific operation: The server passes the generated response data to the speech synthesis engine to generate voice data.

[0165] Step 8:

[0166] The audio data is sent to the user's terminal and played back.

[0167] Input: Audio data

[0168] Output: Audio response to the user

[0169] Specific operation: The server transmits the generated voice data to the user's smartphone or head-mounted display. The user's device plays the received voice data and provides the user with a response.

[0170] In this way, users can enhance their shopping experience in virtual stores through a conversational assistant based on their ideal vocal range and personality.

[0171] Example 2

[0172] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0173] Conventional AI learning models make it difficult to set vocal cords and personality according to individual user needs, and the generated models are not easy to publish or for other users to purchase. For this reason, there was a demand for a system that would allow users to easily create AI models that reflect their ideal vocal cords and personality, and share them with other users.

[0174] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for a user to set his / her ideal vocal cords and personality, a means for receiving the setting information, collecting voice data and text data, and performing preprocessing, a means for generating an AI learning model using the preprocessed data, a means for publishing the generated AI learning model on a platform where it can be published, and a means for other users to purchase the published AI learning model and use it in conversations. This makes it possible for a user to easily create an AI model that reflects his / her ideal vocal cords and personality, and share it with other users.

[0175] "User" refers to an individual or organization that uses the system to set their ideal vocal range and personality and generate and use an AI learning model.

[0176] "Vocal chords" refers to the voice characteristics such as tone, pitch, and speed of the voice that are set by the user.

[0177] "Personality" refers to the personality traits of the AI ​​learning model set by the user (e.g. kind, strict, humorous, etc.).

[0178] "Settings Information" refers to information regarding vocal chords and personality that a User inputs on the Platform.

[0179] "Voice data" refers to voice data collected based on a user's vocal cord settings.

[0180] "Text data" refers to text data collected based on a user's personality settings.

[0181] "Preprocessing" refers to the process of converting collected voice and text data into a format suitable for training an AI learning model.

[0182] "AI learning model" refers to an artificial intelligence model that is generated based on user settings information.

[0183] "Publication" refers to displaying the generated AI learning model on the platform so that other users can access it.

[0184] "Platform" refers to the online system through which Users can set their vocal range and personality, and generate, publish, purchase and use AI learning models.

[0185] "Purchase" refers to the transactional act of other users acquiring published AI learning models.

[0186] "Dialogue" refers to communication between a user and an AI learning model.

[0187] This invention is a system that allows users to set their ideal vocal cords and personality, and generate, publish, purchase, and use AI learning models based on that. A specific embodiment of this system is described below.

[0188] User configuration input

[0189] Users use the platform's web interface to configure their ideal vocal range and personality. Specifically, users select the tone, pitch, speed, and personality traits (e.g., kind, stern, humorous, etc.) of their voice. These configuration information is sent to the server as form input.

[0190] Data collection and preprocessing

[0191] The server receives user configuration information and collects necessary data. It collects voice and text data and performs preprocessing. Preprocessing includes normalizing voice data and tokenizing text data. It uses Python (registered trademark) and TENSORFLOW (registered trademark).

[0192] Generate AI learning models

[0193] The server uses the preprocessed data to generate an AI learning model, uses deep learning techniques to build a neural network using TensorFlow or PyTorch, and trains the model based on the user's settings.

[0194] Publishing the model

[0195] The server publishes the generated AI learning model on the platform. The model is uploaded to cloud storage (e.g. AWS (registered trademark) S3) and a public URL is generated. This URL is displayed on the platform so that other users can access it.

[0196] Purchase and Use by Other Users

[0197] User B purchases the published AI learning model on the platform. The purchase procedure is done through a web interface, and payment is processed using a payment service. After the purchase, User B can use the AI ​​learning model in his or her own interactions.

[0198] Examples

[0199] Suppose User A sets "gentle voice, humorous personality" on the platform. The server receives this setting information, collects voice and text data, and performs preprocessing. Then, it uses TensorFlow to generate an AI learning model and uploads it to cloud storage. The published model is purchased by User B and used to interact with User B.

[0200] Examples of prompt statements

[0201] "Generate an AI model with a gentle voice and a humorous personality."

[0202] This system provides a series of processes for users to set their ideal vocal range and personality, and then generate, publish, purchase, and use AI learning models based on that, allowing users to easily create AI models that meet their needs and share them with other users.

[0203] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0204] Step 1: Enter user settings

[0205] A user configures his / her ideal vocal range and personality using a web interface on the platform. Specifically, the user selects the tone, pitch, speed, and personality traits (e.g., kind, stern, humorous, etc.) of the voice. These configuration information are sent to the server as form input. The input is the user's configuration information, and the output is the configuration information sent to the server.

[0206] Specific behavior:

[0207] A user visits a web page and fills out a configuration form.

[0208] The user clicks the "Submit" button.

[0209] The server receives the configuration information and stores it in a database.

[0210] Step 2: Collect and preprocess data

[0211] The server receives the user's configuration information and collects the necessary data. It collects voice and text data and performs preprocessing. Preprocessing includes normalizing the voice data and tokenizing the text data. It uses Python and TensorFlow. The input is the user's configuration information and the output is the preprocessed data.

[0212] Specific behavior:

[0213] The server collects the voice data based on the setting information.

[0214] The server normalizes the audio data and tokenizes the text data.

[0215] The server temporarily stores the pre-processed data.

[0216] Step 3: Generate an AI learning model

[0217] The server uses the preprocessed data to generate an AI learning model. It uses deep learning techniques to build a neural network using TensorFlow or PyTorch. It trains the model based on the user's settings. The input is the preprocessed data, and the output is the trained AI learning model.

[0218] Specific behavior:

[0219] The server uses TensorFlow to build a neural network.

[0220] The server trains the model using the preprocessed data.

[0221] The server stores the trained model.

[0222] Step 4: Publish the model

[0223] The server publishes the generated AI learning model on the platform. The model is uploaded to cloud storage (e.g. AWS S3) and a public URL is generated. This URL is displayed on the platform so that other users can access it. The input is the trained AI learning model, and the output is the public URL.

[0224] Specific behavior:

[0225] The server uploads the trained model to cloud storage.

[0226] The server generates a public URL.

[0227] The server displays the URL on the platform.

[0228] Step 5: Purchase and use by others

[0229] User B purchases the published AI learning model on the platform. The purchase procedure is done through a web interface, and payment is processed using a payment service. After the purchase, User B can use the AI ​​learning model in his / her own interactions. The input is the publication URL and User B's purchase information, and the output is an AI learning model that User B can use.

[0230] Specific behavior:

[0231] User B selects a model published on the platform.

[0232] User B completes the purchase procedure.

[0233] The server processes the payment through a payment service.

[0234] User B downloads the purchased model and uses it in the interaction.

[0235] (Application example 2)

[0236] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0237] With conventional voice synthesis technology, it was difficult for users to generate personalized voice content that reflected their ideal vocal range and personality. In addition, there were limited ways to purchase personalized voice models created by other users and enjoy content in their voices. This meant that it was not possible to meet the diverse needs of users, and there was an issue of the individualization of voice content not progressing.

[0238] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0239] In this invention, the server includes a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with a user using the AI ​​learning model, a means for creating, purchasing, and selling the AI ​​learning model on a public platform, a means for a user to set his / her ideal vocal cords and personality and generate an AI learning model based on the setting, a means for providing audio content in a personalized voice using the generated AI learning model, and a means for purchasing an AI learning model created by another user and enjoying content in that voice. This allows a user to generate personalized voice content that reflects his / her ideal vocal cords and personality, and to enjoy a variety of voice content using voice models created by other users.

[0240] The "human vocal cords" are the biological structures used by humans to produce sound and are what determine the pitch and quality of the voice.

[0241] "Personality" refers to an individual's characteristics in behavior, thinking, and emotion, and reflects that person's characteristics in dialogue and communication.

[0242] An "AI learning model" is an algorithm or network structure that uses artificial intelligence technology to learn specific data and patterns and perform tasks such as prediction, classification, and generation.

[0243] A "production means" is a method or device for producing or building something for a particular purpose.

[0244] A "means for interacting" is a method or device by which a user and a system exchange information and communicate with each other.

[0245] A "publishable platform" is an online system that enables users to share and make accessible their created content and models to other users.

[0246] A "buying or selling vehicle" is a transactional method or device by which a user obtains or provides particular content or models.

[0247] A "setting means" is a method or device that allows a user to input his / her preferences and requirements into the system so that the system operates based on them.

[0248] "Audio content" means any form of information or entertainment that is primarily audio, including music, podcasts and audiobooks.

[0249] A "personalized voice" is a voice that is customized based on the preferences and requirements of a particular user, and reflects that user's ideal vocal range and personality.

[0250] An "AI learning model created by another user" is an artificial intelligence model that a different user has generated based on their own settings and that can be used by other users.

[0251] The system for implementing this invention is configured as follows: First, the server has a means for creating an AI learning model for imitating human vocal cords and personality. This means provides an interface for the user to set their ideal vocal cords and personality, and generates an AI learning model based on the settings.

[0252] Next, the server has a means for interacting with the user using the generated AI learning model, which analyzes the text and voice data entered by the user and generates an appropriate response using the AI ​​learning model.

[0253] Additionally, the server has the means to create, buy and sell AI learning models on a publicly accessible platform that allows users to publish their ideal vocal range and personality and allows other users to buy or sell the models.

[0254] The server also has a means for providing audio content in a personalized voice using the generated AI learning model, the means for playing content selected by the user in a voice generated using the AI ​​learning model.

[0255] Finally, the server has a means for purchasing an AI learning model created by another user and enjoying content with that voice. This means enables a user to purchase an AI learning model created by another user and play content using that model.

[0256] Hardware and Software Used

[0257] Hardware: Smartphone (iOS or ANDROID (registered trademark))

[0258] Software: Python, TensorFlow, Keras

[0259] Data processing and calculation

[0260] 1. Data Acquisition:

[0261] It provides an interface for users to set their ideal vocal range and personality, inputting settings such as "calm male voice" or "intelligent and friendly."

[0262] 2. Model construction:

[0263] Using TensorFlow and Keras, we build a Long Short-Term Memory (LSTM) network that learns from speech data and generates personalized speech based on user preferences.

[0264] 3. Model training:

[0265] The system generates training data based on the user's settings and trains the model, which then produces a voice that reflects the user's ideal vocal range and personality.

[0266] 4. Save and publish the model:

[0267] Save the trained model and make it accessible to others on a publicly accessible platform.

[0268] Examples

[0269] User A opens the app and sets his ideal vocal range as "calm male voice" and his personality as "intelligent and friendly." The app generates an AI learning model based on this, and User A can listen to audiobooks in that voice. User B can also purchase User A's model and enjoy podcasts in the same voice.

[0270] Examples of prompt statements

[0271] Set the user's ideal vocal range and personality, for example, "calm male voice" or "intelligent and friendly."

[0272] In this way, a system is provided that allows the user to enjoy content in his or her ideal voice.

[0273] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0274] Step 1:

[0275] The user starts the application using the terminal and sets the ideal vocal range and personality. The user inputs settings such as "calm male voice" or "intelligent and friendly." The input data is the setting information of the user's vocal range and personality. The terminal sends this setting information to the server.

[0276] Step 2:

[0277] The server builds an AI learning model based on the received user configuration information. Specifically, it builds an LSTM (Long Short-Term Memory) network using TensorFlow and Keras. The input data is the user configuration information, and the output data is the initialized AI learning model.

[0278] Step 3:

[0279] The server generates training data based on the user's settings. Specifically, it collects voice data and text data that match the user's ideal vocal cords and personality, and prepares this as a training data set. The input data is the user's settings, and the output data is the training data set.

[0280] Step 4:

[0281] The server trains the AI ​​learning model using the training dataset. Specifically, it inputs the training data into the LSTM network and optimizes the model parameters. The input data is the training dataset, and the output data is the trained AI learning model.

[0282] Step 5:

[0283] The server stores the trained AI learning model and uploads it to a publicly available platform, so that other users can purchase or use the model. The input data is the trained AI learning model, and the output data is the published AI learning model.

[0284] Step 6:

[0285] A user uses a terminal to select a published AI learning model and play audio content. Specifically, the user downloads the selected model and uses the model to play audiobooks or podcasts. The input data is the published AI learning model and audio content, and the output data is the audio content played in a personalized voice.

[0286] Step 7:

[0287] Other users use their devices to purchase the published AI learning model and enjoy the content with that voice. Specifically, other users purchase the published model and play the content using that model. The input data is the published AI learning model and a purchase request, and the output data is the purchased AI learning model and the content played with a personalized voice.

[0288] In this way, users can enjoy content using their ideal voice, and other users can also use the model to enjoy a variety of voice content.

[0289] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0290] "Example 1"

[0291] As one embodiment of the present invention, there is a system of an AI learning model combined with an emotion engine. This system recognizes emotions from the user's voice and adjusts the dialogue response of the AI ​​learning model based on the emotion. Specifically, when a user utters a voice indicating an emotion of joy, the emotion engine recognizes the emotion of joy, and the AI ​​learning model generates a dialogue response corresponding to the emotion of joy based on the recognition result. Also, when a user utters a voice indicating an emotion of sadness, the emotion engine recognizes the emotion of sadness, and the AI ​​learning model generates a dialogue response corresponding to the sadness based on the recognition result. In this way, by combining the emotion engine and the AI ​​learning model, dialogue according to the user's emotional state is possible.

[0292] "Example 2"

[0293] Another embodiment of the present invention is a system in which an emotion engine updates the user's emotional state in real time and adjusts the dialogue response of the AI ​​learning model based on the update. Specifically, if the user changes his / her emotional state during the dialogue, the emotion engine recognizes the change in real time, and the AI ​​learning model adjusts the dialogue response based on the recognition result. For example, if the user shows an emotion of joy at the beginning of the dialogue but shows an emotion of sadness in the middle, the emotion engine recognizes the change in emotion in real time, and the AI ​​learning model adjusts the dialogue response from joy to sadness based on the recognition result. In this way, by combining the emotion engine and the AI ​​learning model, a dialogue according to the user's emotional state in real time is possible.

[0294] The process flow of each embodiment will be described below.

[0295] "Example 1"

[0296] Step 1: A user produces a speech sound that reflects the user's emotional state.

[0297] Step 2: The emotion engine recognizes emotions from the user's voice by analyzing voice tone, pitch, stress, and other elements.

[0298] Step 3: The AI ​​learning model receives the recognition results from the emotion engine and generates a dialogue response corresponding to the emotion, which is adapted to the user's emotional state.

[0299] "Example 2"

[0300] Step 1: A user produces speech during a dialogue that reflects changes in the user's emotional state.

[0301] Step 2: The emotion engine recognizes emotional changes in the user's voice in real time by analyzing changes in tone, pitch, volume, and other elements of the voice.

[0302] Step 3: The AI ​​learning model receives the recognition results from the emotion engine and generates a dialogue response that corresponds to the emotion changes, which is adapted to the changes in the user's emotional state.

[0303] Example 1

[0304] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0305] In conventional conversational AI systems, it was difficult to create a dialogue that reflected the individual voice characteristics and personality traits of the user, and responses were not adjusted sufficiently based on emotion recognition. This made it difficult to provide a natural and friendly conversational experience for users. In addition, the process of publishing, purchasing, and selling the generated AI models was complicated, making it difficult to share models between users.

[0306] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0307] In this invention, the server includes a means for creating a machine learning model for imitating human voice characteristics and personality characteristics, a means for interacting with a user using the machine learning model, a means for creating, purchasing, and selling the machine learning model on an online platform that can be made public, and an emotion recognition means for recognizing emotions from the user's voice and adjusting the dialogue response based on the emotions. This makes it possible to provide a natural and friendly dialogue experience that reflects the individual voice characteristics and personality characteristics of the user. In addition, the process of publishing, purchasing, and selling the generated AI model is simplified, making it easy to share the model among users.

[0308] A "machine learning model" is an algorithm that learns from data to mimic voice characteristics or personality traits and perform specific tasks.

[0309] "Voice characteristics" refers to the physical characteristics of speech, such as pitch, tempo, and volume.

[0310] "Personality traits" refer to characteristics that describe an individual's character, such as friendliness or calmness.

[0311] An "emotion recognition means" is a technology for analyzing emotions from a user's voice and recognizing those emotions.

[0312] "Dialogue response" refers to a reply generated in response to a user's input.

[0313] An "online platform" is a system accessible via the Internet that provides a place for users to create, publish, purchase and sell AI models.

[0314] "User" refers to any individual or entity that uses the System.

[0315] "Public" refers to making the generated AI model available for other users to view.

[0316] "Purchase" refers to the act of other users acquiring a published AI model.

[0317] "Sales" refers to the act of providing the generated AI model to other users.

[0318] This invention is a system for creating a machine learning model for imitating human voice characteristics and personality characteristics and for interacting with a user. Furthermore, this system has a function for publishing, purchasing, and selling the generated machine learning model on an online platform. It also includes an emotion recognition means for recognizing emotions from the user's voice and adjusting the dialogue response based on the emotions.

[0319] Generating a Program

[0320] The server generates a machine learning model based on the ideal voice and personality characteristics set by the user on the online platform. In this case, the server collects voice data and personality data, and uses these data to train the machine learning model. Specifically, the following hardware and software are used:

[0321] Hardware: High performance GPU

[0322] Software: Speech processing libraries, natural language processing libraries (e.g. Transformers)

[0323] Data processing and calculation

[0324] The server performs the following steps to process the voice and personality data:

[0325] 1. The server analyzes the voice data uploaded by the user using a voice processing library and extracts voice features.

[0326] 2. The server analyzes the personality data entered by the user using Transformers and extracts personality features.

[0327] 3. The server combines these features to train a machine learning model and generate a model that mimics the user's ideal voice and personality characteristics.

[0328] User interaction

[0329] The terminal uses the generated machine learning model to converse with the user. Specifically, when the user speaks to the terminal, the terminal transmits the voice to the server. The server recognizes emotions from the voice using an emotion recognition means, and the machine learning model generates an appropriate dialogue response based on the recognized emotion and transmits it back to the terminal. The terminal returns the generated dialogue response to the user by voice.

[0330] Publish, buy and sell on the platform

[0331] The server publishes the generated machine learning model on an online platform. The published model is added to a list so that other users can view it. Users can view the published machine learning model and purchase or sell it.

[0332] Examples of concrete examples and prompts

[0333] Examples

[0334] User A sets up a machine learning model with a "calm voice and friendly personality" on an online platform. The server collects User A's voice data and personality data, and extracts features using a voice processing library and Transformers. It then trains the machine learning model using a high-performance GPU to generate a model that mimics User A's ideal voice and personality characteristics. The generated model is used for conversations with User A and is also made public on the online platform. Another User B can like this model and purchase it.

[0335] Examples of prompt statements

[0336] "Create a machine learning model with a calm voice and a friendly personality. If the user makes a speech that indicates the emotion of happiness, generate a dialogue response that corresponds to that emotion."

[0337] In this way, the server, terminal, and user each fulfill their respective roles, allowing the entire system to operate smoothly.

[0338] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0339] Step 1:

[0340] Users access the platform and set their ideal voice and personality characteristics.

[0341] Specifically, users access the platform using a web browser or a dedicated app, select a "calm voice" and a "friendly personality" on the settings screen, and click the "Complete settings" button.

[0342] Input: User's voice and personality settings

[0343] Output: Configuration information sent to the server

[0344] Step 2:

[0345] The server collects voice and personality data and generates a machine learning model.

[0346] Specifically, the server performs the following actions:

[0347] 1. The server analyzes the audio data uploaded by the user using a voice processing library and extracts audio features (e.g. pitch, tempo, volume, etc.).

[0348] 2. The server analyzes the personality data entered by the user using Transformers and extracts personality features (e.g., friendliness, calmness, etc.).

[0349] 3. The server combines these features and uses a high-performance GPU to train a machine learning model to generate a model that mimics the user's ideal voice and personality characteristics.

[0350] Input: User's voice and personality data

[0351] Output: The generated machine learning model

[0352] Step 3:

[0353] The server publishes the generated machine learning model on the platform.

[0354] Specifically, the server publishes the generated model on the platform and displays detailed information about the model (e.g. vocal characteristics, personality traits, price, etc.) so that other users can view this information.

[0355] Input: A generated machine learning model

[0356] Output: Detailed information about the published machine learning model

[0357] Step 4:

[0358] The device uses machine learning models to interact with the user.

[0359] Specifically, when a user speaks to the device, the device transmits the voice to the server. The server uses emotion recognition means to recognize emotions (e.g., interest, curiosity, etc.) from the voice, and the machine learning model generates an appropriate dialogue response and sends it back to the device. The device then returns this response to the user by voice.

[0360] Input: User's voice input

[0361] Output: Dialogue response

[0362] Step 5:

[0363] Other users can buy or sell published machine learning models.

[0364] Specifically, other users can browse the machine learning models published on the platform and purchase the models they like. Once the purchase is complete, the server grants the buyer the right to use the model. Users can also sell the machine learning models they have created on the platform.

[0365] Input: Details of published machine learning models

[0366] Output: The rights to use the machine learning models purchased or sold

[0367] (Application example 1)

[0368] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0369] Conventional conversational AI systems have had difficulty recognizing a user's emotions and generating appropriate conversational responses. In addition, there was a lack of a way for a user to easily create an AI model that imitates their ideal vocal cords and personality and share it with other users. This limited the user experience and made it difficult to realize personalized conversations.

[0370] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0371] In this invention, the server includes a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with a user using the AI ​​learning model, a means for creating, purchasing, and selling the AI ​​learning model on a platform that can be made public, an emotion recognition means for recognizing the user's emotion and adjusting the dialogue response based on the emotion, and a voice synthesis means for converting the dialogue response generated based on the emotion recognition means into voice. This enables personalized dialogue according to the user's emotion, and allows an AI model that imitates an ideal vocal cord and personality to be easily created and shared with other users.

[0372] An "AI learning model" is an artificial intelligence learning algorithm designed to mimic human vocal cords and personality.

[0373] An "interactive means" is a method or device for communicating with a user using an AI learning model.

[0374] A "platform" is an online system or service for creating, purchasing, and selling AI learning models.

[0375] An "emotion recognition means" is a technology or device for analyzing emotions from a user's voice or text and recognizing those emotions.

[0376] A "speech synthesis means" is any technology or device for converting text and generated dialogue responses into speech.

[0377] The "user's ideal vocal cords" refer to the particular vocal characteristics or quality that the user desires.

[0378] A "user's ideal personality" refers to the particular personality traits and interaction style that a user desires.

[0379] "Publicly accessible platform" refers to an online environment that is accessible to other users and enables them to share, buy, and sell AI learning models.

[0380] "Dialogue response" refers to the reply or reaction that an AI learning model generates in response to user input.

[0381] The following system configuration will be described as an embodiment of the present invention.

[0382] System Configuration

[0383] Hardware

[0384] Servers: Servers with high-performance computing power are required, including servers equipped with GPUs for training AI learning models, emotion recognition, and speech synthesis.

[0385] Devices: Includes mobile devices such as smartphones and tablets used by users.

[0386] software

[0387] TensorFlow: Used for training and inference of AI learning models.

[0388] Transformers library: A library for using natural language processing models such as GPT-2.

[0389] EmotionRecognizer library: A library for recognizing emotions from user voice and text.

[0390] TextToSpeech library: A library for converting text to speech.

[0391] Processing flow

[0392] Server Processing

[0393] The server generates an AI learning model based on the ideal vocal range and personality set by the user. The generated model is used to generate appropriate dialogue responses to user input. It also uses emotion recognition means to analyze the user's emotions and adjust the dialogue responses according to those emotions.

[0394] Terminal handling

[0395] The terminal receives input from the user and transmits it to the server, and receives a dialogue response returned from the server, converts it into speech using a speech synthesis means, and provides it to the user.

[0396] Examples

[0397] Configuring Users

[0398] Users can set their ideal vocal range and personality through a smartphone application. For example, if a user desires a "calm voice and a gentle personality," the setting information is sent to the server.

[0399] Generate AI learning models

[0400] The server generates an AI learning model based on the user's settings, which mimics the user's ideal vocal range and personality.

[0401] Emotion Recognition and Dialogue Response

[0402] If a user types, "I'm very tired today," the device sends the input to the server, which uses the EmotionRecognizer library to recognize the user's emotion as "tired" and uses the GPT-2 model to generate a dialogue response such as, "That's tough. Please take some rest."

[0403] Speech synthesis

[0404] The generated dialogue response is converted into speech using the TextToSpeech library and provided to the user via the terminal.

[0405] Examples of prompt statements

[0406] User: I'm really tired today

[0407] Assistant: That's tough. Please take a break.

[0408] In this way, personalized dialogue based on the user's emotions becomes possible, and AI models that mimic ideal vocal chords and personality can be easily created and shared with other users.

[0409] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0410] Step 1:

[0411] Users set their ideal vocal range and personality through a smartphone application.

[0412] Input: User settings (ideal vocal cords and personality)

[0413] Output: Configuration information sent to the server

[0414] Specific operation: The user selects the desired vocal characteristics (e.g., calm voice) and personality (e.g., gentle personality) on the application's settings screen, and saves the settings information. The saved information is sent to the server.

[0415] Step 2:

[0416] The server generates an AI learning model based on the user's settings.

[0417] Input: User settings information

[0418] Output: An AI learning model that mimics the user's ideal vocal chords and personality.

[0419] How it works: The server uses TensorFlow and the Transformers library to train an AI learning model based on the user's preferences. The trained model mimics the user's ideal vocal chords and personality.

[0420] Step 3:

[0421] A user initiates a dialogue through a smartphone application.

[0422] Input: User input (text or voice)

[0423] Output: User input is sent to the server

[0424] Specific operation: The user inputs text or voice into the application's dialogue screen. The input data is sent to the server.

[0425] Step 4:

[0426] The server recognizes emotions based on the user's input.

[0427] Input: Data entered by the user

[0428] Output: Recognized emotion information

[0429] Specific operation: The server uses the EmotionRecognizer library to analyze emotions from the user's input data. As a result of the analysis, the user's emotional information is obtained.

[0430] Step 5:

[0431] The server generates a dialogue response based on the recognized emotion information.

[0432] Input: User input data, recognized emotion information

[0433] Output: Dialogue response (text)

[0434] How it works: The server uses the GPT-2 model to generate appropriate dialogue responses based on the user's input data and the recognized emotion information, and outputs the generated dialogue responses in text format.

[0435] Step 6:

[0436] The server converts the generated dialogue response into speech.

[0437] Input: Dialogue response (text)

[0438] Output: Dialogue response (audio)

[0439] What happens: The server uses the TextToSpeech library to convert the generated dialogue response into speech, which is then sent to the device.

[0440] Step 7:

[0441] The terminal provides the voice data to the user.

[0442] Input: Dialogue response (voice)

[0443] Output: The audio presented to the user

[0444] Specific operation: The terminal plays the voice data received from the server and provides it to the user, who can then hear the dialogue response by voice.

[0445] In this way, a personalized dialogue according to the user's emotions is realized.

[0446] Example 2

[0447] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0448] Conventional AI dialogue systems have the problem that it is difficult to generate dialogue models based on the individual voice characteristics and personality traits of a user, and that they are unable to adjust dialogue responses in real time according to the user's emotional state. Furthermore, there is a lack of a mechanism for other users to easily purchase and use the generated AI models.

[0449] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0450] In this invention, the server includes a means for a user to set ideal voice characteristics and personality characteristics, a means for preprocessing data based on the set voice characteristics and personality characteristics, a means for training a generative AI model using the preprocessed data, a means for publishing the generated AI model on a platform that can be published, a means for other users to purchase the published AI model, a means for using an emotion engine that analyzes the user's emotional state in real time, and a means for adjusting the dialogue response of the AI ​​model based on the analysis result of the emotion engine. This makes it possible to generate a dialogue model based on the individual voice characteristics and personality characteristics of the user, and further makes it possible to adjust the dialogue response in real time according to the user's emotional state. In addition, a mechanism is provided that allows other users to easily purchase and use the generated AI model.

[0451] "User" means an individual or organization that utilizes the system to set their ideal voice and personality characteristics and then uses the generated AI model.

[0452] "Voice characteristics" refer to the characteristics and texture of a user's ideal voice, and are specifically set through a voice sample.

[0453] The "personality traits" refer to the personality and response style of the dialogue model that the user idealizes, and are specifically set as options or numerical data.

[0454] "Data preprocessing" is the process of analyzing user-provided voice samples and personality traits and converting them into a format suitable for machine learning models.

[0455] A "generative AI model" is an artificial intelligence model that is trained using preprocessed data to generate dialogue responses based on a user's ideal voice characteristics and personality traits.

[0456] The "Platform" refers to an online space where the generated AI models are made public and can be viewed and purchased by other users.

[0457] An "emotion engine" is software or hardware that analyzes a user's emotional state in real time and adjusts the dialogue response of an AI model based on the results.

[0458] "Dialogue response adjustment" is the process of changing the content and tone of the dialogue generated by the AI ​​model based on the analysis results of the emotion engine.

[0459] "Purchase means" refers to the procedures and systems through which users can obtain published AI models, including online payments and database management.

[0460] This invention is a system that allows users to set their ideal voice characteristics and personality traits, creates a generative AI model based on them, and uses it for dialogue.Furthermore, we provide a platform that allows other users to purchase and use the generated AI model.

[0461] The server provides an interface for users to set their ideal voice characteristics and personality traits. Users provide basic data for the ideal AI model by uploading voice samples and selecting personality traits. Voice analysis libraries such as voice processing libraries are used to analyze the voice samples, and appropriate data conversion methods are used to quantify the personality traits.

[0462] The server uses the preprocessed data to train a generative AI model. This process uses machine learning frameworks such as TensorFlow and PyTorch, and uses GPUs to quickly train the model. Once trained, the model will be able to generate dialogue responses that are close to the user's ideal.

[0463] The generated AI model is published on the platform by the server. The published model can be viewed and purchased by other users, and the purchase procedure is carried out through a database (e.g. MySQL (registered trademark)) managed by the server. User B can purchase the published AI model and use it in his / her own conversations.

[0464] In addition, the server uses an emotion engine to analyze the user's emotional state in real time. The emotion engine uses an NLP library (e.g., spaCy) or a voice analysis library to analyze emotions from the user's voice and text. Based on the analysis results of the emotion engine, the AI ​​model's dialogue response is adjusted. For example, if a user shows an emotion of joy at the beginning of a dialogue, but shows an emotion of sadness midway through, the AI ​​model generates a dialogue response that corresponds to that change.

[0465] As a concrete example, User A sets "a cheerful and energetic voice" and "a friendly personality" on the platform. The server generates an AI learning model based on this setting. The generated model is used for dialogue with User A and is made public on the platform. User B purchases this model and uses it for his own dialogue.

[0466] An example of a prompt is, "Please generate an AI model with a bright, cheerful voice and a friendly personality."

[0467] In this way, it is possible to generate a dialogue model based on the individual voice characteristics and personality traits of the user, and furthermore, to adjust the dialogue response in real time according to the user's emotional state. In addition, a mechanism is provided that allows other users to easily purchase and use the generated AI model.

[0468] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0469] Step 1:

[0470] A user sets his / her ideal voice characteristics and personality traits on the platform. Specifically, the user uploads a voice sample and selects personality traits. As input, the voice sample file and the selection data of personality traits are provided. As output, these data are sent to the server.

[0471] Step 2:

[0472] The server receives the voice samples uploaded by the user and performs spectral analysis using a voice processing library. The voice sample file is used as input. Specifically, the voice data is decomposed into frequency components and features are extracted. The analyzed voice feature data is obtained as output.

[0473] Step 3:

[0474] The server converts the personality traits selected by the user into numerical data. The personality trait selection data is used as input. The server then converts the selected personality traits into numerical data and formats them in a format suitable for the machine learning model. The server then outputs the numerical personality trait data.

[0475] Step 4:

[0476] The server trains a generative AI model using the preprocessed voice feature data and the quantified personality trait data. The voice feature data and personality trait data are used as input. In concrete terms, the machine learning model is trained using TensorFlow or PyTorch, and processed at high speed using a GPU. The trained AI model is obtained as output.

[0477] Step 5:

[0478] The server publishes the generated AI model on the platform. The trained AI model is used as input. The specific operation is to register the AI ​​model in the platform's database so that other users can view and purchase it. The output is the published AI model.

[0479] Step 6:

[0480] User B purchases a published AI model. The information on the published AI model is used as input. In concrete terms, User B performs a purchase procedure on the platform, and the purchase is completed through a database managed by the server. The output is the AI ​​model purchased by User B.

[0481] Step 7:

[0482] The server uses an emotion engine to analyze the user's emotional state in real time. The user's voice and text data are used as input. Specifically, emotions are analyzed using an NLP library (e.g. spaCy) or a voice analysis library. The analyzed emotional state data is obtained as output.

[0483] Step 8:

[0484] The server adjusts the dialogue response of the AI ​​model based on the analysis results of the emotion engine. The analyzed emotional state data is used as input. As a specific operation, the server adjusts the dialogue response generation algorithm of the AI ​​model to generate a response according to the user's emotional state. As output, an adjusted dialogue response is obtained.

[0485] (Application example 2)

[0486] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0487] Conventional AI dialogue systems have difficulty recognizing the user's emotional state in real time and responding accordingly. There were also only limited systems that allowed users to set their ideal vocal cords and personality and create an AI learning model based on that. Furthermore, there was a lack of platforms that allowed these AI learning models to be made available to other users and to be purchased or sold. This made it difficult to increase user satisfaction.

[0488] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0489] In this invention, the server includes a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with a user using the AI ​​learning model, a means for creating, purchasing, and selling the AI ​​learning model on a publicly available platform, a means for recognizing the user's emotional state in real time using an emotion engine and adjusting a dialogue response based on the emotional state, and a means for including an application installed on a smartphone or a head-mounted display. This enables a dialogue response according to the user's emotional state, thereby increasing the user's satisfaction.

[0490] An "AI learning model" is an artificial intelligence learning model created to mimic the user's ideal vocal cords and personality.

[0491] A "dialogue response" is a reply generated by an AI learning model in a dialogue with a user.

[0492] A "platform" is an online system that allows users to publish, create, buy, and sell AI learning models.

[0493] An "emotion engine" is an engine for recognizing a user's emotional state in real time and adjusting the dialogue response based on the recognition results.

[0494] A "smartphone" is a portable information terminal that has mobile phone functions and can connect to the Internet and install applications.

[0495] A "head-mounted display" is a display device worn on the head and is used to provide virtual reality and augmented reality experiences.

[0496] An "application" is a software program that is installed on a smartphone or head-mounted display and provides specific functions.

[0497] The system for implementing this invention uses the following hardware and software. The hardware requires a smartphone and a head-mounted display. The software uses the OpenAI API and the EmotionEngine library.

[0498] System Configuration

[0499] 1. Server:

[0500] Means for creating an AI learning model: The server creates an AI learning model based on the ideal vocal range and personality set by the user. Using the OpenAI API, it generates prompt sentences based on the user's settings and builds the AI ​​learning model.

[0501] How to generate dialogue responses: The server receives input from the user and recognizes their emotional state using the EmotionEngine library. It then generates appropriate dialogue responses based on the recognized emotional state using the OpenAI API.

[0502] Means of providing the platform: The server provides an online platform for publishing, creating, purchasing, and selling AI learning models. On this platform, users can publish their ideal vocal range and personality, and other users can purchase or sell them.

[0503] 2. Terminal:

[0504] Smartphones and head-mounted displays: Through applications installed on these devices, users can interact with the AI ​​learning models. The applications communicate with the server and receive dialogue responses in real time.

[0505] Processing flow

[0506] 1. User setting: The user sets his / her ideal vocal range and personality using a smartphone or a head-mounted display. This setting information is sent to the server.

[0507] 2. Creating an AI learning model: The server creates an AI learning model using the OpenAI API based on the received configuration information.

[0508] 3. Emotional state recognition: When the user initiates a dialogue, the device sends the user's input to the server, which uses the EmotionEngine library to recognize the user's emotional state in real time.

[0509] 4. Generate dialogue responses: Based on the recognized emotional state, the server generates appropriate dialogue responses using the OpenAI API and sends them to the device.

[0510] 5. Dialogue execution: The terminal displays or plays audibly the dialogue response received to the user.

[0511] Examples

[0512] For example, if a user says, "I'm so happy today!", the device sends this input to the server, which uses the EmotionEngine library to recognize the "joy" emotion and the OpenAI API to generate a bright, cheerful response. The response is then sent to the device, where it is displayed or played aloud to the user.

[0513] Examples of prompt statements

[0514] User input: I'm so happy today!, Emotional state: joy

[0515] In this way, a dialogue response according to the user's emotional state is possible, and user satisfaction can be increased.

[0516] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0517] Step 1:

[0518] Users can use a smartphone or a head-mounted display to set their ideal vocal range and personality.

[0519] Input: The user's ideal vocal and personality preferences.

[0520] How it works: The user enters vocal and personality parameters into the application's settings screen.

[0521] Output: The configuration information is sent to the server.

[0522] Step 2:

[0523] Based on the configuration information received by the server, an AI learning model is created using the OpenAI API.

[0524] Input: User configuration information.

[0525] How it works: The server converts the configuration information into a prompt text and sends it to the OpenAI API, which generates an AI learning model based on the prompt text.

[0526] Output: The created AI learning model.

[0527] Step 3:

[0528] When the user initiates an interaction, the terminal transmits the user's input to the server.

[0529] Input: User interactive input.

[0530] How it works: The user enters text or voice input into an application's dialogue box, and the device sends this input to the server.

[0531] Output: User interaction input data is sent to the server.

[0532] Step 4:

[0533] The server uses the EmotionEngine library to recognize the user's emotional state in real time.

[0534] Input: User interaction input data.

[0535] How it works: The server passes dialogue input data to the EmotionEngine library, which analyzes the emotional state.

[0536] Output: Perceived emotional state.

[0537] Step 5:

[0538] The server generates appropriate dialogue responses based on the recognized emotional state using the OpenAI API.

[0539] Input: The user's dialogue input data and perceived emotional state.

[0540] How it works: The server generates a prompt and sends it to the OpenAI API, which generates a dialogue response based on the prompt.

[0541] Output: The generated dialogue response.

[0542] Step 6:

[0543] The server transmits the generated dialogue response to the terminal.

[0544] Input: The generated dialogue response.

[0545] Operation: The server sends the generated dialogue response to the terminal.

[0546] Output: Dialogue response data is sent to the terminal.

[0547] Step 7:

[0548] The terminal displays or plays audibly the dialogue response received by the terminal to the user.

[0549] Input: Dialogue response data.

[0550] Operation: The terminal displays the dialogue response data as text or plays it aloud.

[0551] Output: The interactive response is provided to the user.

[0552] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0553] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0554] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0555] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0556] [Second embodiment]

[0557] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0558] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0559] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0560] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0561] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0562] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0563] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0564] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0565] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0566] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0567] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0568] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0569] "Example 1"

[0570] The present invention is a system including a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with a user using the AI ​​learning model, and a means for creating, purchasing, and selling the AI ​​learning model on a platform that can be made public. Specifically, a user sets his or her ideal vocal cords and personality on the platform, and an AI learning model is created based on the settings. The created AI learning model is used for interacting with the user and is made public to other users on the platform. The published AI learning model can be purchased and sold by other users.

[0571] "Example 2"

[0572] As a specific example, User A sets his / her ideal vocal range and personality on the platform, and an AI learning model is created based on that setting. The created AI learning model is used for dialogue with User A and is also made available to other users on the platform. The published AI learning model can be purchased by another User B and used for dialogue with User B.

[0573] The process flow of each embodiment will be described below.

[0574] "Example 1"

[0575] Step 1: The user sets their ideal vocal range and personality on the platform. This setting includes vocal characteristics such as pitch, speed, and intonation, and personality characteristics such as the range and intensity of emotional expression such as joy, anger, sadness, and happiness.

[0576] Step 2: An AI learning model is created based on the settings. This AI learning model has the ability to mimic the user's ideal vocal chords and personality.

[0577] Step 3: The created AI learning model is used to interact with the user, who can then interact with the AI ​​that has the vocal range and personality of their dreams.

[0578] Step 4: The created AI learning model is made publicly available to other users on the platform. The published AI learning model can be purchased or sold by other users.

[0579] "Example 2"

[0580] Step 1: User A sets his / her ideal vocal cords and personality on the platform. This setting includes vocal cord characteristics such as voice pitch, speed, intonation, and personality characteristics such as range and intensity of emotional expression such as joy, anger, sadness, and happiness.

[0581] Step 2: An AI learning model is created based on the settings. This AI learning model has the ability to mimic User A's ideal vocal cords and personality.

[0582] Step 3: The created AI learning model is used to interact with User A. User A can interact with an AI that has the vocal range and personality of their ideal user.

[0583] Step 4: The created AI learning model is made available to other users on the platform. The published AI learning model is purchased by other user B.

[0584] Step 5: The purchased AI learning model is used to interact with User B. User B can interact with an AI that has User A's ideal vocal range and personality.

[0585] Example 1

[0586] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0587] Conventional AI learning models have difficulty reflecting the user's ideal vocal cords and personality, and it is also difficult to share or sell the generated models with other users. For this reason, it has not been possible to create AI learning models that meet the diverse needs of users and to effectively use those models. Furthermore, there has been a lack of a platform for users to share and sell the models they have generated with other users.

[0588] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0589] In this invention, the server includes a means for a user to access the platform and set an ideal vocal cord and personality, a means for the server to receive the user's settings and generate an AI learning model using voice synthesis software and a personality model generation algorithm, and a means for the server to store the generated AI learning model on the platform and enable the user to interact with it. This allows a user to easily create an AI learning model that reflects their ideal vocal cord and personality, and to share or sell the model with other users.

[0590] "User" refers to an individual or organization that uses the system to set their ideal vocal range and personality, and to create, interact with, publish, purchase, and sell AI learning models.

[0591] The "Platform" refers to an online system that allows Users to access and set their ideal vocal range and personality, and to create, publish, purchase, and sell AI learning models.

[0592] "Server" means a computer system that receives user settings, generates AI learning models using voice synthesis software and personality model generation algorithms, and stores, publishes, and sells them on the Platform.

[0593] "Voice synthesis software" refers to software for generating voice data based on the characteristics of the vocal cords set by the user.

[0594] The "personality model generation algorithm" is an algorithm for generating personality data based on personality characteristics set by the user.

[0595] An "AI learning model" is an artificial intelligence model that mimics a user's vocal cords and personality, generated based on the user's settings.

[0596] "Dialogue" refers to the act of a user communicating with an AI learning model through text or voice.

[0597] "Publication" refers to displaying the AI ​​learning model created by a user on the platform so that other users can view and purchase it.

[0598] "Purchase" refers to the act of other users paying a fee to obtain a published AI learning model.

[0599] "Sales" refers to the act of a user providing an AI learning model they have created to other users and receiving payment in return.

[0600] "Right to Use" means the right of the Purchaser to use the AI ​​learning model acquired.

[0601] This invention is a system in which a user sets their ideal vocal cords and personality, generates an AI learning model based on that, and then interacts with, publishes, purchases, and sells the model. A specific embodiment of this system will be described below.

[0602] First, the user accesses the platform using a terminal (PC, smartphone, tablet, etc.). The platform provides an interface for setting the vocal range and personality. The user selects the pitch of the voice, tone, speaking speed, and personality traits (e.g. friendly, calm, humorous, etc.).

[0603] The server then receives the user-entered configuration data, and uses voice synthesis software (e.g., Google Text-to-Speech API) and personality model generation algorithms (e.g., OpenAI's GPT-3) to generate voice data and personality data based on the user's configuration. These data are then combined to create an AI learning model.

[0604] The server stores the generated AI learning model on the platform. The user can then interact with the AI ​​learning model through the device. When interacting, the text or voice entered by the user is sent to the server, which uses the AI ​​learning model to generate an appropriate response and returns it to the device.

[0605] For example, if a user types "What's the weather today?" in a chat window, the server will use the AI ​​learning model to generate a response such as "It's sunny today" and display it in the chat window.

[0606] Furthermore, if the user wishes, the server will publish the generated AI learning model on the platform. The published model will be available for other users to view. Other users can select a model they are interested in from the published list and click the "Purchase" button. Once the purchase procedure is complete, the server will grant the purchaser the right to use the AI ​​learning model. The purchaser will be able to use this model through their device.

[0607] As a specific example, User A wants to create an AI learning model with a "calm voice and humorous personality." User A sets the following through the interface: "Voice pitch: medium-low," "Tone: calm," and "Personality: humorous." The server receives these settings and uses voice synthesis software and a personality model generation algorithm to generate voice data and personality data based on the settings, creating an AI learning model. The created model can then be used by User A for conversations. User A can also publish this model on the platform, making it available for other users to purchase.

[0608] An example of a prompt is, "Please create an AI learning model with a calm voice and a humorous personality. Please use a medium-low pitch and a calm tone."

[0609] In this way, the system allows AI learning models to be created, interacted with, published, and sold, with servers, terminals, and users each playing their respective roles.

[0610] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0611] Step 1:

[0612] Users access the platform and set their ideal vocal range and personality.

[0613] Specifically, the user uses a terminal to access the platform's URL and log in. Then, they go to the settings screen and input their voice pitch, tone, speaking speed, and personality traits.

[0614] Input: User's vocal and personality settings (e.g. voice pitch, mid-low, tone, calm, personality, humorous)

[0615] Output: The configuration data is sent to the server.

[0616] Step 2:

[0617] The server receives the user's settings and generates an AI learning model.

[0618] Specifically, the server analyzes the received configuration data, calls voice synthesis software (e.g., Google Text-to-Speech API) to generate voice data, and simultaneously generates personality data using a personality model generation algorithm (e.g., OpenAI's GPT-3). These data are then integrated to create an AI learning model.

[0619] Input: User's configuration data

[0620] Output: The generated AI learning model

[0621] Step 3:

[0622] The server stores the generated AI learning model on the platform and allows users to interact with it.

[0623] Specifically, the server stores the generated AI learning model in a database. When the user starts a dialogue, the server calls the model and generates a response according to the user's input.

[0624] Input: Generated AI learning model

[0625] Output: AI learning model stored on the platform

[0626] Step 4:

[0627] The user interacts with the AI ​​learning model.

[0628] Specifically, a user inputs text into a chat window through a terminal. The server receives the input text, uses the AI ​​learning model to generate an appropriate response, and displays it in the chat window.

[0629] Input: User text input (e.g. "What's the weather like today?")

[0630] Output: Response from the AI ​​learning model (e.g. "It's sunny today")

[0631] Step 5:

[0632] The server publishes the user's AI learning model on the platform.

[0633] Specifically, when a user clicks the "Publish" button, the server adds the model to a published list, from which other users can view the model.

[0634] Input: User's publishing instructions

[0635] Output: AI learning model added to the public list

[0636] Step 6:

[0637] Other users can purchase published AI learning models.

[0638] Specifically, other users select a model they are interested in from the public list and click the "Purchase" button. Once the purchase procedure is completed, the server grants the purchaser the right to use the model.

[0639] Input: Purchase instructions from other users

[0640] Output: The model usage rights granted to the buyer.

[0641] Step 7:

[0642] The server grants the buyer the rights to use the model.

[0643] Specifically, the server adds the model usage rights to the buyer's account, allowing the buyer to use the model through their device.

[0644] Input: Purchase procedure completion notification

[0645] Output: Model usage rights added to buyer's account

[0646] (Application example 1)

[0647] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0648] The shopping experience in conventional virtual stores had the problem that users needed a lot of time and effort to obtain product information. In addition, it was difficult for users to find products that matched their preferences and needs, and the shopping experience was often unsatisfactory. Furthermore, the conversational assistant was not customized based on the user's vocal cords or personality, so the conversation with the user was sometimes unnatural. To solve these problems, it is necessary to provide a conversational assistant using an AI learning model that mimics the user's ideal vocal cords and personality.

[0649] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0650] In this invention, the server includes a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with a user using the AI ​​learning model, a means for creating, purchasing, and selling the AI ​​learning model on a platform that can be made public, an interactive assistant means for supporting the user's shopping in a virtual store, a means for installing the interactive assistant means in a smartphone or a head-mounted display, and a means for the interactive assistant means to provide product information based on the user's ideal vocal cords and personality. This allows the user to improve their shopping experience in the virtual store through an interactive assistant based on their own ideal vocal cords and personality.

[0651] An "AI learning model for mimicking human vocal cords and personality" is an artificial intelligence learning model designed to reproduce the characteristics of a human voice, speaking style, and personality.

[0652] "Means of interacting with a user" refers to methods and technologies for achieving communication with a user.

[0653] A "publishable platform" is an online system that enables users to share, buy, or sell user-created content or data with other users.

[0654] A "virtual store" is a virtual store operated on the Internet where users can browse and purchase products online.

[0655] An "interactive assistant means" is a technique or method for providing information or support to a user through dialogue.

[0656] A "smartphone" is a highly functional mobile phone that can connect to the Internet and use applications.

[0657] A "head-mounted display" is a display device that provides visual information to a user when worn on the head.

[0658] "Means for providing product information" refers to methods or techniques for conveying detailed product information and features to users.

[0659] In order to implement the present invention, the following system configuration and program are required.

[0660] First, the server creates an AI learning model to mimic human vocal cords and personality. This AI learning model is generated based on the ideal vocal cords and personality set by the user. Specifically, the user sets their ideal vocal cords and personality on the platform and sends the setting information to the server. The server generates an AI learning model based on this information. The generated AI learning model is used to interact with the user.

[0661] The server then provides a means for interacting with the user using the generated AI learning model. The user can do this through a smartphone or a head-mounted display. The interactive assistant generates appropriate responses based on the user's input and plays them aloud using a speech synthesis engine.

[0662] In addition, the server provides a means to create, buy and sell AI learning models on a public platform where users can publish their ideal vocal range and personality, and other users can buy and sell the AI ​​learning models.

[0663] In the virtual store, the interactive assistant means assists the user in shopping. Specifically, when the user asks a question about a product, the interactive assistant means uses an AI learning model to provide appropriate product information. For example, if the user asks, "I'm looking for new running shoes. Do you have any recommendations?", the interactive assistant means responds, "How about these running shoes? They're lightweight and highly cushioned, making them ideal for long runs."

[0664] The following hardware and software are used to realize this system:

[0665] Hardware: Servers, smartphones, head-mounted displays

[0666] Software: OpenAI API, speech synthesis engine

[0667] Examples of specific prompts are as follows:

[0668] User: "I'm looking for new running shoes. Any recommendations?"

[0669] Assistant: "How about these running shoes? They're lightweight and well-cushioned, perfect for long runs."

[0670] In this way, users can enhance their shopping experience in virtual stores through a conversational assistant based on their ideal vocal range and personality.

[0671] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0672] Step 1:

[0673] Users access the platform and set their ideal vocal range and personality.

[0674] Input: User's vocal and personality settings

[0675] Output: Configuration information sent to the server

[0676] How it works: Through the platform's interface, the user inputs the tone of voice, speaking style, personality traits, etc. This configuration information is sent to the server.

[0677] Step 2:

[0678] The server generates an AI learning model based on the configuration information received.

[0679] Input: User's vocal and personality settings

[0680] Output: AI learning model

[0681] Specific operation: The server analyzes the received configuration information and uses it as data to generate an AI learning model. It uses the OpenAI API to generate an AI learning model based on the configuration information.

[0682] Step 3:

[0683] The generated AI learning model is used to prepare for interaction with the user.

[0684] Input: AI learning model

[0685] Output: Interaction ready state

[0686] Specific operation: The server integrates the generated AI learning model into the dialogue system, making it possible to dialogue with the user.

[0687] Step 4:

[0688] Users access the conversational assistant through a smartphone or head-mounted display.

[0689] Input: User's access request

[0690] Output: Launch of interactive assistant

[0691] Specific operation: A user starts an application on a smartphone or head-mounted display to access the conversational assistant. The application connects to the server and starts the conversational assistant.

[0692] Step 5:

[0693] The user inputs a question to the interactive assistant.

[0694] Input: User question

[0695] Output: The question data is sent to the server.

[0696] Specific operation: The user inputs a question to the conversational assistant by text or voice, and the input question is sent to the server.

[0697] Step 6:

[0698] The server generates a response based on the user's question.

[0699] Input: User question data

[0700] Output: Response data

[0701] Specific operation: The server analyzes the received question data and generates appropriate responses using an AI learning model. It uses the OpenAI API to generate natural responses to questions.

[0702] Step 7:

[0703] The generated response is converted into speech by a speech synthesis engine.

[0704] Input: Response data

[0705] Output: Audio data

[0706] Specific operation: The server passes the generated response data to the speech synthesis engine to generate voice data.

[0707] Step 8:

[0708] The audio data is sent to the user's terminal and played back.

[0709] Input: Audio data

[0710] Output: Audio response to the user

[0711] Specific operation: The server transmits the generated voice data to the user's smartphone or head-mounted display. The user's device plays the received voice data and provides the user with a response.

[0712] In this way, users can enhance their shopping experience in virtual stores through a conversational assistant based on their ideal vocal range and personality.

[0713] Example 2

[0714] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0715] Conventional AI learning models make it difficult to set vocal cords and personality according to individual user needs, and the generated models are not easy to publish or for other users to purchase. For this reason, there was a demand for a system that would allow users to easily create AI models that reflect their ideal vocal cords and personality, and share them with other users.

[0716] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for a user to set his / her ideal vocal cords and personality, a means for receiving the setting information, collecting voice data and text data, and performing preprocessing, a means for generating an AI learning model using the preprocessed data, a means for publishing the generated AI learning model on a platform where it can be published, and a means for other users to purchase the published AI learning model and use it in conversations. This makes it possible for a user to easily create an AI model that reflects his / her ideal vocal cords and personality, and share it with other users.

[0717] "User" refers to an individual or organization that uses the system to set their ideal vocal range and personality and generate and use an AI learning model.

[0718] "Vocal chords" refers to the voice characteristics such as tone, pitch, and speed of the voice that are set by the user.

[0719] "Personality" refers to the personality traits of the AI ​​learning model set by the user (e.g. kind, strict, humorous, etc.).

[0720] "Settings Information" refers to information regarding vocal chords and personality that a User inputs on the Platform.

[0721] "Voice data" refers to voice data collected based on a user's vocal cord settings.

[0722] "Text data" refers to text data collected based on a user's personality settings.

[0723] "Preprocessing" refers to the process of converting collected voice and text data into a format suitable for training an AI learning model.

[0724] "AI learning model" refers to an artificial intelligence model that is generated based on user settings information.

[0725] "Publication" refers to displaying the generated AI learning model on the platform so that other users can access it.

[0726] "Platform" refers to the online system through which Users can set their vocal range and personality, and generate, publish, purchase and use AI learning models.

[0727] "Purchase" refers to the transactional act of other users acquiring published AI learning models.

[0728] "Dialogue" refers to communication between a user and an AI learning model.

[0729] This invention is a system that allows users to set their ideal vocal cords and personality, and generate, publish, purchase, and use AI learning models based on that. A specific embodiment of this system is described below.

[0730] User configuration input

[0731] Users use the platform's web interface to configure their ideal vocal range and personality. Specifically, users select the tone, pitch, speed, and personality traits (e.g., kind, stern, humorous, etc.) of their voice. These configuration information is sent to the server as form input.

[0732] Data collection and preprocessing

[0733] The server receives the user's configuration information and collects the necessary data. It collects voice and text data and performs preprocessing. Preprocessing includes normalizing the voice data and tokenizing the text data. It uses Python and TensorFlow.

[0734] Generate AI learning models

[0735] The server uses the preprocessed data to generate an AI learning model, uses deep learning techniques to build a neural network using TensorFlow or PyTorch, and trains the model based on the user's settings.

[0736] Publishing the model

[0737] The server publishes the generated AI learning model on the platform. The model is uploaded to cloud storage (e.g. AWS S3) and a public URL is generated. This URL is displayed on the platform so that other users can access it.

[0738] Purchase and Use by Other Users

[0739] User B purchases the published AI learning model on the platform. The purchase procedure is done through a web interface, and payment is processed using a payment service. After the purchase, User B can use the AI ​​learning model in his or her own interactions.

[0740] Examples

[0741] Suppose User A sets "gentle voice, humorous personality" on the platform. The server receives this setting information, collects voice and text data, and performs preprocessing. Then, it uses TensorFlow to generate an AI learning model and uploads it to cloud storage. The published model is purchased by User B and used to interact with User B.

[0742] Examples of prompt statements

[0743] "Generate an AI model with a gentle voice and a humorous personality."

[0744] This system provides a series of processes for users to set their ideal vocal range and personality, and then generate, publish, purchase, and use AI learning models based on that, allowing users to easily create AI models that meet their needs and share them with other users.

[0745] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0746] Step 1: Enter user settings

[0747] A user configures his / her ideal vocal range and personality using a web interface on the platform. Specifically, the user selects the tone, pitch, speed, and personality traits (e.g., kind, stern, humorous, etc.) of the voice. These configuration information are sent to the server as form input. The input is the user's configuration information, and the output is the configuration information sent to the server.

[0748] Specific behavior:

[0749] A user visits a web page and fills out a configuration form.

[0750] The user clicks the "Submit" button.

[0751] The server receives the configuration information and stores it in a database.

[0752] Step 2: Collect and preprocess data

[0753] The server receives the user's configuration information and collects the necessary data. It collects voice and text data and performs preprocessing. Preprocessing includes normalizing the voice data and tokenizing the text data. It uses Python and TensorFlow. The input is the user's configuration information and the output is the preprocessed data.

[0754] Specific behavior:

[0755] The server collects the voice data based on the setting information.

[0756] The server normalizes the audio data and tokenizes the text data.

[0757] The server temporarily stores the pre-processed data.

[0758] Step 3: Generate an AI learning model

[0759] The server uses the preprocessed data to generate an AI learning model. It uses deep learning techniques to build a neural network using TensorFlow or PyTorch. It trains the model based on the user's settings. The input is the preprocessed data, and the output is the trained AI learning model.

[0760] Specific behavior:

[0761] The server uses TensorFlow to build a neural network.

[0762] The server trains the model using the preprocessed data.

[0763] The server stores the trained model.

[0764] Step 4: Publish the model

[0765] The server publishes the generated AI learning model on the platform. The model is uploaded to cloud storage (e.g. AWS S3) and a public URL is generated. This URL is displayed on the platform so that other users can access it. The input is the trained AI learning model, and the output is the public URL.

[0766] Specific behavior:

[0767] The server uploads the trained model to cloud storage.

[0768] The server generates a public URL.

[0769] The server displays the URL on the platform.

[0770] Step 5: Purchase and use by others

[0771] User B purchases the published AI learning model on the platform. The purchase procedure is done through a web interface, and payment is processed using a payment service. After the purchase, User B can use the AI ​​learning model in his / her own interactions. The input is the publication URL and User B's purchase information, and the output is an AI learning model that User B can use.

[0772] Specific behavior:

[0773] User B selects a model published on the platform.

[0774] User B completes the purchase procedure.

[0775] The server processes the payment through a payment service.

[0776] User B downloads the purchased model and uses it in the interaction.

[0777] (Application example 2)

[0778] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0779] With conventional voice synthesis technology, it was difficult for users to generate personalized voice content that reflected their ideal vocal range and personality. In addition, there were limited ways to purchase personalized voice models created by other users and enjoy content in their voices. This meant that it was not possible to meet the diverse needs of users, and there was an issue of the individualization of voice content not progressing.

[0780] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0781] In this invention, the server includes a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with a user using the AI ​​learning model, a means for creating, purchasing, and selling the AI ​​learning model on a public platform, a means for a user to set his / her ideal vocal cords and personality and generate an AI learning model based on the setting, a means for providing audio content in a personalized voice using the generated AI learning model, and a means for purchasing an AI learning model created by another user and enjoying content in that voice. This allows a user to generate personalized voice content that reflects his / her ideal vocal cords and personality, and to enjoy a variety of voice content using voice models created by other users.

[0782] The "human vocal cords" are the biological structures used by humans to produce sound and are what determine the pitch and quality of the voice.

[0783] "Personality" refers to an individual's characteristics in behavior, thinking, and emotion, and reflects that person's characteristics in dialogue and communication.

[0784] An "AI learning model" is an algorithm or network structure that uses artificial intelligence technology to learn specific data and patterns and perform tasks such as prediction, classification, and generation.

[0785] A "production means" is a method or device for producing or building something for a particular purpose.

[0786] A "means for interacting" is a method or device by which a user and a system exchange information and communicate with each other.

[0787] A "publishable platform" is an online system that enables users to share and make accessible their created content and models to other users.

[0788] A "buying or selling vehicle" is a transactional method or device by which a user obtains or provides particular content or models.

[0789] A "setting means" is a method or device that allows a user to input his / her preferences and requirements into the system so that the system operates based on them.

[0790] "Audio content" means any form of information or entertainment that is primarily audio, including music, podcasts and audiobooks.

[0791] A "personalized voice" is a voice that is customized based on the preferences and requirements of a particular user, and reflects that user's ideal vocal range and personality.

[0792] An "AI learning model created by another user" is an artificial intelligence model that a different user has generated based on their own settings and that can be used by other users.

[0793] The system for implementing this invention is configured as follows: First, the server has a means for creating an AI learning model for imitating human vocal cords and personality. This means provides an interface for the user to set their ideal vocal cords and personality, and generates an AI learning model based on the settings.

[0794] Next, the server has a means for interacting with the user using the generated AI learning model, which analyzes the text and voice data entered by the user and generates an appropriate response using the AI ​​learning model.

[0795] Additionally, the server has the means to create, buy and sell AI learning models on a publicly accessible platform that allows users to publish their ideal vocal range and personality and allows other users to buy or sell the models.

[0796] The server also has a means for providing audio content in a personalized voice using the generated AI learning model, the means for playing content selected by the user in a voice generated using the AI ​​learning model.

[0797] Finally, the server has a means for purchasing an AI learning model created by another user and enjoying content with that voice. This means enables a user to purchase an AI learning model created by another user and play content using that model.

[0798] Hardware and Software Used

[0799] Hardware: Smartphone (iOS or Android)

[0800] Software: Python, TensorFlow, Keras

[0801] Data processing and calculation

[0802] 1. Data Acquisition:

[0803] It provides an interface for users to set their ideal vocal range and personality, inputting settings such as "calm male voice" or "intelligent and friendly."

[0804] 2. Model construction:

[0805] Using TensorFlow and Keras, we build a Long Short-Term Memory (LSTM) network that learns from speech data and generates personalized speech based on user preferences.

[0806] 3. Model training:

[0807] The system generates training data based on the user's settings and trains the model, which then produces a voice that reflects the user's ideal vocal range and personality.

[0808] 4. Save and publish the model:

[0809] Save the trained model and make it accessible to others on a publicly accessible platform.

[0810] Examples

[0811] User A opens the app and sets his ideal vocal range as "calm male voice" and his personality as "intelligent and friendly." The app generates an AI learning model based on this, and User A can listen to audiobooks in that voice. User B can also purchase User A's model and enjoy podcasts in the same voice.

[0812] Examples of prompt statements

[0813] Set the user's ideal vocal range and personality, for example, "calm male voice" or "intelligent and friendly."

[0814] In this way, a system is provided that allows the user to enjoy content in his or her ideal voice.

[0815] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0816] Step 1:

[0817] A user starts the application using a terminal and sets the ideal vocal range and personality. The user inputs settings such as "calm male voice" or "intelligent and friendly." The input data is the setting information of the user's vocal range and personality. The terminal sends this setting information to the server.

[0818] Step 2:

[0819] The server builds an AI learning model based on the received user configuration information. Specifically, it builds an LSTM (Long Short-Term Memory) network using TensorFlow and Keras. The input data is the user configuration information, and the output data is the initialized AI learning model.

[0820] Step 3:

[0821] The server generates training data based on the user's settings. Specifically, it collects voice data and text data that match the user's ideal vocal cords and personality, and prepares this as a training data set. The input data is the user's settings, and the output data is the training data set.

[0822] Step 4:

[0823] The server trains the AI ​​learning model using the training dataset. Specifically, it inputs the training data into the LSTM network and optimizes the model parameters. The input data is the training dataset, and the output data is the trained AI learning model.

[0824] Step 5:

[0825] The server stores the trained AI learning model and uploads it to a publicly available platform, so that other users can purchase or use the model. The input data is the trained AI learning model, and the output data is the published AI learning model.

[0826] Step 6:

[0827] A user uses a terminal to select a published AI learning model and play audio content. Specifically, the user downloads the selected model and uses the model to play audiobooks or podcasts. The input data is the published AI learning model and audio content, and the output data is the audio content played in a personalized voice.

[0828] Step 7:

[0829] Other users use their devices to purchase the published AI learning model and enjoy the content with that voice. Specifically, other users purchase the published model and play the content using that model. The input data is the published AI learning model and a purchase request, and the output data is the purchased AI learning model and the content played with a personalized voice.

[0830] In this way, users can enjoy content in their ideal voice, and other users can also use the model to enjoy a variety of voice content.

[0831] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0832] "Example 1"

[0833] As one embodiment of the present invention, there is a system of an AI learning model combined with an emotion engine. This system recognizes emotions from the user's voice and adjusts the dialogue response of the AI ​​learning model based on the emotion. Specifically, when a user utters a voice indicating an emotion of joy, the emotion engine recognizes the emotion of joy, and the AI ​​learning model generates a dialogue response corresponding to the emotion of joy based on the recognition result. Also, when a user utters a voice indicating an emotion of sadness, the emotion engine recognizes the emotion of sadness, and the AI ​​learning model generates a dialogue response corresponding to the sadness based on the recognition result. In this way, by combining the emotion engine and the AI ​​learning model, dialogue according to the user's emotional state is possible.

[0834] "Example 2"

[0835] Another embodiment of the present invention is a system in which an emotion engine updates the user's emotional state in real time and adjusts the dialogue response of the AI ​​learning model based on the update. Specifically, if the user changes his / her emotional state during the dialogue, the emotion engine recognizes the change in real time, and the AI ​​learning model adjusts the dialogue response based on the recognition result. For example, if the user shows an emotion of joy at the beginning of the dialogue but shows an emotion of sadness in the middle, the emotion engine recognizes the change in emotion in real time, and the AI ​​learning model adjusts the dialogue response from joy to sadness based on the recognition result. In this way, by combining the emotion engine and the AI ​​learning model, a dialogue according to the user's emotional state in real time is possible.

[0836] The process flow of each embodiment will be described below.

[0837] "Example 1"

[0838] Step 1: A user produces a speech sound that reflects the user's emotional state.

[0839] Step 2: The emotion engine recognizes emotions from the user's voice by analyzing voice tone, pitch, stress, and other elements.

[0840] Step 3: The AI ​​learning model receives the recognition results from the emotion engine and generates a dialogue response corresponding to the emotion, which is adapted to the user's emotional state.

[0841] "Example 2"

[0842] Step 1: A user produces speech during a dialogue that reflects changes in the user's emotional state.

[0843] Step 2: The emotion engine recognizes emotional changes in the user's voice in real time by analyzing changes in tone, pitch, volume, and other elements of the voice.

[0844] Step 3: The AI ​​learning model receives the recognition results from the emotion engine and generates a dialogue response that corresponds to the emotion changes, which is adapted to the changes in the user's emotional state.

[0845] Example 1

[0846] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0847] In conventional conversational AI systems, it was difficult to create a dialogue that reflected the individual voice characteristics and personality traits of the user, and responses were not adjusted sufficiently based on emotion recognition. This made it difficult to provide a natural and friendly conversational experience for users. In addition, the process of publishing, purchasing, and selling the generated AI models was complicated, making it difficult to share models between users.

[0848] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0849] In this invention, the server includes a means for creating a machine learning model for imitating human voice characteristics and personality characteristics, a means for interacting with a user using the machine learning model, a means for creating, purchasing, and selling the machine learning model on an online platform that can be made public, and an emotion recognition means for recognizing emotions from the user's voice and adjusting the dialogue response based on the emotions. This makes it possible to provide a natural and friendly dialogue experience that reflects the individual voice characteristics and personality characteristics of the user. In addition, the process of publishing, purchasing, and selling the generated AI model is simplified, making it easy to share the model among users.

[0850] A "machine learning model" is an algorithm that learns from data to mimic voice characteristics or personality traits and perform specific tasks.

[0851] "Voice characteristics" refers to the physical characteristics of speech, such as pitch, tempo, and volume.

[0852] "Personality traits" refer to characteristics that describe an individual's character, such as friendliness or calmness.

[0853] An "emotion recognition means" is a technology for analyzing emotions from a user's voice and recognizing those emotions.

[0854] "Dialogue response" refers to a reply generated in response to a user's input.

[0855] An "online platform" is a system accessible via the Internet that provides a place for users to create, publish, purchase and sell AI models.

[0856] "User" refers to any individual or entity that uses the System.

[0857] "Public" refers to making the generated AI model available for other users to view.

[0858] "Purchase" refers to the act of other users acquiring a published AI model.

[0859] "Sales" refers to the act of providing the generated AI model to other users.

[0860] This invention is a system for creating a machine learning model for imitating human voice characteristics and personality characteristics and for interacting with a user. Furthermore, this system has a function for publishing, purchasing, and selling the generated machine learning model on an online platform. It also includes an emotion recognition means for recognizing emotions from the user's voice and adjusting the dialogue response based on the emotions.

[0861] Generating a Program

[0862] The server generates a machine learning model based on the ideal voice and personality characteristics set by the user on the online platform. In this case, the server collects voice data and personality data, and uses these data to train the machine learning model. Specifically, the following hardware and software are used:

[0863] Hardware: High performance GPU

[0864] Software: Speech processing libraries, natural language processing libraries (e.g. Transformers)

[0865] Data processing and calculation

[0866] The server performs the following steps to process the voice and personality data:

[0867] 1. The server analyzes the voice data uploaded by the user using a voice processing library and extracts voice features.

[0868] 2. The server analyzes the personality data entered by the user using Transformers and extracts personality features.

[0869] 3. The server combines these features to train a machine learning model and generate a model that mimics the user's ideal voice and personality characteristics.

[0870] User interaction

[0871] The terminal uses the generated machine learning model to converse with the user. Specifically, when the user speaks to the terminal, the terminal transmits the voice to the server. The server recognizes emotions from the voice using an emotion recognition means, and the machine learning model generates an appropriate dialogue response based on the recognized emotion and transmits it back to the terminal. The terminal returns the generated dialogue response to the user by voice.

[0872] Publish, buy and sell on the platform

[0873] The server publishes the generated machine learning model on an online platform. The published model is added to a list so that other users can view it. Users can view the published machine learning model and purchase or sell it.

[0874] Examples of concrete examples and prompts

[0875] Examples

[0876] User A sets up a machine learning model with a "calm voice and friendly personality" on an online platform. The server collects User A's voice data and personality data, and extracts features using a voice processing library and Transformers. It then trains the machine learning model using a high-performance GPU to generate a model that mimics User A's ideal voice and personality characteristics. The generated model is used for conversations with User A and is also made public on the online platform. Another User B can like this model and purchase it.

[0877] Examples of prompt statements

[0878] "Create a machine learning model with a calm voice and a friendly personality. If the user makes a speech that indicates the emotion of happiness, generate a dialogue response that corresponds to that emotion."

[0879] In this way, the server, terminal, and user each fulfill their respective roles, allowing the entire system to operate smoothly.

[0880] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0881] Step 1:

[0882] Users access the platform and set their ideal voice and personality characteristics.

[0883] Specifically, users access the platform using a web browser or a dedicated app, select a "calm voice" and a "friendly personality" on the settings screen, and click the "Complete settings" button.

[0884] Input: User's voice and personality settings

[0885] Output: Configuration information sent to the server

[0886] Step 2:

[0887] The server collects voice and personality data and generates a machine learning model.

[0888] Specifically, the server performs the following actions:

[0889] 1. The server analyzes the audio data uploaded by the user using a voice processing library and extracts audio features (e.g. pitch, tempo, volume, etc.).

[0890] 2. The server analyzes the personality data entered by the user using Transformers and extracts personality features (e.g., friendliness, calmness, etc.).

[0891] 3. The server combines these features and uses a high-performance GPU to train a machine learning model to generate a model that mimics the user's ideal voice and personality characteristics.

[0892] Input: User's voice and personality data

[0893] Output: The generated machine learning model

[0894] Step 3:

[0895] The server publishes the generated machine learning model on the platform.

[0896] Specifically, the server publishes the generated model on the platform and displays detailed information about the model (e.g. vocal characteristics, personality traits, price, etc.) so that other users can view this information.

[0897] Input: A generated machine learning model

[0898] Output: Detailed information about the published machine learning model

[0899] Step 4:

[0900] The device uses machine learning models to interact with the user.

[0901] Specifically, when a user speaks to the device, the device transmits the voice to the server. The server uses emotion recognition means to recognize emotions (e.g., interest, curiosity, etc.) from the voice, and the machine learning model generates an appropriate dialogue response and sends it back to the device. The device then returns this response to the user by voice.

[0902] Input: User's voice input

[0903] Output: Dialogue response

[0904] Step 5:

[0905] Other users can buy or sell published machine learning models.

[0906] Specifically, other users can browse the machine learning models published on the platform and purchase the models they like. Once the purchase is complete, the server grants the buyer the right to use the model. Users can also sell the machine learning models they have created on the platform.

[0907] Input: Details of published machine learning models

[0908] Output: The rights to use the machine learning models purchased or sold

[0909] (Application example 1)

[0910] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0911] Conventional conversational AI systems have had difficulty recognizing a user's emotions and generating appropriate conversational responses. In addition, there was a lack of a way for a user to easily create an AI model that imitates their ideal vocal cords and personality and share it with other users. This limited the user experience and made it difficult to realize personalized conversations.

[0912] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0913] In this invention, the server includes a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with a user using the AI ​​learning model, a means for creating, purchasing, and selling the AI ​​learning model on a platform that can be made public, an emotion recognition means for recognizing the user's emotion and adjusting the dialogue response based on the emotion, and a voice synthesis means for converting the dialogue response generated based on the emotion recognition means into voice. This enables personalized dialogue according to the user's emotion, and allows an AI model that imitates an ideal vocal cord and personality to be easily created and shared with other users.

[0914] An "AI learning model" is an artificial intelligence learning algorithm designed to mimic human vocal cords and personality.

[0915] An "interactive means" is a method or device for communicating with a user using an AI learning model.

[0916] A "platform" is an online system or service for creating, purchasing, and selling AI learning models.

[0917] An "emotion recognition means" is a technology or device for analyzing emotions from a user's voice or text and recognizing those emotions.

[0918] A "speech synthesis means" is any technology or device for converting text and generated dialogue responses into speech.

[0919] The "user's ideal vocal cords" refer to the particular vocal characteristics or quality that the user desires.

[0920] A "user's ideal personality" refers to the particular personality traits and interaction style that a user desires.

[0921] "Publicly accessible platform" refers to an online environment that is accessible to other users and enables them to share, buy, and sell AI learning models.

[0922] "Dialogue response" refers to the reply or reaction that an AI learning model generates in response to user input.

[0923] The following system configuration will be described as an embodiment of the present invention.

[0924] System Configuration

[0925] Hardware

[0926] Servers: Servers with high-performance computing power are required, including servers equipped with GPUs for training AI learning models, emotion recognition, and speech synthesis.

[0927] Devices: Includes mobile devices such as smartphones and tablets used by users.

[0928] software

[0929] TensorFlow: Used for training and inference of AI learning models.

[0930] Transformers library: A library for using natural language processing models such as GPT-2.

[0931] EmotionRecognizer library: A library for recognizing emotions from user voice and text.

[0932] TextToSpeech library: A library for converting text to speech.

[0933] Processing flow

[0934] Server Processing

[0935] The server generates an AI learning model based on the ideal vocal range and personality set by the user. The generated model is used to generate appropriate dialogue responses to user input. It also uses emotion recognition means to analyze the user's emotions and adjust the dialogue responses according to those emotions.

[0936] Terminal handling

[0937] The terminal receives input from the user and transmits it to the server, and receives a dialogue response returned from the server, converts it into speech using a speech synthesis means, and provides it to the user.

[0938] Examples

[0939] Configuring Users

[0940] Users can set their ideal vocal range and personality through a smartphone application. For example, if a user desires a "calm voice and a gentle personality," the setting information is sent to the server.

[0941] Generate AI learning models

[0942] The server generates an AI learning model based on the user's settings, which mimics the user's ideal vocal range and personality.

[0943] Emotion Recognition and Dialogue Response

[0944] If a user types, "I'm very tired today," the device sends the input to the server, which uses the EmotionRecognizer library to recognize the user's emotion as "tired" and uses the GPT-2 model to generate a dialogue response such as, "That's tough. Please take some rest."

[0945] Speech synthesis

[0946] The generated dialogue response is converted into speech using the TextToSpeech library and provided to the user via the terminal.

[0947] Examples of prompt statements

[0948] User: I'm really tired today

[0949] Assistant: That's tough. Please take a break.

[0950] In this way, personalized dialogue based on the user's emotions becomes possible, and AI models that mimic ideal vocal chords and personality can be easily created and shared with other users.

[0951] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0952] Step 1:

[0953] Users set their ideal vocal range and personality through a smartphone application.

[0954] Input: User settings (ideal vocal cords and personality)

[0955] Output: Configuration information sent to the server

[0956] Specific operation: The user selects the desired vocal characteristics (e.g., calm voice) and personality (e.g., gentle personality) on the application's settings screen, and saves the settings information. The saved information is sent to the server.

[0957] Step 2:

[0958] The server generates an AI learning model based on the user's settings.

[0959] Input: User settings information

[0960] Output: An AI learning model that mimics the user's ideal vocal chords and personality.

[0961] How it works: The server uses TensorFlow and the Transformers library to train an AI learning model based on the user's preferences. The trained model mimics the user's ideal vocal chords and personality.

[0962] Step 3:

[0963] A user initiates a dialogue through a smartphone application.

[0964] Input: User input (text or voice)

[0965] Output: User input is sent to the server

[0966] Specific operation: The user inputs text or voice into the application's dialogue screen. The input data is sent to the server.

[0967] Step 4:

[0968] The server recognizes emotions based on the user's input.

[0969] Input: Data entered by the user

[0970] Output: Recognized emotion information

[0971] Specific operation: The server uses the EmotionRecognizer library to analyze emotions from the user's input data. As a result of the analysis, the user's emotional information is obtained.

[0972] Step 5:

[0973] The server generates a dialogue response based on the recognized emotion information.

[0974] Input: User input data, recognized emotion information

[0975] Output: Dialogue response (text)

[0976] How it works: The server uses the GPT-2 model to generate appropriate dialogue responses based on the user's input data and the recognized emotion information, and outputs the generated dialogue responses in text format.

[0977] Step 6:

[0978] The server converts the generated dialogue response into speech.

[0979] Input: Dialogue response (text)

[0980] Output: Dialogue response (audio)

[0981] What happens: The server uses the TextToSpeech library to convert the generated dialogue response into speech, which is then sent to the device.

[0982] Step 7:

[0983] The terminal provides the voice data to the user.

[0984] Input: Dialogue response (voice)

[0985] Output: The audio presented to the user

[0986] Specific operation: The terminal plays the voice data received from the server and provides it to the user, who can then hear the dialogue response by voice.

[0987] In this way, a personalized dialogue according to the user's emotions is realized.

[0988] Example 2

[0989] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0990] Conventional AI dialogue systems have the problem that it is difficult to generate dialogue models based on the individual voice characteristics and personality traits of a user, and that they are unable to adjust dialogue responses in real time according to the user's emotional state. Furthermore, there is a lack of a mechanism for other users to easily purchase and use the generated AI models.

[0991] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0992] In this invention, the server includes a means for a user to set ideal voice characteristics and personality characteristics, a means for preprocessing data based on the set voice characteristics and personality characteristics, a means for training a generative AI model using the preprocessed data, a means for publishing the generated AI model on a platform that can be published, a means for other users to purchase the published AI model, a means for using an emotion engine that analyzes the user's emotional state in real time, and a means for adjusting the dialogue response of the AI ​​model based on the analysis result of the emotion engine. This makes it possible to generate a dialogue model based on the individual voice characteristics and personality characteristics of the user, and further makes it possible to adjust the dialogue response in real time according to the user's emotional state. In addition, a mechanism is provided that allows other users to easily purchase and use the generated AI model.

[0993] "User" means an individual or organization that utilizes the system to set their ideal voice and personality characteristics and then uses the generated AI model.

[0994] "Voice characteristics" refer to the characteristics and texture of a user's ideal voice, and are specifically set through a voice sample.

[0995] The "personality traits" refer to the personality and response style of the dialogue model that the user idealizes, and are specifically set as options or numerical data.

[0996] "Data preprocessing" is the process of analyzing user-provided voice samples and personality traits and converting them into a format suitable for machine learning models.

[0997] A “generative AI model” is an artificial intelligence model that is trained using preprocessed data to generate dialogue responses based on a user’s ideal voice characteristics and personality traits.

[0998] The "Platform" refers to an online space where the generated AI models are made public and can be viewed and purchased by other users.

[0999] An "emotion engine" is software or hardware that analyzes a user's emotional state in real time and adjusts the dialogue response of an AI model based on the results.

[1000] "Dialogue response adjustment" is the process of changing the content and tone of the dialogue generated by the AI ​​model based on the analysis results of the emotion engine.

[1001] "Purchase means" refers to the procedures and systems through which users can obtain published AI models, including online payments and database management.

[1002] This invention is a system that allows users to set their ideal voice characteristics and personality traits, creates a generative AI model based on them, and uses it for dialogue.Furthermore, we provide a platform that allows other users to purchase and use the generated AI model.

[1003] The server provides an interface for users to set their ideal voice characteristics and personality traits. Users provide basic data for the ideal AI model by uploading voice samples and selecting personality traits. Voice analysis libraries such as voice processing libraries are used to analyze the voice samples, and appropriate data conversion methods are used to quantify the personality traits.

[1004] The server uses the preprocessed data to train a generative AI model. This process uses machine learning frameworks such as TensorFlow and PyTorch, and uses GPUs to quickly train the model. Once trained, the model will be able to generate dialogue responses that are close to the user's ideal.

[1005] The generated AI model is published on the platform by the server. The published model can be viewed and purchased by other users, and the purchase procedure is carried out through a database (e.g., MySQL) managed by the server. User B can purchase the published AI model and use it in his / her own interactions.

[1006] In addition, the server uses an emotion engine to analyze the user's emotional state in real time. The emotion engine uses an NLP library (e.g., spaCy) or a voice analysis library to analyze emotions from the user's voice and text. Based on the analysis results of the emotion engine, the AI ​​model's dialogue response is adjusted. For example, if a user shows an emotion of joy at the beginning of a dialogue, but shows an emotion of sadness midway through, the AI ​​model generates a dialogue response that corresponds to that change.

[1007] As a concrete example, User A sets "a cheerful and energetic voice" and "a friendly personality" on the platform. The server generates an AI learning model based on this setting. The generated model is used for dialogue with User A and is made public on the platform. User B purchases this model and uses it for his own dialogue.

[1008] An example of a prompt is, "Please generate an AI model with a bright, cheerful voice and a friendly personality."

[1009] In this way, it is possible to generate a dialogue model based on the individual voice characteristics and personality traits of the user, and furthermore, to adjust the dialogue response in real time according to the user's emotional state. In addition, a mechanism is provided that allows other users to easily purchase and use the generated AI model.

[1010] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1011] Step 1:

[1012] A user sets his / her ideal voice characteristics and personality traits on the platform. Specifically, the user uploads a voice sample and selects personality traits. As input, the voice sample file and the selection data of personality traits are provided. As output, these data are sent to the server.

[1013] Step 2:

[1014] The server receives the voice samples uploaded by the user and performs spectral analysis using a voice processing library. The voice sample file is used as input. Specifically, the voice data is decomposed into frequency components and features are extracted. The analyzed voice feature data is obtained as output.

[1015] Step 3:

[1016] The server converts the personality traits selected by the user into numerical data. The personality trait selection data is used as input. The server then converts the selected personality traits into numerical data and formats them in a format suitable for the machine learning model. The server then outputs the numerical personality trait data.

[1017] Step 4:

[1018] The server trains a generative AI model using the preprocessed voice feature data and the quantified personality trait data. The voice feature data and personality trait data are used as input. In concrete terms, the machine learning model is trained using TensorFlow or PyTorch, and processed at high speed using a GPU. The trained AI model is obtained as output.

[1019] Step 5:

[1020] The server publishes the generated AI model on the platform. The trained AI model is used as input. The specific operation is to register the AI ​​model in the platform's database so that other users can view and purchase it. The output is the published AI model.

[1021] Step 6:

[1022] User B purchases a published AI model. The information on the published AI model is used as input. In concrete terms, User B performs a purchase procedure on the platform, and the purchase is completed through a database managed by the server. The output is the AI ​​model purchased by User B.

[1023] Step 7:

[1024] The server uses an emotion engine to analyze the user's emotional state in real time. The user's voice and text data are used as input. Specifically, emotions are analyzed using an NLP library (e.g. spaCy) or a voice analysis library. The analyzed emotional state data is obtained as output.

[1025] Step 8:

[1026] The server adjusts the dialogue response of the AI ​​model based on the analysis results of the emotion engine. The analyzed emotional state data is used as input. As a specific operation, the server adjusts the dialogue response generation algorithm of the AI ​​model to generate a response according to the user's emotional state. As output, an adjusted dialogue response is obtained.

[1027] (Application example 2)

[1028] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1029] Conventional AI dialogue systems have difficulty recognizing the user's emotional state in real time and responding accordingly. There were also only limited systems that allowed users to set their ideal vocal cords and personality and create an AI learning model based on that. Furthermore, there was a lack of platforms that allowed these AI learning models to be made available to other users and to be purchased or sold. This made it difficult to increase user satisfaction.

[1030] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1031] In this invention, the server includes a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with a user using the AI ​​learning model, a means for creating, purchasing, and selling the AI ​​learning model on a publicly available platform, a means for recognizing the user's emotional state in real time using an emotion engine and adjusting a dialogue response based on the emotional state, and a means for including an application installed on a smartphone or a head-mounted display. This enables a dialogue response according to the user's emotional state, thereby increasing the user's satisfaction.

[1032] An "AI learning model" is an artificial intelligence learning model created to mimic the user's ideal vocal cords and personality.

[1033] A "dialogue response" is a reply generated by an AI learning model in a dialogue with a user.

[1034] A "platform" is an online system that allows users to publish, create, buy, and sell AI learning models.

[1035] An "emotion engine" is an engine for recognizing a user's emotional state in real time and adjusting the dialogue response based on the recognition results.

[1036] A "smartphone" is a portable information terminal that has mobile phone functions and can connect to the Internet and install applications.

[1037] A "head-mounted display" is a display device worn on the head and is used to provide virtual reality and augmented reality experiences.

[1038] An "application" is a software program that is installed on a smartphone or head-mounted display and provides specific functions.

[1039] The system for implementing this invention uses the following hardware and software. The hardware requires a smartphone and a head-mounted display. The software uses the OpenAI API and the EmotionEngine library.

[1040] System Configuration

[1041] 1. Server:

[1042] Means for creating an AI learning model: The server creates an AI learning model based on the ideal vocal range and personality set by the user. Using the OpenAI API, it generates prompt sentences based on the user's settings and builds the AI ​​learning model.

[1043] How to generate dialogue responses: The server receives input from the user and recognizes their emotional state using the EmotionEngine library. It then generates appropriate dialogue responses based on the recognized emotional state using the OpenAI API.

[1044] Means of providing the platform: The server provides an online platform for publishing, creating, purchasing, and selling AI learning models. On this platform, users can publish their ideal vocal range and personality, and other users can purchase or sell them.

[1045] 2. Terminal:

[1046] Smartphones and head-mounted displays: Through applications installed on these devices, users can interact with the AI ​​learning models. The applications communicate with the server and receive dialogue responses in real time.

[1047] Processing flow

[1048] 1. User setting: The user sets his / her ideal vocal range and personality using a smartphone or a head-mounted display. This setting information is sent to the server.

[1049] 2. Creating an AI learning model: The server creates an AI learning model using the OpenAI API based on the received configuration information.

[1050] 3. Emotional state recognition: When the user initiates a dialogue, the device sends the user's input to the server, which uses the EmotionEngine library to recognize the user's emotional state in real time.

[1051] 4. Generate dialogue responses: Based on the recognized emotional state, the server generates appropriate dialogue responses using the OpenAI API and sends them to the device.

[1052] 5. Dialogue execution: The terminal displays or plays audibly the dialogue response received to the user.

[1053] Examples

[1054] For example, if a user says, "I'm so happy today!", the device sends this input to the server, which uses the EmotionEngine library to recognize the "joy" emotion and the OpenAI API to generate a bright, cheerful response. The response is then sent to the device, where it is displayed or played aloud to the user.

[1055] Examples of prompt statements

[1056] User input: I'm so happy today!, Emotional state: joy

[1057] In this way, a dialogue response according to the user's emotional state is possible, and user satisfaction can be increased.

[1058] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1059] Step 1:

[1060] Users can use a smartphone or a head-mounted display to set their ideal vocal range and personality.

[1061] Input: The user's ideal vocal and personality preferences.

[1062] How it works: The user enters vocal and personality parameters into the application's settings screen.

[1063] Output: The configuration information is sent to the server.

[1064] Step 2:

[1065] Based on the configuration information received by the server, an AI learning model is created using the OpenAI API.

[1066] Input: User configuration information.

[1067] How it works: The server converts the configuration information into a prompt text and sends it to the OpenAI API, which generates an AI learning model based on the prompt text.

[1068] Output: The created AI learning model.

[1069] Step 3:

[1070] When the user initiates an interaction, the terminal transmits the user's input to the server.

[1071] Input: User interactive input.

[1072] How it works: The user enters text or voice input into an application's dialogue box, and the device sends this input to the server.

[1073] Output: User interaction input data is sent to the server.

[1074] Step 4:

[1075] The server uses the EmotionEngine library to recognize the user's emotional state in real time.

[1076] Input: User interaction input data.

[1077] How it works: The server passes dialogue input data to the EmotionEngine library, which analyzes the emotional state.

[1078] Output: Perceived emotional state.

[1079] Step 5:

[1080] The server generates appropriate dialogue responses based on the recognized emotional state using the OpenAI API.

[1081] Input: The user's dialogue input data and perceived emotional state.

[1082] How it works: The server generates a prompt and sends it to the OpenAI API, which generates a dialogue response based on the prompt.

[1083] Output: The generated dialogue response.

[1084] Step 6:

[1085] The server transmits the generated dialogue response to the terminal.

[1086] Input: The generated dialogue response.

[1087] Operation: The server sends the generated dialogue response to the terminal.

[1088] Output: Dialogue response data is sent to the terminal.

[1089] Step 7:

[1090] The terminal displays or plays audibly the dialogue response received by the terminal to the user.

[1091] Input: Dialogue response data.

[1092] Operation: The terminal displays the dialogue response data as text or plays it aloud.

[1093] Output: The interactive response is provided to the user.

[1094] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1095] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1096] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1097] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1098] [Third embodiment]

[1099] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1100] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1101] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1102] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1103] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1104] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1105] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1106] 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in FIG. 6, in the data processing device 12, a specific process is performed by the processor 28.

[1107] The storage 32 stores a specific processing program 56 .

[1108] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1109] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1110] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1111] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1112] "Example 1"

[1113] The present invention is a system including a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with a user using the AI ​​learning model, and a means for creating, purchasing, and selling the AI ​​learning model on a platform that can be made public. Specifically, a user sets his or her ideal vocal cords and personality on the platform, and an AI learning model is created based on the settings. The created AI learning model is used for interacting with the user and is made public to other users on the platform. The published AI learning model can be purchased and sold by other users.

[1114] "Example 2"

[1115] As a specific example, User A sets his / her ideal vocal range and personality on the platform, and an AI learning model is created based on that setting. The created AI learning model is used for dialogue with User A and is also made available to other users on the platform. The published AI learning model can be purchased by another User B and used for dialogue with User B.

[1116] The process flow of each embodiment will be described below.

[1117] "Example 1"

[1118] Step 1: The user sets their ideal vocal range and personality on the platform. This setting includes vocal characteristics such as pitch, speed, and intonation, and personality characteristics such as the range and intensity of emotional expression such as joy, anger, sadness, and happiness.

[1119] Step 2: An AI learning model is created based on the settings. This AI learning model has the ability to mimic the user's ideal vocal chords and personality.

[1120] Step 3: The created AI learning model is used to interact with the user, who can then interact with the AI ​​that has the vocal range and personality they desire.

[1121] Step 4: The created AI learning model is made publicly available to other users on the platform. The published AI learning model can be purchased or sold by other users.

[1122] "Example 2"

[1123] Step 1: User A sets his / her ideal vocal cords and personality on the platform. This setting includes vocal cord characteristics such as voice pitch, speed, intonation, and personality characteristics such as range and intensity of emotional expression such as joy, anger, sadness, and happiness.

[1124] Step 2: An AI learning model is created based on the settings. This AI learning model has the ability to mimic User A's ideal vocal cords and personality.

[1125] Step 3: The created AI learning model is used to interact with User A. User A can interact with an AI that has the vocal range and personality of their ideal user.

[1126] Step 4: The created AI learning model is made available to other users on the platform. The published AI learning model is purchased by other user B.

[1127] Step 5: The purchased AI learning model is used to interact with User B. User B can interact with an AI that has User A's ideal vocal range and personality.

[1128] Example 1

[1129] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1130] Conventional AI learning models have difficulty reflecting the user's ideal vocal cords and personality, and it is also difficult to share or sell the generated models with other users. For this reason, it has not been possible to create AI learning models that meet the diverse needs of users and to effectively use those models. Furthermore, there has been a lack of a platform for users to share and sell the models they have generated with other users.

[1131] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1132] In this invention, the server includes a means for a user to access the platform and set an ideal vocal cord and personality, a means for the server to receive the user's settings and generate an AI learning model using voice synthesis software and a personality model generation algorithm, and a means for the server to store the generated AI learning model on the platform and enable the user to interact with it. This allows a user to easily create an AI learning model that reflects their ideal vocal cord and personality, and to share or sell the model with other users.

[1133] "User" refers to an individual or organization that uses the system to set their ideal vocal range and personality, and to create, interact with, publish, purchase, and sell AI learning models.

[1134] The "Platform" refers to an online system that allows Users to access and set their ideal vocal range and personality, and to create, publish, purchase, and sell AI learning models.

[1135] "Server" means a computer system that receives user settings, generates AI learning models using voice synthesis software and personality model generation algorithms, and stores, publishes, and sells them on the Platform.

[1136] "Voice synthesis software" refers to software for generating voice data based on the characteristics of the vocal cords set by the user.

[1137] The "personality model generation algorithm" is an algorithm for generating personality data based on personality characteristics set by the user.

[1138] An "AI learning model" is an artificial intelligence model that mimics a user's vocal cords and personality, generated based on the user's settings.

[1139] "Dialogue" refers to the act of a user communicating with an AI learning model through text or voice.

[1140] "Publication" refers to displaying the AI ​​learning model created by a user on the platform so that other users can view and purchase it.

[1141] "Purchase" refers to the act of other users paying a fee to obtain a published AI learning model.

[1142] "Sales" refers to the act of a user providing an AI learning model they have created to other users and receiving payment in return.

[1143] "Right to Use" means the right of the Purchaser to use the AI ​​learning model acquired.

[1144] This invention is a system in which a user sets their ideal vocal cords and personality, generates an AI learning model based on that, and then interacts with, publishes, purchases, and sells the model. A specific embodiment of this system will be described below.

[1145] First, the user accesses the platform using a terminal (PC, smartphone, tablet, etc.). The platform provides an interface for setting the vocal range and personality. The user selects the pitch of the voice, tone, speaking speed, and personality traits (e.g. friendly, calm, humorous, etc.).

[1146] The server then receives the user-entered configuration data, and uses voice synthesis software (e.g., Google Text-to-Speech API) and personality model generation algorithms (e.g., OpenAI's GPT-3) to generate voice data and personality data based on the user's configuration. These data are then combined to create an AI learning model.

[1147] The server stores the generated AI learning model on the platform. The user can then interact with the AI ​​learning model through the device. When interacting, the text or voice entered by the user is sent to the server, which uses the AI ​​learning model to generate an appropriate response and returns it to the device.

[1148] For example, if a user types "What's the weather today?" in a chat window, the server will use the AI ​​learning model to generate a response such as "It's sunny today" and display it in the chat window.

[1149] Furthermore, if the user wishes, the server will publish the generated AI learning model on the platform. The published model will be available for other users to view. Other users can select a model they are interested in from the published list and click the "Purchase" button. Once the purchase procedure is complete, the server will grant the purchaser the right to use the AI ​​learning model. The purchaser will be able to use this model through their device.

[1150] As a specific example, User A wants to create an AI learning model with a "calm voice and humorous personality." User A sets the following through the interface: "Voice pitch: medium-low," "Tone: calm," and "Personality: humorous." The server receives these settings and uses voice synthesis software and a personality model generation algorithm to generate voice data and personality data based on the settings, creating an AI learning model. The created model can then be used by User A for conversations. User A can also publish this model on the platform, making it available for other users to purchase.

[1151] An example of a prompt is, "Please create an AI learning model with a calm voice and a humorous personality. Please use a medium-low pitch and a calm tone."

[1152] In this way, the system allows AI learning models to be created, interacted with, published, and sold, with servers, terminals, and users each playing their respective roles.

[1153] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1154] Step 1:

[1155] Users access the platform and set their ideal vocal range and personality.

[1156] Specifically, the user uses a terminal to access the platform's URL and log in. Then, they go to the settings screen and input their voice pitch, tone, speaking speed, and personality traits.

[1157] Input: User's vocal and personality settings (e.g. voice pitch, mid-low, tone, calm, personality, humorous)

[1158] Output: The configuration data is sent to the server.

[1159] Step 2:

[1160] The server receives the user's settings and generates an AI learning model.

[1161] Specifically, the server analyzes the received configuration data, calls voice synthesis software (e.g., Google Text-to-Speech API) to generate voice data, and simultaneously generates personality data using a personality model generation algorithm (e.g., OpenAI's GPT-3). These data are then integrated to create an AI learning model.

[1162] Input: User's configuration data

[1163] Output: The generated AI learning model

[1164] Step 3:

[1165] The server stores the generated AI learning model on the platform and allows users to interact with it.

[1166] Specifically, the server stores the generated AI learning model in a database. When the user starts a dialogue, the server calls the model and generates a response according to the user's input.

[1167] Input: Generated AI learning model

[1168] Output: AI learning model stored on the platform

[1169] Step 4:

[1170] The user interacts with the AI ​​learning model.

[1171] Specifically, a user inputs text into a chat window through a terminal. The server receives the input text, uses the AI ​​learning model to generate an appropriate response, and displays it in the chat window.

[1172] Input: User text input (e.g. "What's the weather like today?")

[1173] Output: Response from the AI ​​learning model (e.g. "It's sunny today")

[1174] Step 5:

[1175] The server publishes the user's AI learning model on the platform.

[1176] Specifically, when a user clicks the "Publish" button, the server adds the model to a published list, from which other users can view the model.

[1177] Input: User's publishing instructions

[1178] Output: AI learning model added to the public list

[1179] Step 6:

[1180] Other users can purchase published AI learning models.

[1181] Specifically, other users select a model they are interested in from the public list and click the "Purchase" button. Once the purchase procedure is completed, the server grants the purchaser the right to use the model.

[1182] Input: Purchase instructions from other users

[1183] Output: The model usage rights granted to the buyer.

[1184] Step 7:

[1185] The server grants the buyer the rights to use the model.

[1186] Specifically, the server adds the model usage rights to the buyer's account, allowing the buyer to use the model through their device.

[1187] Input: Purchase procedure completion notification

[1188] Output: Model usage rights added to buyer's account

[1189] (Application example 1)

[1190] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1191] The shopping experience in conventional virtual stores had the problem that users needed a lot of time and effort to obtain product information. In addition, it was difficult for users to find products that matched their preferences and needs, and the shopping experience was often unsatisfactory. Furthermore, the conversational assistant was not customized based on the user's vocal cords or personality, so the conversation with the user was sometimes unnatural. To solve these problems, it is necessary to provide a conversational assistant using an AI learning model that mimics the user's ideal vocal cords and personality.

[1192] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1193] In this invention, the server includes a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with a user using the AI ​​learning model, a means for creating, purchasing, and selling the AI ​​learning model on a platform that can be made public, an interactive assistant means for supporting the user's shopping in a virtual store, a means for installing the interactive assistant means in a smartphone or a head-mounted display, and a means for the interactive assistant means to provide product information based on the user's ideal vocal cords and personality. This allows the user to improve their shopping experience in the virtual store through an interactive assistant based on their own ideal vocal cords and personality.

[1194] An "AI learning model for mimicking human vocal cords and personality" is an artificial intelligence learning model designed to reproduce the characteristics of a human voice, speaking style, and personality.

[1195] "Means of interacting with a user" refers to methods and technologies for achieving communication with a user.

[1196] A "publishable platform" is an online system that enables users to share, buy, or sell user-created content or data with other users.

[1197] A "virtual store" is a virtual store operated on the Internet where users can browse and purchase products online.

[1198] An "interactive assistant means" is a technique or method for providing information or support to a user through dialogue.

[1199] A "smartphone" is a highly functional mobile phone that can connect to the Internet and use applications.

[1200] A "head-mounted display" is a display device that provides visual information to a user when worn on the head.

[1201] "Means for providing product information" refers to methods or techniques for conveying detailed product information and features to users.

[1202] In order to implement the present invention, the following system configuration and program are required.

[1203] First, the server creates an AI learning model to mimic human vocal cords and personality. This AI learning model is generated based on the ideal vocal cords and personality set by the user. Specifically, the user sets their ideal vocal cords and personality on the platform and sends the setting information to the server. The server generates an AI learning model based on this information. The generated AI learning model is used to interact with the user.

[1204] The server then provides a means for interacting with the user using the generated AI learning model. The user can do this through a smartphone or a head-mounted display. The interactive assistant generates appropriate responses based on the user's input and plays them aloud using a speech synthesis engine.

[1205] In addition, the server provides a means to create, buy and sell AI learning models on a public platform where users can publish their ideal vocal range and personality, and other users can buy and sell the AI ​​learning models.

[1206] In the virtual store, the interactive assistant means assists the user in shopping. Specifically, when the user asks a question about a product, the interactive assistant means uses an AI learning model to provide appropriate product information. For example, if the user asks, "I'm looking for new running shoes. Do you have any recommendations?", the interactive assistant means responds, "How about these running shoes? They're lightweight and highly cushioned, making them ideal for long runs."

[1207] The following hardware and software are used to realize this system:

[1208] Hardware: Servers, smartphones, head-mounted displays

[1209] Software: OpenAI API, speech synthesis engine

[1210] Examples of specific prompts are as follows:

[1211] User: "I'm looking for new running shoes. Any recommendations?"

[1212] Assistant: "How about these running shoes? They're lightweight and well-cushioned, perfect for long runs."

[1213] In this way, users can enhance their shopping experience in virtual stores through a conversational assistant based on their ideal vocal range and personality.

[1214] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1215] Step 1:

[1216] Users access the platform and set their ideal vocal range and personality.

[1217] Input: User's vocal and personality settings

[1218] Output: Configuration information sent to the server

[1219] How it works: Through the platform's interface, the user inputs the tone of voice, speaking style, personality traits, etc. This configuration information is sent to the server.

[1220] Step 2:

[1221] The server generates an AI learning model based on the configuration information received.

[1222] Input: User's vocal and personality settings

[1223] Output: AI learning model

[1224] Specific operation: The server analyzes the received configuration information and uses it as data to generate an AI learning model. It uses the OpenAI API to generate an AI learning model based on the configuration information.

[1225] Step 3:

[1226] The generated AI learning model is used to prepare for interaction with the user.

[1227] Input: AI learning model

[1228] Output: Interaction ready state

[1229] Specific operation: The server integrates the generated AI learning model into the dialogue system, making it possible to dialogue with the user.

[1230] Step 4:

[1231] Users access the conversational assistant through a smartphone or head-mounted display.

[1232] Input: User's access request

[1233] Output: Launch of interactive assistant

[1234] Specific operation: A user starts an application on a smartphone or head-mounted display to access the conversational assistant. The application connects to the server and starts the conversational assistant.

[1235] Step 5:

[1236] The user inputs a question to the interactive assistant.

[1237] Input: User question

[1238] Output: The question data is sent to the server.

[1239] Specific operation: The user inputs a question to the conversational assistant by text or voice, and the input question is sent to the server.

[1240] Step 6:

[1241] The server generates a response based on the user's question.

[1242] Input: User question data

[1243] Output: Response data

[1244] Specific operation: The server analyzes the received question data and generates appropriate responses using an AI learning model. It uses the OpenAI API to generate natural responses to questions.

[1245] Step 7:

[1246] The generated response is converted into speech by a speech synthesis engine.

[1247] Input: Response data

[1248] Output: Audio data

[1249] Specific operation: The server passes the generated response data to the speech synthesis engine to generate voice data.

[1250] Step 8:

[1251] The audio data is sent to the user's terminal and played back.

[1252] Input: Audio data

[1253] Output: Audio response to the user

[1254] Specific operation: The server transmits the generated voice data to the user's smartphone or head-mounted display. The user's device plays the received voice data and provides the user with a response.

[1255] In this way, users can enhance their shopping experience in virtual stores through a conversational assistant based on their ideal vocal range and personality.

[1256] Example 2

[1257] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1258] Conventional AI learning models make it difficult to set vocal cords and personality according to individual user needs, and the generated models are not easy to publish or for other users to purchase. For this reason, there was a demand for a system that would allow users to easily create AI models that reflect their ideal vocal cords and personality, and share them with other users.

[1259] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for a user to set his / her ideal vocal cords and personality, a means for receiving the setting information, collecting voice data and text data, and performing preprocessing, a means for generating an AI learning model using the preprocessed data, a means for publishing the generated AI learning model on a platform where it can be published, and a means for other users to purchase the published AI learning model and use it in conversations. This makes it possible for a user to easily create an AI model that reflects his / her ideal vocal cords and personality, and share it with other users.

[1260] "User" refers to an individual or organization that uses the system to set their ideal vocal range and personality and generate and use an AI learning model.

[1261] "Vocal chords" refers to the voice characteristics such as tone, pitch, and speed of the voice that are set by the user.

[1262] "Personality" refers to the personality traits of the AI ​​learning model set by the user (e.g. kind, strict, humorous, etc.).

[1263] "Settings Information" refers to information regarding vocal chords and personality that a User inputs on the Platform.

[1264] "Voice data" refers to voice data collected based on a user's vocal cord settings.

[1265] "Text data" refers to text data collected based on a user's personality settings.

[1266] "Preprocessing" refers to the process of converting collected voice and text data into a format suitable for training an AI learning model.

[1267] "AI learning model" refers to an artificial intelligence model that is generated based on user settings information.

[1268] "Publication" refers to displaying the generated AI learning model on the platform so that other users can access it.

[1269] "Platform" refers to the online system through which Users can set their vocal range and personality, and generate, publish, purchase and use AI learning models.

[1270] "Purchase" refers to the transactional act of other users acquiring published AI learning models.

[1271] "Dialogue" refers to communication between a user and an AI learning model.

[1272] This invention is a system that allows users to set their ideal vocal cords and personality, and generate, publish, purchase, and use AI learning models based on that. A specific embodiment of this system is described below.

[1273] User configuration input

[1274] Users use the platform's web interface to configure their ideal vocal range and personality. Specifically, users select the tone, pitch, speed, and personality traits (e.g., kind, stern, humorous, etc.) of their voice. These configuration information is sent to the server as form input.

[1275] Data collection and preprocessing

[1276] The server receives the user's configuration information and collects the necessary data. It collects voice and text data and performs preprocessing. Preprocessing includes normalizing the voice data and tokenizing the text data. It uses Python and TensorFlow.

[1277] Generate AI learning models

[1278] The server uses the preprocessed data to generate an AI learning model, uses deep learning techniques to build a neural network using TensorFlow or PyTorch, and trains the model based on the user's settings.

[1279] Publishing the model

[1280] The server publishes the generated AI learning model on the platform. The model is uploaded to cloud storage (e.g. AWS S3) and a public URL is generated. This URL is displayed on the platform so that other users can access it.

[1281] Purchase and Use by Other Users

[1282] User B purchases the published AI learning model on the platform. The purchase procedure is done through a web interface, and payment is processed using a payment service. After the purchase, User B can use the AI ​​learning model in his or her own interactions.

[1283] Examples

[1284] Suppose User A sets "gentle voice, humorous personality" on the platform. The server receives this setting information, collects voice and text data, and performs preprocessing. Then, it uses TensorFlow to generate an AI learning model and uploads it to cloud storage. The published model is purchased by User B and used to interact with User B.

[1285] Examples of prompt statements

[1286] "Generate an AI model with a gentle voice and a humorous personality."

[1287] This system provides a series of processes for users to set their ideal vocal range and personality, and then generate, publish, purchase, and use AI learning models based on that, allowing users to easily create AI models that meet their needs and share them with other users.

[1288] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1289] Step 1: Enter user settings

[1290] A user configures his / her ideal vocal range and personality using a web interface on the platform. Specifically, the user selects the tone, pitch, speed, and personality traits (e.g., kind, stern, humorous, etc.) of the voice. These configuration information are sent to the server as form input. The input is the user's configuration information, and the output is the configuration information sent to the server.

[1291] Specific behavior:

[1292] A user visits a web page and fills out a configuration form.

[1293] The user clicks the "Submit" button.

[1294] The server receives the configuration information and stores it in a database.

[1295] Step 2: Collect and preprocess data

[1296] The server receives the user's configuration information and collects the necessary data. It collects voice and text data and performs preprocessing. Preprocessing includes normalizing the voice data and tokenizing the text data. It uses Python and TensorFlow. The input is the user's configuration information and the output is the preprocessed data.

[1297] Specific behavior:

[1298] The server collects the voice data based on the setting information.

[1299] The server normalizes the audio data and tokenizes the text data.

[1300] The server temporarily stores the pre-processed data.

[1301] Step 3: Generate an AI learning model

[1302] The server uses the preprocessed data to generate an AI learning model. It uses deep learning techniques to build a neural network using TensorFlow or PyTorch. It trains the model based on the user's settings. The input is the preprocessed data, and the output is the trained AI learning model.

[1303] Specific behavior:

[1304] The server uses TensorFlow to build a neural network.

[1305] The server trains the model using the preprocessed data.

[1306] The server stores the trained model.

[1307] Step 4: Publish the model

[1308] The server publishes the generated AI learning model on the platform. The model is uploaded to cloud storage (e.g. AWS S3) and a public URL is generated. This URL is displayed on the platform so that other users can access it. The input is the trained AI learning model, and the output is the public URL.

[1309] Specific behavior:

[1310] The server uploads the trained model to cloud storage.

[1311] The server generates a public URL.

[1312] The server displays the URL on the platform.

[1313] Step 5: Purchase and use by others

[1314] User B purchases the published AI learning model on the platform. The purchase procedure is done through a web interface, and payment is processed using a payment service. After the purchase, User B can use the AI ​​learning model in his / her own interactions. The input is the publication URL and User B's purchase information, and the output is an AI learning model that User B can use.

[1315] Specific behavior:

[1316] User B selects a model published on the platform.

[1317] User B completes the purchase.

[1318] The server processes the payment through a payment service.

[1319] User B downloads the purchased model and uses it in the interaction.

[1320] (Application example 2)

[1321] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1322] With conventional voice synthesis technology, it was difficult for users to generate personalized voice content that reflected their ideal vocal range and personality. In addition, there were limited ways to purchase personalized voice models created by other users and enjoy content in their voices. This meant that it was not possible to meet the diverse needs of users, and there was an issue of the individualization of voice content not progressing.

[1323] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1324] In this invention, the server includes a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with a user using the AI ​​learning model, a means for creating, purchasing, and selling the AI ​​learning model on a public platform, a means for a user to set his / her ideal vocal cords and personality and generate an AI learning model based on the setting, a means for providing audio content in a personalized voice using the generated AI learning model, and a means for purchasing an AI learning model created by another user and enjoying content in that voice. This allows a user to generate personalized voice content that reflects his / her ideal vocal cords and personality, and to enjoy a variety of voice content using voice models created by other users.

[1325] The "human vocal cords" are the biological structures used by humans to produce sound and are what determine the pitch and quality of the voice.

[1326] "Personality" refers to an individual's characteristics in behavior, thinking, and emotion, and reflects that person's characteristics in dialogue and communication.

[1327] An "AI learning model" is an algorithm or network structure that uses artificial intelligence technology to learn specific data and patterns and perform tasks such as prediction, classification, and generation.

[1328] A "production means" is a method or device for producing or building something for a particular purpose.

[1329] A "means for interacting" is a method or device by which a user and a system exchange information and communicate with each other.

[1330] A "publishable platform" is an online system that enables users to share and make accessible their created content and models to other users.

[1331] A "buying or selling vehicle" is a transactional method or device by which a user obtains or provides particular content or models.

[1332] A "setting means" is a method or device that allows a user to input his / her preferences and requirements into the system so that the system operates based on them.

[1333] "Audio content" means any form of information or entertainment that is primarily audio, including music, podcasts and audiobooks.

[1334] A "personalized voice" is a voice that is customized based on the preferences and requirements of a particular user, and reflects that user's ideal vocal range and personality.

[1335] An "AI learning model created by another user" is an artificial intelligence model that a different user has generated based on their own settings and that can be used by other users.

[1336] The system for implementing this invention is configured as follows: First, the server has a means for creating an AI learning model for imitating human vocal cords and personality. This means provides an interface for the user to set their ideal vocal cords and personality, and generates an AI learning model based on the settings.

[1337] Next, the server has a means for interacting with the user using the generated AI learning model, which analyzes the text and voice data entered by the user and generates an appropriate response using the AI ​​learning model.

[1338] Additionally, the server has the means to create, buy and sell AI learning models on a publicly accessible platform that allows users to publish their ideal vocal range and personality and allows other users to buy or sell the models.

[1339] The server also has a means for providing audio content in a personalized voice using the generated AI learning model, the means for playing content selected by the user in a voice generated using the AI ​​learning model.

[1340] Finally, the server has a means for purchasing an AI learning model created by another user and enjoying content with that voice. This means enables a user to purchase an AI learning model created by another user and play content using that model.

[1341] Hardware and Software Used

[1342] Hardware: Smartphone (iOS or Android)

[1343] Software: Python, TensorFlow, Keras

[1344] Data processing and calculation

[1345] 1. Data Acquisition:

[1346] It provides an interface for users to set their ideal vocal range and personality, inputting settings such as "calm male voice" or "intelligent and friendly."

[1347] 2. Model construction:

[1348] Using TensorFlow and Keras, we build a Long Short-Term Memory (LSTM) network that learns from speech data and generates personalized speech based on user preferences.

[1349] 3. Model training:

[1350] The system generates training data based on the user's settings and trains the model, which then produces a voice that reflects the user's ideal vocal range and personality.

[1351] 4. Save and publish the model:

[1352] Save the trained model and make it accessible to others on a publicly accessible platform.

[1353] Examples

[1354] User A opens the app and sets his ideal vocal range as "calm male voice" and his personality as "intelligent and friendly." The app generates an AI learning model based on this, and User A can listen to audiobooks in that voice. User B can also purchase User A's model and enjoy podcasts in the same voice.

[1355] Examples of prompt statements

[1356] Set the user's ideal vocal range and personality, for example, "calm male voice" or "intelligent and friendly."

[1357] In this way, a system is provided that allows the user to enjoy content in his or her ideal voice.

[1358] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1359] Step 1:

[1360] A user starts the application using a terminal and sets the ideal vocal range and personality. The user inputs settings such as "calm male voice" or "intelligent and friendly." The input data is the setting information of the user's vocal range and personality. The terminal sends this setting information to the server.

[1361] Step 2:

[1362] The server builds an AI learning model based on the received user configuration information. Specifically, it builds an LSTM (Long Short-Term Memory) network using TensorFlow and Keras. The input data is the user configuration information, and the output data is the initialized AI learning model.

[1363] Step 3:

[1364] The server generates training data based on the user's settings. Specifically, it collects voice data and text data that match the user's ideal vocal cords and personality, and prepares this as a training data set. The input data is the user's settings, and the output data is the training data set.

[1365] Step 4:

[1366] The server trains the AI ​​learning model using the training dataset. Specifically, it inputs the training data into the LSTM network and optimizes the model parameters. The input data is the training dataset, and the output data is the trained AI learning model.

[1367] Step 5:

[1368] The server stores the trained AI learning model and uploads it to a publicly available platform, so that other users can purchase or use the model. The input data is the trained AI learning model, and the output data is the published AI learning model.

[1369] Step 6:

[1370] A user uses a terminal to select a published AI learning model and play audio content. Specifically, the user downloads the selected model and uses the model to play audiobooks or podcasts. The input data is the published AI learning model and audio content, and the output data is the audio content played in a personalized voice.

[1371] Step 7:

[1372] Other users use their devices to purchase the published AI learning model and enjoy the content with that voice. Specifically, other users purchase the published model and play the content using that model. The input data is the published AI learning model and a purchase request, and the output data is the purchased AI learning model and the content played with a personalized voice.

[1373] In this way, users can enjoy content using their ideal voice, and other users can also use the model to enjoy a variety of voice content.

[1374] In addition, an emotion engine that estimates the emotion of the user may be further combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[1375] "Example 1"

[1376] As one embodiment of the present invention, there is a system of an AI learning model combined with an emotion engine. This system recognizes emotions from the user's voice and adjusts the dialogue response of the AI ​​learning model based on the emotion. Specifically, when a user utters a voice indicating an emotion of joy, the emotion engine recognizes the emotion of joy, and the AI ​​learning model generates a dialogue response corresponding to the emotion of joy based on the recognition result. Also, when a user utters a voice indicating an emotion of sadness, the emotion engine recognizes the emotion of sadness, and the AI ​​learning model generates a dialogue response corresponding to the sadness based on the recognition result. In this way, by combining the emotion engine and the AI ​​learning model, dialogue according to the user's emotional state is possible.

[1377] "Example 2"

[1378] Another embodiment of the present invention is a system in which an emotion engine updates the user's emotional state in real time and adjusts the dialogue response of the AI ​​learning model based on the update. Specifically, if the user changes his / her emotional state during the dialogue, the emotion engine recognizes the change in real time, and the AI ​​learning model adjusts the dialogue response based on the recognition result. For example, if the user shows an emotion of joy at the beginning of the dialogue but shows an emotion of sadness in the middle, the emotion engine recognizes the change in emotion in real time, and the AI ​​learning model adjusts the dialogue response from joy to sadness based on the recognition result. In this way, by combining the emotion engine and the AI ​​learning model, a dialogue according to the user's emotional state in real time is possible.

[1379] The process flow of each embodiment will be described below.

[1380] "Example 1"

[1381] Step 1: A user produces a speech sound that reflects the user's emotional state.

[1382] Step 2: The emotion engine recognizes emotions from the user's voice by analyzing voice tone, pitch, stress, and other elements.

[1383] Step 3: The AI ​​learning model receives the recognition results from the emotion engine and generates a dialogue response corresponding to the emotion, which is adapted to the user's emotional state.

[1384] "Example 2"

[1385] Step 1: A user produces speech during a dialogue that reflects changes in the user's emotional state.

[1386] Step 2: The emotion engine recognizes emotional changes in the user's voice in real time by analyzing changes in tone, pitch, volume, and other elements of the voice.

[1387] Step 3: The AI ​​learning model receives the recognition results from the emotion engine and generates a dialogue response that corresponds to the emotion changes, which is adapted to the changes in the user's emotional state.

[1388] Example 1

[1389] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1390] In conventional conversational AI systems, it was difficult to create a dialogue that reflected the individual voice characteristics and personality traits of the user, and responses were not adjusted sufficiently based on emotion recognition. This made it difficult to provide a natural and friendly conversational experience for users. In addition, the process of publishing, purchasing, and selling the generated AI models was complicated, making it difficult to share models between users.

[1391] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1392] In this invention, the server includes a means for creating a machine learning model for imitating human voice characteristics and personality characteristics, a means for interacting with a user using the machine learning model, a means for creating, purchasing, and selling the machine learning model on an online platform that can be made public, and an emotion recognition means for recognizing emotions from the user's voice and adjusting the dialogue response based on the emotions. This makes it possible to provide a natural and friendly dialogue experience that reflects the individual voice characteristics and personality characteristics of the user. In addition, the process of publishing, purchasing, and selling the generated AI model is simplified, making it easy to share the model among users.

[1393] A "machine learning model" is an algorithm that learns from data to mimic voice characteristics or personality traits and perform specific tasks.

[1394] "Voice characteristics" refers to the physical characteristics of speech, such as pitch, tempo, and volume.

[1395] "Personality traits" refer to characteristics that describe an individual's character, such as friendliness or calmness.

[1396] An "emotion recognition means" is a technology for analyzing emotions from a user's voice and recognizing those emotions.

[1397] "Dialogue response" refers to a reply generated in response to a user's input.

[1398] An "online platform" is a system accessible via the Internet that provides a place for users to create, publish, purchase and sell AI models.

[1399] "User" refers to any individual or entity that uses the System.

[1400] "Public" refers to making the generated AI model available for other users to view.

[1401] "Purchase" refers to the act of other users acquiring a published AI model.

[1402] "Sales" refers to the act of providing the generated AI model to other users.

[1403] This invention is a system for creating a machine learning model for imitating human voice characteristics and personality characteristics and for interacting with a user. Furthermore, this system has a function for publishing, purchasing, and selling the generated machine learning model on an online platform. It also includes an emotion recognition means for recognizing emotions from the user's voice and adjusting the dialogue response based on the emotions.

[1404] Generating a Program

[1405] The server generates a machine learning model based on the ideal voice and personality characteristics set by the user on the online platform. In this case, the server collects voice data and personality data, and uses these data to train the machine learning model. Specifically, the following hardware and software are used:

[1406] Hardware: High performance GPU

[1407] Software: Speech processing libraries, natural language processing libraries (e.g. Transformers)

[1408] Data processing and calculation

[1409] The server performs the following steps to process the voice and personality data:

[1410] 1. The server analyzes the voice data uploaded by the user using a voice processing library and extracts voice features.

[1411] 2. The server analyzes the personality data entered by the user using Transformers and extracts personality features.

[1412] 3. The server combines these features to train a machine learning model and generate a model that mimics the user's ideal voice and personality characteristics.

[1413] User interaction

[1414] The terminal uses the generated machine learning model to converse with the user. Specifically, when the user speaks to the terminal, the terminal transmits the voice to the server. The server recognizes emotions from the voice using an emotion recognition means, and the machine learning model generates an appropriate dialogue response based on the recognized emotion and transmits it back to the terminal. The terminal returns the generated dialogue response to the user by voice.

[1415] Publish, buy and sell on the platform

[1416] The server publishes the generated machine learning model on an online platform. The published model is added to a list so that other users can view it. Users can view the published machine learning model and purchase or sell it.

[1417] Examples of concrete examples and prompts

[1418] Examples

[1419] User A sets up a machine learning model with a "calm voice and friendly personality" on an online platform. The server collects User A's voice data and personality data, and extracts features using a voice processing library and Transformers. It then trains the machine learning model using a high-performance GPU to generate a model that mimics User A's ideal voice and personality characteristics. The generated model is used for conversations with User A and is also made public on the online platform. Another User B can like this model and purchase it.

[1420] Examples of prompt statements

[1421] "Create a machine learning model with a calm voice and a friendly personality. If the user makes a speech that indicates the emotion of happiness, generate a dialogue response that corresponds to that emotion."

[1422] In this way, the server, terminal, and user each fulfill their respective roles, allowing the entire system to operate smoothly.

[1423] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1424] Step 1:

[1425] Users access the platform and set their ideal voice and personality characteristics.

[1426] Specifically, users access the platform using a web browser or a dedicated app, select a "calm voice" and a "friendly personality" on the settings screen, and click the "Complete settings" button.

[1427] Input: User's voice and personality settings

[1428] Output: Configuration information sent to the server

[1429] Step 2:

[1430] The server collects voice and personality data and generates a machine learning model.

[1431] Specifically, the server performs the following actions:

[1432] 1. The server analyzes the audio data uploaded by the user using a voice processing library and extracts audio features (e.g. pitch, tempo, volume, etc.).

[1433] 2. The server analyzes the personality data entered by the user using Transformers and extracts personality features (e.g., friendliness, calmness, etc.).

[1434] 3. The server combines these features and uses a high-performance GPU to train a machine learning model to generate a model that mimics the user's ideal voice and personality characteristics.

[1435] Input: User's voice and personality data

[1436] Output: The generated machine learning model

[1437] Step 3:

[1438] The server publishes the generated machine learning model on the platform.

[1439] Specifically, the server publishes the generated model on the platform and displays detailed information about the model (e.g. vocal characteristics, personality traits, price, etc.) so that other users can view this information.

[1440] Input: A generated machine learning model

[1441] Output: Detailed information about the published machine learning model

[1442] Step 4:

[1443] The device uses machine learning models to interact with the user.

[1444] Specifically, when a user speaks to the device, the device transmits the voice to the server. The server uses emotion recognition means to recognize emotions (e.g., interest, curiosity, etc.) from the voice, and the machine learning model generates an appropriate dialogue response and sends it back to the device. The device then returns this response to the user by voice.

[1445] Input: User's voice input

[1446] Output: Dialogue response

[1447] Step 5:

[1448] Other users can buy or sell published machine learning models.

[1449] Specifically, other users can browse the machine learning models published on the platform and purchase the models they like. Once the purchase is complete, the server grants the buyer the right to use the model. Users can also sell the machine learning models they have created on the platform.

[1450] Input: Details of published machine learning models

[1451] Output: The rights to use the machine learning models purchased or sold

[1452] (Application example 1)

[1453] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1454] Conventional conversational AI systems have had difficulty recognizing a user's emotions and generating appropriate conversational responses. In addition, there was a lack of a way for a user to easily create an AI model that imitates their ideal vocal cords and personality and share it with other users. This limited the user experience and made it difficult to realize personalized conversations.

[1455] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1456] In this invention, the server includes a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with a user using the AI ​​learning model, a means for creating, purchasing, and selling the AI ​​learning model on a platform that can be made public, an emotion recognition means for recognizing the user's emotion and adjusting the dialogue response based on the emotion, and a voice synthesis means for converting the dialogue response generated based on the emotion recognition means into voice. This enables personalized dialogue according to the user's emotion, and allows an AI model that imitates an ideal vocal cord and personality to be easily created and shared with other users.

[1457] An "AI learning model" is an artificial intelligence learning algorithm designed to mimic human vocal cords and personality.

[1458] An "interactive means" is a method or device for communicating with a user using an AI learning model.

[1459] A "platform" is an online system or service for creating, purchasing, and selling AI learning models.

[1460] An "emotion recognition means" is a technology or device for analyzing emotions from a user's voice or text and recognizing those emotions.

[1461] A "speech synthesis means" is any technology or device for converting text and generated dialogue responses into speech.

[1462] The "user's ideal vocal cords" refer to the particular vocal characteristics or quality that the user desires.

[1463] A "user's ideal personality" refers to the particular personality traits and interaction style that a user desires.

[1464] "Publicly accessible platform" refers to an online environment that is accessible to other users and enables them to share, buy, and sell AI learning models.

[1465] "Dialogue response" refers to the reply or reaction that an AI learning model generates in response to user input.

[1466] The following system configuration will be described as an embodiment of the present invention.

[1467] System Configuration

[1468] Hardware

[1469] Servers: Servers with high-performance computing power are required, including servers equipped with GPUs for training AI learning models, emotion recognition, and speech synthesis.

[1470] Devices: Includes mobile devices such as smartphones and tablets used by users.

[1471] software

[1472] TensorFlow: Used for training and inference of AI learning models.

[1473] Transformers library: A library for using natural language processing models such as GPT-2.

[1474] EmotionRecognizer library: A library for recognizing emotions from user voice and text.

[1475] TextToSpeech library: A library for converting text to speech.

[1476] Processing flow

[1477] Server Processing

[1478] The server generates an AI learning model based on the ideal vocal range and personality set by the user. The generated model is used to generate appropriate dialogue responses to user input. It also uses emotion recognition means to analyze the user's emotions and adjust the dialogue responses according to those emotions.

[1479] Terminal handling

[1480] The terminal receives input from the user and transmits it to the server, and receives a dialogue response returned from the server, converts it into speech using a speech synthesis means, and provides it to the user.

[1481] Examples

[1482] Configuring Users

[1483] Users can set their ideal vocal range and personality through a smartphone application. For example, if a user desires a "calm voice and a gentle personality," the setting information is sent to the server.

[1484] Generate AI learning models

[1485] The server generates an AI learning model based on the user's settings, which mimics the user's ideal vocal range and personality.

[1486] Emotion Recognition and Dialogue Response

[1487] If a user types, "I'm very tired today," the device sends the input to the server, which uses the EmotionRecognizer library to recognize the user's emotion as "tired" and uses the GPT-2 model to generate a dialogue response such as, "That's tough. Please take some rest."

[1488] Speech synthesis

[1489] The generated dialogue response is converted into speech using the TextToSpeech library and provided to the user via the terminal.

[1490] Examples of prompt statements

[1491] User: I'm really tired today

[1492] Assistant: That's tough. Please take a break.

[1493] In this way, personalized dialogue based on the user's emotions becomes possible, and AI models that mimic ideal vocal chords and personality can be easily created and shared with other users.

[1494] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1495] Step 1:

[1496] Users set their ideal vocal range and personality through a smartphone application.

[1497] Input: User settings (ideal vocal cords and personality)

[1498] Output: Configuration information sent to the server

[1499] Specific operation: The user selects the desired vocal characteristics (e.g., calm voice) and personality (e.g., gentle personality) on the application's settings screen, and saves the settings information. The saved information is sent to the server.

[1500] Step 2:

[1501] The server generates an AI learning model based on the user's settings.

[1502] Input: User settings information

[1503] Output: An AI learning model that mimics the user's ideal vocal chords and personality.

[1504] How it works: The server uses TensorFlow and the Transformers library to train an AI learning model based on the user's preferences. The trained model mimics the user's ideal vocal chords and personality.

[1505] Step 3:

[1506] A user initiates a dialogue through a smartphone application.

[1507] Input: User input (text or voice)

[1508] Output: User input is sent to the server

[1509] Specific operation: The user inputs text or voice into the application's dialogue screen. The input data is sent to the server.

[1510] Step 4:

[1511] The server recognizes emotions based on the user's input.

[1512] Input: Data entered by the user

[1513] Output: Recognized emotion information

[1514] Specific operation: The server uses the EmotionRecognizer library to analyze emotions from the user's input data. As a result of the analysis, the user's emotional information is obtained.

[1515] Step 5:

[1516] The server generates a dialogue response based on the recognized emotion information.

[1517] Input: User input data, recognized emotion information

[1518] Output: Dialogue response (text)

[1519] How it works: The server uses the GPT-2 model to generate appropriate dialogue responses based on the user's input data and the recognized emotion information, and outputs the generated dialogue responses in text format.

[1520] Step 6:

[1521] The server converts the generated dialogue response into speech.

[1522] Input: Dialogue response (text)

[1523] Output: Dialogue response (audio)

[1524] What happens: The server uses the TextToSpeech library to convert the generated dialogue response into speech, which is then sent to the device.

[1525] Step 7:

[1526] The terminal provides the voice data to the user.

[1527] Input: Dialogue response (voice)

[1528] Output: The audio presented to the user

[1529] Specific operation: The terminal plays the voice data received from the server and provides it to the user, who can then hear the dialogue response by voice.

[1530] In this way, a personalized dialogue according to the user's emotions is realized.

[1531] Example 2

[1532] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1533] Conventional AI dialogue systems have the problem that it is difficult to generate dialogue models based on the individual voice characteristics and personality traits of a user, and that they are unable to adjust dialogue responses in real time according to the user's emotional state. Furthermore, there is a lack of a mechanism for other users to easily purchase and use the generated AI models.

[1534] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1535] In this invention, the server includes a means for a user to set ideal voice characteristics and personality characteristics, a means for preprocessing data based on the set voice characteristics and personality characteristics, a means for training a generative AI model using the preprocessed data, a means for publishing the generated AI model on a platform that can be published, a means for other users to purchase the published AI model, a means for using an emotion engine that analyzes the user's emotional state in real time, and a means for adjusting the dialogue response of the AI ​​model based on the analysis result of the emotion engine. This makes it possible to generate a dialogue model based on the individual voice characteristics and personality characteristics of the user, and further makes it possible to adjust the dialogue response in real time according to the user's emotional state. In addition, a mechanism is provided that allows other users to easily purchase and use the generated AI model.

[1536] "User" means an individual or organization that utilizes the system to set their ideal voice and personality characteristics and then uses the generated AI model.

[1537] "Voice characteristics" refer to the characteristics and texture of a user's ideal voice, and are specifically set through a voice sample.

[1538] The "personality traits" refer to the personality and response style of the dialogue model that the user idealizes, and are specifically set as options or numerical data.

[1539] "Data preprocessing" is the process of analyzing user-provided voice samples and personality traits and converting them into a format suitable for machine learning models.

[1540] A "generative AI model" is an artificial intelligence model that is trained using preprocessed data to generate dialogue responses based on a user's ideal voice characteristics and personality traits.

[1541] The "Platform" refers to an online space where the generated AI models are made public and can be viewed and purchased by other users.

[1542] An "emotion engine" is software or hardware that analyzes a user's emotional state in real time and adjusts the dialogue response of an AI model based on the results.

[1543] "Dialogue response adjustment" is the process of changing the content and tone of the dialogue generated by the AI ​​model based on the analysis results of the emotion engine.

[1544] "Purchase means" refers to the procedures and systems through which users can obtain published AI models, including online payments and database management.

[1545] This invention is a system that allows users to set their ideal voice characteristics and personality traits, creates a generative AI model based on them, and uses it for dialogue.Furthermore, we provide a platform that allows other users to purchase and use the generated AI model.

[1546] The server provides an interface for users to set their ideal voice characteristics and personality traits. Users provide basic data for the ideal AI model by uploading voice samples and selecting personality traits. Voice analysis libraries such as voice processing libraries are used to analyze the voice samples, and appropriate data conversion methods are used to quantify the personality traits.

[1547] The server uses the preprocessed data to train a generative AI model. This process uses machine learning frameworks such as TensorFlow and PyTorch, and uses GPUs to quickly train the model. Once trained, the model will be able to generate dialogue responses that are close to the user's ideal.

[1548] The generated AI model is published on the platform by the server. The published model can be viewed and purchased by other users, and the purchase procedure is carried out through a database (e.g., MySQL) managed by the server. User B can purchase the published AI model and use it in his / her own interactions.

[1549] In addition, the server uses an emotion engine to analyze the user's emotional state in real time. The emotion engine uses an NLP library (e.g., spaCy) or a voice analysis library to analyze emotions from the user's voice and text. Based on the analysis results of the emotion engine, the AI ​​model's dialogue response is adjusted. For example, if a user shows an emotion of joy at the beginning of a dialogue, but shows an emotion of sadness midway through, the AI ​​model generates a dialogue response that corresponds to that change.

[1550] As a concrete example, User A sets "a cheerful and energetic voice" and "a friendly personality" on the platform. The server generates an AI learning model based on this setting. The generated model is used for dialogue with User A and is made public on the platform. User B purchases this model and uses it for his own dialogue.

[1551] An example of a prompt is, "Please generate an AI model with a bright, cheerful voice and a friendly personality."

[1552] In this way, it is possible to generate a dialogue model based on the individual voice characteristics and personality traits of the user, and furthermore, to adjust the dialogue response in real time according to the user's emotional state. In addition, a mechanism is provided that allows other users to easily purchase and use the generated AI model.

[1553] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1554] Step 1:

[1555] A user sets his / her ideal voice characteristics and personality traits on the platform. Specifically, the user uploads a voice sample and selects personality traits. As input, the voice sample file and the selection data of personality traits are provided. As output, these data are sent to the server.

[1556] Step 2:

[1557] The server receives the voice samples uploaded by the user and performs spectral analysis using a voice processing library. The voice sample file is used as input. Specifically, the voice data is decomposed into frequency components and features are extracted. The analyzed voice feature data is obtained as output.

[1558] Step 3:

[1559] The server converts the personality traits selected by the user into numerical data. The personality trait selection data is used as input. The server then converts the selected personality traits into numerical data and formats them in a format suitable for the machine learning model. The server then outputs the numerical personality trait data.

[1560] Step 4:

[1561] The server trains a generative AI model using the preprocessed voice feature data and the quantified personality trait data. The voice feature data and personality trait data are used as input. In concrete terms, the machine learning model is trained using TensorFlow or PyTorch, and processed at high speed using a GPU. The trained AI model is obtained as output.

[1562] Step 5:

[1563] The server publishes the generated AI model on the platform. The trained AI model is used as input. The specific operation is to register the AI ​​model in the platform's database so that other users can view and purchase it. The output is the published AI model.

[1564] Step 6:

[1565] User B purchases a published AI model. The information on the published AI model is used as input. In concrete terms, User B performs a purchase procedure on the platform, and the purchase is completed through a database managed by the server. The output is the AI ​​model purchased by User B.

[1566] Step 7:

[1567] The server uses an emotion engine to analyze the user's emotional state in real time. The user's voice and text data are used as input. Specifically, emotions are analyzed using an NLP library (e.g. spaCy) or a voice analysis library. The analyzed emotional state data is obtained as output.

[1568] Step 8:

[1569] The server adjusts the dialogue response of the AI ​​model based on the analysis results of the emotion engine. The analyzed emotional state data is used as input. As a specific operation, the server adjusts the dialogue response generation algorithm of the AI ​​model to generate a response according to the user's emotional state. As output, an adjusted dialogue response is obtained.

[1570] (Application example 2)

[1571] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1572] Conventional AI dialogue systems have difficulty recognizing the user's emotional state in real time and responding accordingly. There were also only limited systems that allowed users to set their ideal vocal cords and personality and create an AI learning model based on that. Furthermore, there was a lack of platforms that allowed these AI learning models to be made available to other users and to be purchased or sold. This made it difficult to increase user satisfaction.

[1573] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1574] In this invention, the server includes a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with a user using the AI ​​learning model, a means for creating, purchasing, and selling the AI ​​learning model on a publicly available platform, a means for recognizing the user's emotional state in real time using an emotion engine and adjusting a dialogue response based on the emotional state, and a means for including an application installed on a smartphone or a head-mounted display. This enables a dialogue response according to the user's emotional state, thereby increasing the user's satisfaction.

[1575] An "AI learning model" is an artificial intelligence learning model created to mimic the user's ideal vocal cords and personality.

[1576] A "dialogue response" is a reply generated by an AI learning model in a dialogue with a user.

[1577] A "platform" is an online system that allows users to publish, create, buy, and sell AI learning models.

[1578] An "emotion engine" is an engine for recognizing a user's emotional state in real time and adjusting the dialogue response based on the recognition results.

[1579] A "smartphone" is a portable information terminal that has mobile phone functions and can connect to the Internet and install applications.

[1580] A "head-mounted display" is a display device worn on the head and is used to provide virtual reality and augmented reality experiences.

[1581] An "application" is a software program that is installed on a smartphone or head-mounted display and provides specific functions.

[1582] The system for implementing this invention uses the following hardware and software. The hardware requires a smartphone and a head-mounted display. The software uses the OpenAI API and the EmotionEngine library.

[1583] System Configuration

[1584] 1. Server:

[1585] Means for creating an AI learning model: The server creates an AI learning model based on the ideal vocal range and personality set by the user. Using the OpenAI API, it generates prompt sentences based on the user's settings and builds the AI ​​learning model.

[1586] How to generate dialogue responses: The server receives input from the user and recognizes their emotional state using the EmotionEngine library. It then generates appropriate dialogue responses based on the recognized emotional state using the OpenAI API.

[1587] Means of providing the platform: The server provides an online platform for publishing, creating, purchasing, and selling AI learning models. On this platform, users can publish their ideal vocal range and personality, and other users can purchase or sell them.

[1588] 2. Terminal:

[1589] Smartphones and head-mounted displays: Through applications installed on these devices, users can interact with the AI ​​learning models. The applications communicate with the server and receive dialogue responses in real time.

[1590] Processing flow

[1591] 1. User setting: The user sets his / her ideal vocal range and personality using a smartphone or a head-mounted display. This setting information is sent to the server.

[1592] 2. Creating an AI learning model: The server creates an AI learning model using the OpenAI API based on the received configuration information.

[1593] 3. Emotional state recognition: When the user initiates a dialogue, the device sends the user's input to the server, which uses the EmotionEngine library to recognize the user's emotional state in real time.

[1594] 4. Generate dialogue responses: Based on the recognized emotional state, the server generates appropriate dialogue responses using the OpenAI API and sends them to the device.

[1595] 5. Dialogue execution: The terminal displays or plays audibly the dialogue response received to the user.

[1596] Examples

[1597] For example, if a user says, "I'm so happy today!", the device sends this input to the server, which uses the EmotionEngine library to recognize the "joy" emotion and the OpenAI API to generate a bright, cheerful response. The response is then sent to the device, where it is displayed or played aloud to the user.

[1598] Examples of prompt statements

[1599] User input: I'm so happy today!, Emotional state: joy

[1600] In this way, a dialogue response according to the user's emotional state is possible, and user satisfaction can be increased.

[1601] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1602] Step 1:

[1603] Users can use a smartphone or a head-mounted display to set their ideal vocal range and personality.

[1604] Input: The user's ideal vocal and personality preferences.

[1605] How it works: The user enters vocal and personality parameters into the application's settings screen.

[1606] Output: The configuration information is sent to the server.

[1607] Step 2:

[1608] Based on the configuration information received by the server, an AI learning model is created using the OpenAI API.

[1609] Input: User configuration information.

[1610] How it works: The server converts the configuration information into a prompt text and sends it to the OpenAI API, which generates an AI learning model based on the prompt text.

[1611] Output: The created AI learning model.

[1612] Step 3:

[1613] When the user initiates an interaction, the terminal transmits the user's input to the server.

[1614] Input: User interactive input.

[1615] How it works: The user enters text or voice input into an application's dialogue box, and the device sends this input to the server.

[1616] Output: User interaction input data is sent to the server.

[1617] Step 4:

[1618] The server uses the EmotionEngine library to recognize the user's emotional state in real time.

[1619] Input: User interaction input data.

[1620] How it works: The server passes dialogue input data to the EmotionEngine library, which analyzes the emotional state.

[1621] Output: Perceived emotional state.

[1622] Step 5:

[1623] The server generates appropriate dialogue responses based on the recognized emotional state using the OpenAI API.

[1624] Input: The user's dialogue input data and perceived emotional state.

[1625] How it works: The server generates a prompt and sends it to the OpenAI API, which generates a dialogue response based on the prompt.

[1626] Output: The generated dialogue response.

[1627] Step 6:

[1628] The server transmits the generated dialogue response to the terminal.

[1629] Input: The generated dialogue response.

[1630] Operation: The server sends the generated dialogue response to the terminal.

[1631] Output: Dialogue response data is sent to the terminal.

[1632] Step 7:

[1633] The terminal displays or plays audibly the dialogue response received by the terminal to the user.

[1634] Input: Dialogue response data.

[1635] Operation: The terminal displays the dialogue response data as text or plays it aloud.

[1636] Output: The interactive response is provided to the user.

[1637] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1638] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1639] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1640] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1641] [Fourth embodiment]

[1642] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1643] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1644] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1645] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1646] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1647] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1648] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1649] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1650] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1651] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1652] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1653] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1654] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1655] "Example 1"

[1656] The present invention is a system including a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with a user using the AI ​​learning model, and a means for creating, purchasing, and selling the AI ​​learning model on a platform that can be made public. Specifically, a user sets his or her ideal vocal cords and personality on the platform, and an AI learning model is created based on the settings. The created AI learning model is used for interacting with the user and is made public to other users on the platform. The published AI learning model can be purchased and sold by other users.

[1657] "Example 2"

[1658] As a specific example, User A sets his / her ideal vocal range and personality on the platform, and an AI learning model is created based on that setting. The created AI learning model is used for dialogue with User A and is also made available to other users on the platform. The published AI learning model can be purchased by another User B and used for dialogue with User B.

[1659] The process flow of each embodiment will be described below.

[1660] "Example 1"

[1661] Step 1: The user sets their ideal vocal range and personality on the platform. This setting includes vocal characteristics such as pitch, speed, and intonation, and personality characteristics such as the range and intensity of emotional expression such as joy, anger, sadness, and happiness.

[1662] Step 2: An AI learning model is created based on the settings. This AI learning model has the ability to mimic the user's ideal vocal chords and personality.

[1663] Step 3: The created AI learning model is used to interact with the user, who can then interact with the AI ​​that has the vocal range and personality of their dreams.

[1664] Step 4: The created AI learning model is made publicly available to other users on the platform. The published AI learning model can be purchased or sold by other users.

[1665] "Example 2"

[1666] Step 1: User A sets his / her ideal vocal cords and personality on the platform. This setting includes vocal cord characteristics such as voice pitch, speed, intonation, and personality characteristics such as range and intensity of emotional expression such as joy, anger, sadness, and happiness.

[1667] Step 2: An AI learning model is created based on the settings. This AI learning model has the ability to mimic User A's ideal vocal cords and personality.

[1668] Step 3: The created AI learning model is used to interact with User A. User A can interact with an AI that has the vocal range and personality of their ideal user.

[1669] Step 4: The created AI learning model is made available to other users on the platform. The published AI learning model is purchased by other user B.

[1670] Step 5: The purchased AI learning model is used to interact with User B. User B can interact with an AI that has User A's ideal vocal range and personality.

[1671] Example 1

[1672] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1673] Conventional AI learning models have difficulty reflecting the user's ideal vocal cords and personality, and it is also difficult to share or sell the generated models with other users. For this reason, it has not been possible to create AI learning models that meet the diverse needs of users and to effectively use those models. Furthermore, there has been a lack of a platform for users to share and sell the models they have generated with other users.

[1674] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1675] In this invention, the server includes a means for a user to access the platform and set an ideal vocal cord and personality, a means for the server to receive the user's settings and generate an AI learning model using voice synthesis software and a personality model generation algorithm, and a means for the server to store the generated AI learning model on the platform and enable the user to interact with it. This allows a user to easily create an AI learning model that reflects their ideal vocal cord and personality, and to share or sell the model with other users.

[1676] "User" refers to an individual or organization that uses the system to set their ideal vocal range and personality, and to create, interact with, publish, purchase, and sell AI learning models.

[1677] The "Platform" refers to an online system that allows Users to access and set their ideal vocal range and personality, and to create, publish, purchase, and sell AI learning models.

[1678] "Server" means a computer system that receives user settings, generates AI learning models using voice synthesis software and personality model generation algorithms, and stores, publishes, and sells them on the Platform.

[1679] "Voice synthesis software" refers to software for generating voice data based on the characteristics of the vocal cords set by the user.

[1680] The "personality model generation algorithm" is an algorithm for generating personality data based on personality characteristics set by the user.

[1681] An "AI learning model" is an artificial intelligence model that mimics a user's vocal cords and personality, generated based on the user's settings.

[1682] "Dialogue" refers to the act of a user communicating with an AI learning model through text or voice.

[1683] "Publication" refers to displaying the AI ​​learning model created by a user on the platform so that other users can view and purchase it.

[1684] "Purchase" refers to the act of other users paying a fee to obtain a published AI learning model.

[1685] "Sales" refers to the act of a user providing an AI learning model they have created to other users and receiving payment in return.

[1686] "Right to Use" means the right of the Purchaser to use the AI ​​learning model acquired.

[1687] This invention is a system in which a user sets their ideal vocal cords and personality, generates an AI learning model based on that, and then interacts with, publishes, purchases, and sells the model. A specific embodiment of this system will be described below.

[1688] First, the user accesses the platform using a terminal (PC, smartphone, tablet, etc.). The platform provides an interface for setting the vocal range and personality. The user selects the pitch of the voice, tone, speaking speed, and personality traits (e.g. friendly, calm, humorous, etc.).

[1689] The server then receives the user-entered configuration data, and uses voice synthesis software (e.g., Google Text-to-Speech API) and personality model generation algorithms (e.g., OpenAI's GPT-3) to generate voice data and personality data based on the user's configuration. These data are then combined to create an AI learning model.

[1690] The server stores the generated AI learning model on the platform. The user can then interact with the AI ​​learning model through the device. When interacting, the text or voice entered by the user is sent to the server, which uses the AI ​​learning model to generate an appropriate response and returns it to the device.

[1691] For example, if a user types "What's the weather today?" into a chat window, the server will use the AI ​​learning model to generate a response such as "It's sunny today" and display it in the chat window.

[1692] Furthermore, if the user wishes, the server will publish the generated AI learning model on the platform. The published model will be available for other users to view. Other users can select a model they are interested in from the published list and click the "Purchase" button. Once the purchase procedure is complete, the server will grant the purchaser the right to use the AI ​​learning model. The purchaser will be able to use this model through their device.

[1693] As a specific example, User A wants to create an AI learning model with a "calm voice and humorous personality." User A sets the following through the interface: "Voice pitch: medium-low," "Tone: calm," and "Personality: humorous." The server receives these settings and uses voice synthesis software and a personality model generation algorithm to generate voice data and personality data based on the settings, creating an AI learning model. The created model can then be used by User A for conversations. User A can also publish this model on the platform, making it available for other users to purchase.

[1694] An example of a prompt is, "Please create an AI learning model with a calm voice and a humorous personality. Please use a medium-low pitch and a calm tone."

[1695] In this way, the system allows AI learning models to be created, interacted with, published, and sold, with servers, terminals, and users each playing their respective roles.

[1696] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1697] Step 1:

[1698] Users access the platform and set their ideal vocal range and personality.

[1699] Specifically, the user uses a terminal to access the platform's URL and log in. Then, they go to the settings screen and input their voice pitch, tone, speaking speed, and personality traits.

[1700] Input: User's vocal and personality settings (e.g. voice pitch, mid-low, tone, calm, personality, humorous)

[1701] Output: The configuration data is sent to the server.

[1702] Step 2:

[1703] The server receives the user's settings and generates an AI learning model.

[1704] Specifically, the server analyzes the received configuration data, calls voice synthesis software (e.g., Google Text-to-Speech API) to generate voice data, and simultaneously generates personality data using a personality model generation algorithm (e.g., OpenAI's GPT-3). These data are then integrated to create an AI learning model.

[1705] Input: User's configuration data

[1706] Output: The generated AI learning model

[1707] Step 3:

[1708] The server stores the generated AI learning model on the platform and allows users to interact with it.

[1709] Specifically, the server stores the generated AI learning model in a database. When the user starts a dialogue, the server calls the model and generates a response according to the user's input.

[1710] Input: Generated AI learning model

[1711] Output: AI learning model stored on the platform

[1712] Step 4:

[1713] The user interacts with the AI ​​learning model.

[1714] Specifically, a user inputs text into a chat window through a terminal. The server receives the input text, uses the AI ​​learning model to generate an appropriate response, and displays it in the chat window.

[1715] Input: User text input (e.g. "What's the weather like today?")

[1716] Output: Response from the AI ​​learning model (e.g. "It's sunny today")

[1717] Step 5:

[1718] The server publishes the user's AI learning model on the platform.

[1719] Specifically, when a user clicks the "Publish" button, the server adds the model to a published list, from which other users can view the model.

[1720] Input: User's publishing instructions

[1721] Output: AI learning model added to the public list

[1722] Step 6:

[1723] Other users can purchase published AI learning models.

[1724] Specifically, other users select a model they are interested in from the public list and click the "Purchase" button. Once the purchase procedure is completed, the server grants the purchaser the right to use the model.

[1725] Input: Purchase instructions from other users

[1726] Output: The model usage rights granted to the buyer.

[1727] Step 7:

[1728] The server grants the buyer the rights to use the model.

[1729] Specifically, the server adds the model usage rights to the buyer's account, allowing the buyer to use the model through their device.

[1730] Input: Purchase procedure completion notification

[1731] Output: Model usage rights added to buyer's account

[1732] (Application example 1)

[1733] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1734] The shopping experience in conventional virtual stores had the problem that users needed a lot of time and effort to obtain product information. In addition, it was difficult for users to find products that matched their preferences and needs, and the shopping experience was often unsatisfactory. Furthermore, the conversational assistant was not customized based on the user's vocal cords or personality, so the conversation with the user was sometimes unnatural. To solve these problems, it is necessary to provide a conversational assistant using an AI learning model that mimics the user's ideal vocal cords and personality.

[1735] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1736] In this invention, the server includes a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with a user using the AI ​​learning model, a means for creating, purchasing, and selling the AI ​​learning model on a platform that can be made public, an interactive assistant means for supporting the user's shopping in a virtual store, a means for installing the interactive assistant means in a smartphone or a head-mounted display, and a means for the interactive assistant means to provide product information based on the user's ideal vocal cords and personality. This allows the user to improve their shopping experience in the virtual store through an interactive assistant based on their own ideal vocal cords and personality.

[1737] An "AI learning model for mimicking human vocal cords and personality" is an artificial intelligence learning model designed to reproduce the characteristics of a human voice, speaking style, and personality.

[1738] "Means of interacting with a user" refers to methods and technologies for achieving communication with a user.

[1739] A "publishable platform" is an online system that enables users to share, buy, or sell user-created content or data with other users.

[1740] A "virtual store" is a virtual store operated on the Internet where users can browse and purchase products online.

[1741] An "interactive assistant means" is a technique or method for providing information or support to a user through dialogue.

[1742] A "smartphone" is a highly functional mobile phone that can connect to the Internet and use applications.

[1743] A "head-mounted display" is a display device that provides visual information to a user when worn on the head.

[1744] "Means for providing product information" refers to methods or techniques for conveying detailed product information and features to users.

[1745] In order to implement the present invention, the following system configuration and program are required.

[1746] First, the server creates an AI learning model to mimic human vocal cords and personality. This AI learning model is generated based on the ideal vocal cords and personality set by the user. Specifically, the user sets their ideal vocal cords and personality on the platform and sends the setting information to the server. The server generates an AI learning model based on this information. The generated AI learning model is used to interact with the user.

[1747] The server then provides a means for interacting with the user using the generated AI learning model. The user can do this through a smartphone or a head-mounted display. The interactive assistant generates appropriate responses based on the user's input and plays them aloud using a speech synthesis engine.

[1748] In addition, the server provides a means to create, buy and sell AI learning models on a public platform where users can publish their ideal vocal range and personality, and other users can buy and sell the AI ​​learning models.

[1749] In the virtual store, the interactive assistant means assists the user in shopping. Specifically, when the user asks a question about a product, the interactive assistant means uses an AI learning model to provide appropriate product information. For example, if the user asks, "I'm looking for new running shoes. Do you have any recommendations?", the interactive assistant means responds, "How about these running shoes? They're lightweight and highly cushioned, making them ideal for long runs."

[1750] The following hardware and software are used to realize this system:

[1751] Hardware: Servers, smartphones, head-mounted displays

[1752] Software: OpenAI API, speech synthesis engine

[1753] Examples of specific prompts are as follows:

[1754] User: "I'm looking for new running shoes. Any recommendations?"

[1755] Assistant: "How about these running shoes? They're lightweight and well-cushioned, perfect for long runs."

[1756] In this way, users can enhance their shopping experience in virtual stores through a conversational assistant based on their ideal vocal range and personality.

[1757] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1758] Step 1:

[1759] Users access the platform and set their ideal vocal range and personality.

[1760] Input: User's vocal and personality settings

[1761] Output: Configuration information sent to the server

[1762] How it works: Through the platform's interface, the user inputs the tone of voice, speaking style, personality traits, etc. This configuration information is sent to the server.

[1763] Step 2:

[1764] The server generates an AI learning model based on the configuration information received.

[1765] Input: User's vocal and personality settings

[1766] Output: AI learning model

[1767] Specific operation: The server analyzes the received configuration information and uses it as data to generate an AI learning model. It uses the OpenAI API to generate an AI learning model based on the configuration information.

[1768] Step 3:

[1769] The generated AI learning model is used to prepare for interaction with the user.

[1770] Input: AI learning model

[1771] Output: Interaction ready state

[1772] Specific operation: The server integrates the generated AI learning model into the dialogue system, making it possible to dialogue with the user.

[1773] Step 4:

[1774] Users access the conversational assistant through a smartphone or head-mounted display.

[1775] Input: User's access request

[1776] Output: Launch of interactive assistant

[1777] Specific operation: A user starts an application on a smartphone or head-mounted display to access the conversational assistant. The application connects to the server and starts the conversational assistant.

[1778] Step 5:

[1779] The user inputs a question to the interactive assistant.

[1780] Input: User question

[1781] Output: The question data is sent to the server.

[1782] Specific operation: The user inputs a question to the conversational assistant by text or voice, and the input question is sent to the server.

[1783] Step 6:

[1784] The server generates a response based on the user's question.

[1785] Input: User question data

[1786] Output: Response data

[1787] Specific operation: The server analyzes the received question data and generates appropriate responses using an AI learning model. It uses the OpenAI API to generate natural responses to questions.

[1788] Step 7:

[1789] The generated response is converted into speech by a speech synthesis engine.

[1790] Input: Response data

[1791] Output: Audio data

[1792] Specific operation: The server passes the generated response data to the speech synthesis engine to generate voice data.

[1793] Step 8:

[1794] The audio data is sent to the user's terminal and played back.

[1795] Input: Audio data

[1796] Output: Audio response to the user

[1797] Specific operation: The server transmits the generated voice data to the user's smartphone or head-mounted display. The user's device plays the received voice data and provides the user with a response.

[1798] In this way, users can enhance their shopping experience in virtual stores through a conversational assistant based on their ideal vocal range and personality.

[1799] Example 2

[1800] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1801] Conventional AI learning models make it difficult to set vocal cords and personality according to individual user needs, and the generated models are not easy to publish or for other users to purchase. For this reason, there was a demand for a system that would allow users to easily create AI models that reflect their ideal vocal cords and personality, and share them with other users.

[1802] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for a user to set his / her ideal vocal cords and personality, a means for receiving the setting information, collecting voice data and text data, and performing preprocessing, a means for generating an AI learning model using the preprocessed data, a means for publishing the generated AI learning model on a platform where it can be published, and a means for other users to purchase the published AI learning model and use it in conversations. This makes it possible for a user to easily create an AI model that reflects his / her ideal vocal cords and personality, and share it with other users.

[1803] "User" refers to an individual or organization that uses the system to set their ideal vocal range and personality and generate and use an AI learning model.

[1804] "Vocal chords" refers to the voice characteristics such as tone, pitch, and speed of the voice that are set by the user.

[1805] "Personality" refers to the personality traits of the AI ​​learning model set by the user (e.g. kind, strict, humorous, etc.).

[1806] "Settings Information" refers to information regarding vocal chords and personality that a User inputs on the Platform.

[1807] "Voice data" refers to voice data collected based on a user's vocal cord settings.

[1808] "Text data" refers to text data collected based on a user's personality settings.

[1809] "Preprocessing" refers to the process of converting collected voice and text data into a format suitable for training an AI learning model.

[1810] "AI learning model" refers to an artificial intelligence model that is generated based on user settings information.

[1811] "Publication" refers to displaying the generated AI learning model on the platform so that other users can access it.

[1812] "Platform" refers to the online system through which Users can set their vocal range and personality, and generate, publish, purchase and use AI learning models.

[1813] "Purchase" refers to the transactional act of other users acquiring published AI learning models.

[1814] "Dialogue" refers to communication between a user and an AI learning model.

[1815] This invention is a system that allows users to set their ideal vocal cords and personality, and generate, publish, purchase, and use AI learning models based on that. A specific embodiment of this system is described below.

[1816] User configuration input

[1817] Users use the platform's web interface to configure their ideal vocal range and personality. Specifically, users select the tone, pitch, speed, and personality traits (e.g., kind, stern, humorous, etc.) of their voice. These configuration information is sent to the server as form input.

[1818] Data collection and preprocessing

[1819] The server receives the user's configuration information and collects the necessary data. It collects voice and text data and performs preprocessing. Preprocessing includes normalizing the voice data and tokenizing the text data. It uses Python and TensorFlow.

[1820] Generate AI learning models

[1821] The server uses the preprocessed data to generate an AI learning model, uses deep learning techniques to build a neural network using TensorFlow or PyTorch, and trains the model based on the user's settings.

[1822] Publishing the model

[1823] The server publishes the generated AI learning model on the platform. The model is uploaded to cloud storage (e.g. AWS S3) and a public URL is generated. This URL is displayed on the platform so that other users can access it.

[1824] Purchase and Use by Other Users

[1825] User B purchases the published AI learning model on the platform. The purchase procedure is done through a web interface, and payment is processed using a payment service. After the purchase, User B can use the AI ​​learning model in his or her own interactions.

[1826] Examples

[1827] Suppose User A sets "gentle voice, humorous personality" on the platform. The server receives this setting information, collects voice and text data, and performs preprocessing. Then, it uses TensorFlow to generate an AI learning model and uploads it to cloud storage. The published model is purchased by User B and used to interact with User B.

[1828] Examples of prompt statements

[1829] "Generate an AI model with a gentle voice and a humorous personality."

[1830] This system provides a series of processes for users to set their ideal vocal range and personality, and then generate, publish, purchase, and use AI learning models based on that, allowing users to easily create AI models that meet their needs and share them with other users.

[1831] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1832] Step 1: Enter user settings

[1833] A user configures his / her ideal vocal range and personality using a web interface on the platform. Specifically, the user selects the tone, pitch, speed, and personality traits (e.g., kind, stern, humorous, etc.) of the voice. These configuration information are sent to the server as form input. The input is the user's configuration information, and the output is the configuration information sent to the server.

[1834] Specific behavior:

[1835] A user visits a web page and fills out a configuration form.

[1836] The user clicks the "Submit" button.

[1837] The server receives the configuration information and stores it in a database.

[1838] Step 2: Collect and preprocess data

[1839] The server receives the user's configuration information and collects the necessary data. It collects voice and text data and performs preprocessing. Preprocessing includes normalizing the voice data and tokenizing the text data. It uses Python and TensorFlow. The input is the user's configuration information and the output is the preprocessed data.

[1840] Specific behavior:

[1841] The server collects the voice data based on the setting information.

[1842] The server normalizes the audio data and tokenizes the text data.

[1843] The server temporarily stores the pre-processed data.

[1844] Step 3: Generate an AI learning model

[1845] The server uses the preprocessed data to generate an AI learning model. It uses deep learning techniques to build a neural network using TensorFlow or PyTorch. It trains the model based on the user's settings. The input is the preprocessed data, and the output is the trained AI learning model.

[1846] Specific behavior:

[1847] The server uses TensorFlow to build a neural network.

[1848] The server trains the model using the preprocessed data.

[1849] The server stores the trained model.

[1850] Step 4: Publish the model

[1851] The server publishes the generated AI learning model on the platform. The model is uploaded to cloud storage (e.g. AWS S3) and a public URL is generated. This URL is displayed on the platform so that other users can access it. The input is the trained AI learning model, and the output is the public URL.

[1852] Specific behavior:

[1853] The server uploads the trained model to cloud storage.

[1854] The server generates a public URL.

[1855] The server displays the URL on the platform.

[1856] Step 5: Purchase and use by others

[1857] User B purchases the published AI learning model on the platform. The purchase procedure is done through a web interface, and payment is processed using a payment service. After the purchase, User B can use the AI ​​learning model in his / her own interactions. The input is the publication URL and User B's purchase information, and the output is an AI learning model that User B can use.

[1858] Specific behavior:

[1859] User B selects a model published on the platform.

[1860] User B completes the purchase procedure.

[1861] The server processes the payment through a payment service.

[1862] User B downloads the purchased model and uses it in the interaction.

[1863] (Application example 2)

[1864] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1865] With conventional voice synthesis technology, it was difficult for users to generate personalized voice content that reflected their ideal vocal range and personality. In addition, there were limited ways to purchase personalized voice models created by other users and enjoy content in their voices. This meant that it was not possible to meet the diverse needs of users, and there was an issue of the individualization of voice content not progressing.

[1866] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1867] In this invention, the server includes a means for creating an AI learning model for imitating human vocal cords and personality, a means for interacting with a user using the AI ​​learning model, a means for creating, purchasing, and selling the AI ​​learning model on a public platform, a means for a user to set his / her ideal vocal cords and personality and generate an AI learning model based on the setting, a means for providing audio content in a personalized voice using the generated AI learning model, and a means for purchasing an AI learning model created by another user and enjoying content in that voice. This allows a user to generate personalized voice content that reflects his / her ideal vocal cords and personality, and to enjoy a variety of voice content using voice models created by other users.

[1868] The "human vocal cords" are the biological structures used by humans to produce sound and are what determine the pitch and quality of the voice.

[1869] "Personality" refers to an individual's characteristics in behavior, thinking, and emotion, and reflects that person's characteristics in dialogue and communication.

[1870] An "AI learning model" is an algorithm or network structure that uses artificial intelligence technology to learn specific data and patterns and perform tasks such as prediction, classification, and generation.

[1871] A "production means" is a method or device for producing or building something for a particular purpose.

[1872] A "means for interacting" is a method or device by which a user and a system exchange information and communicate with each other.

[1873] A "publishable platform" is an online system that enables users to share and make accessible their created content and models to other users.

[1874] A "buying or selling vehicle" is a transactional method or device by which a user obtains or provides particular content or models.

[1875] A "setting means" is a method or device that allows a user to input his / her preferences and requirements into the system so that the system operates based on them.

[1876] "Audio content" means any form of information or entertainment that is primarily audio, including music, podcasts and audiobooks.

[1877] A "personalized voice" is a voice that is customized based on the preferences and requirements of a particular user, and reflects that user's ideal vocal range and personality.

[1878] An "AI learning model created by another user" is an artificial intelligence model that a different user has generated based on their own settings and that can be used b...

Claims

1. means for pre-processing the data based on voice characteristics and personality characteristics set by a user; means for training an AI learning model using the preprocessed data; A means for publishing the trained AI learning model on a publicly accessible platform; A means for other users to purchase the published AI learning model; means for analyzing the emotional state of said other users in real time using an emotion engine; A means for adjusting a dialogue response of the AI ​​learning model based on the analysis result of the emotional state of the other user; A system including:

2. The adjusting means generates a dialogue response to the other user using the AI ​​learning model based on a prompt sentence corresponding to the emotion of the other user. The system of claim 1 .

3. The data is collected voice data and text data. The system of claim 1 .

Images