Avatar generation system, avatar generation method, and program

Abstract

To easily create an avatar in accordance with attributes of a dialogue partner.SOLUTION: An avatar generation system according to an aspect of the present disclosure includes: a receiving unit for receiving persona information indicating a persona that characterizes a dialogue partner of a user; a persona generation unit for generating a persona generation prompt indicating features of the persona by using a large-scale language model on the basis of the persona information received by the receiving unit; a storage unit for storing a plurality of element data indicating components of an avatar as the dialogue partner; an avatar generation unit for selecting the element data stored in the storage unit for the persona generation prompt generated by the persona generation unit, and generating an avatar by using the selected element data; and an output unit for outputting an avatar corresponding to the persona on the basis of the persona generation prompt generated by the persona generation unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an avatar generation system, an avatar generation method, and a program. [Background technology]

[0002] Conventionally, there is known a technology for generating a virtual character called an avatar, which virtually represents a person, in a virtual space. For example, a virtual pseudo-human image generation system described in Patent Document 1 stores a motion pattern of a virtual pseudo-human image model and model data of the virtual pseudo-human image model, and generates a moving virtual pseudo-human image by applying the motion pattern to the model data. In this virtual pseudo-human image generation system, an idling motion pattern for giving the virtual pseudo-human image model an idling motion with slight head and body movements is stored, and when the virtual pseudo-human image model to be generated does not move for a certain period of time, the idling motion pattern is read out and a virtual pseudo-human image model with an idling motion is generated. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 3153141 Summary of the Invention [Problem to be solved by the invention]

[0004] However, although the above-described virtual pseudo-human image generation system generates a virtual pseudo-human image model using the movement patterns and model data of the virtual pseudo-human image model, it is not possible to create an avatar that corresponds to the attributes of the person with whom the conversation is taking place.

[0005] The present disclosure has been made in consideration of the above circumstances, and aims to provide an avatar generation system, an avatar generation method, and a program that enable users to easily create an avatar that matches the attributes of a conversation partner. [Means for solving the problem]

[0006] The present disclosure has been made to solve the above-mentioned problems, and one aspect of the present disclosure is an avatar generation system comprising: a reception unit that receives persona information indicating a persona that characterizes a user's dialogue partner; a persona generation unit that generates a persona generation prompt that indicates characteristics of the persona based on the persona information received by the reception unit using a large-scale language model; a memory unit that stores a plurality of element data that indicate components of an avatar that will be the dialogue partner; an avatar generation unit that selects the element data stored in the memory unit for the persona generation prompt generated by the persona generation unit and generates an avatar using the selected element data; and an output unit that outputs an avatar corresponding to the persona based on the persona generation prompt generated by the persona generation unit.

[0007] Another aspect of the present disclosure is an avatar generation method including the steps of: an avatar generation system storing in a storage unit a plurality of element data indicating components of an avatar as a user's conversation partner; the avatar generation system receiving persona information indicating a persona that characterizes the user's conversation partner; the avatar generation system generating a persona generation prompt indicating characteristics of the persona based on the received persona information using a large-scale language model; the avatar generation system selecting the element data stored in the storage unit for the generated persona generation prompt and generating an avatar using the selected element data; and the avatar generation system outputting an avatar corresponding to the persona based on the generated persona generation prompt.

[0008] Another aspect of the present disclosure is a program that causes a computer of an avatar generation system to execute the following steps: the avatar generation system stores in a memory unit a plurality of element data indicating components of an avatar as a user's conversation partner; the avatar generation system receives persona information indicating a persona that characterizes the user's conversation partner; the avatar generation system generates a persona generation prompt indicating characteristics of the persona using a large-scale language model based on the received persona information; the avatar generation system selects the element data stored in the memory unit for the generated persona generation prompt and generates an avatar using the selected element data; and the avatar generation system outputs an avatar corresponding to the persona based on the generated persona generation prompt. [Effects of the Invention]

[0009] According to one aspect of the present invention, an avatar can be easily created according to the attributes of a conversation partner. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing an example of the configuration of a dialogue support system 1 according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an outline of processing performed by the dialogue support system 1 according to the embodiment. [Figure 3] 4 is a flowchart showing an example of a processing procedure of the dialogue support system 1 according to the embodiment. [Figure 4] 10 is a diagram illustrating an example of customer-defined information including a variable name and a character string according to an embodiment. FIG. [Figure 5] FIG. 10 is a diagram illustrating an example of a persona generation prompt according to an embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of variables created from a persona generation prompt in the embodiment. [Figure 7]FIG. 10 is a diagram showing an example of preset data included in element data in the embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of a persona designation prompt according to an embodiment. [Figure 9] FIG. 10 is a diagram showing an example of persona designation information in the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] An avatar generation system, an avatar generation method, and a program to which the present invention is applied will be described below with reference to the drawings.

[0012] FIG. 1 is a block diagram showing an example of the configuration of a dialogue support system 1 according to an embodiment. A dialogue support system 1 according to an embodiment supports a dialogue between a user and an avatar characterized by a specific persona. The dialogue support system 1 generates a persona based on information specified by the user, and controls an avatar corresponding to the generated persona, allowing the user and the avatar to engage in role-playing. Role-playing is, for example, training new employees, sales training, language training, communication training, etc., with the user and a pseudo-avatar as conversation partners. Role-playing according to an embodiment also includes those between people of different nationalities or places of origin.

[0013] The dialogue support system 1 includes, for example, a processing server device 100, a generation server device 200, and a user terminal device 300. The processing server device 100, the generation server device 200, and the user terminal device 300 are communicatively connected via a network NW such as the Internet. The processing server device 100, the generation server device 200, and the user terminal device 300 may be connected via either wired or wireless communication, and may include a general-purpose network such as the Internet, or a private network such as local 5G or WiFi (registered trademark). The processing server device 100, the generation server device 200, and the user terminal device 300 may have a communication interface, such as a network interface card (NIC) or a wireless communication module for connecting to a network, to exchange information with each other.

[0014] The user terminal device 300 is, for example, an information processing device operated by a user who interacts with an avatar. The user terminal device 300 includes, for example, a speaker, a microphone, a display device, an operation unit, and a processing unit such as a CPU.

[0015] The processing server device 100 is, for example, a server device that includes a processor that performs processing in response to requests received from the generation server device 200 and the user terminal device 300 and transmits processing results to the generation server device 200 and the user terminal device 300. The processing server device 100 includes, for example, a customer generation unit 110, a dialogue control unit 120, an operation control unit 130, and a storage unit 140. The customer generation unit 110, the dialogue control unit 120, and the operation control unit 130 are functional units realized by, for example, an information processing circuit that performs various processes by causing a central processing unit (CPU) to execute a program. Furthermore, some or all of these functional units may be realized by hardware such as a large-scale integration (LSI), an application-specific integrated circuit (ASIC), or a field-programmable gate array (FPGA), or may be realized by a combination of software and hardware. The storage unit 140 is realized by, for example, a hard disk drive (HDD), a solid state drive (SSD), a flash memory, an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), or a random access memory (RAM), or a hybrid storage device using a combination of these. Part or all of the storage unit 140 may be realized by an external storage device accessible via various networks. An example of an external storage device is a network attached storage (NAS) device.

[0016] The customer generation unit 110 generates customer information. The customer information is information indicating a customer envisioned by the user. The customer corresponds to, for example, an avatar that will be the user's conversation partner in a role-play. The customer generation unit 110 includes, for example, a reception unit 111 and a customer definition unit 112. The reception unit 111 receives persona information based on information received from the user terminal device 300. The persona information is customer definition information indicating a persona that characterizes the customer (conversation partner) for the user. The persona may be a virtual character image, or a character image based on information about a real person. Furthermore, the persona information may be information based in part on information about a real person, or information about a person who has already passed away. The customer definition unit 112 generates customer information based on the persona information received by the reception unit 111.

[0017] The dialogue control unit 120 controls an avatar corresponding to a persona based on the persona generation prompt generated by the persona generation unit 211, and performs processing to control a dialogue between the avatar and a user. The dialogue control unit 120 includes, for example, an utterance acquisition unit 121, an emotion parameter processing unit 122, a response prompt generation unit 123, a response text conversion unit 124, and a conversation history generation unit 125. The utterance acquisition unit 121 acquires utterance information indicating the user's utterance input from the user terminal device 300, and converts the acquired utterance information into text data. Emotion parameter processing unit 122 performs processing to set and update emotion parameters. Emotion parameters are numerical values ​​that indicate the emotion of an avatar (customer). Emotion parameters are information that expresses emotions such as joy, anger, sadness, enjoyment, confidence, confusion, and fear on a scale of 1 to 5. Note that, although the present embodiment describes a configuration related to the emotion of the customer, such as emotion parameter processing unit 122, the present invention is not limited to this, and the configuration related to the emotion of the customer may not be provided. The response prompt generation unit 123 generates a response prompt including text data of the user's voice and emotion parameters, and transmits the generated response prompt to the generation server device 200. The response text conversion unit 124 converts the response text acquired from the generation server device 200 into voice data. The conversation history generating unit 125 generates history information indicating the history of conversations between the user and the avatar.

[0018] The movement control unit 130 performs processing to control the movement of the avatar. The movement control unit 130 includes, for example, an avatar generation unit 131, a voice generation unit 132, a voice information processing unit 133, a motion processing unit 134, an emote processing unit 135, and a lip sync processing unit 136. The avatar generation unit 131 selects element data stored in the storage unit 140 in response to the persona generation prompt generated by the persona generation unit 211, and generates an avatar using the selected element data. The element data for generating an avatar is, for example, image data representing the avatar's face, body, and other basic features corresponding to the age, sex, nationality, or place of origin. The element data for generating an avatar may include image data representing clothing.

[0019] The voice generating unit 132 generates voice data to be output to the user based on the element data stored in the storage unit 140. The voice generating unit 132 generates voice data that reproduces the actual voice of the customer reading out loud, for example. The element data stored in the storage unit 140 may include element data for generating a sound. The element data for generating a voice may be synthetic voice data corresponding to, for example, age, gender, nationality, or place of origin. The element data for generating a voice may be synthetic voice data corresponding to elements including, for example, speaking style (e.g., catchphrases, backchannels, dialects), tone of voice (e.g., speaking speed), voice pitch, or tone of voice. The speaking style and tone of voice may reflect the general speaking style and culture specific to a specific country or region. Differences in speaking style may be due, for example, to differences in the number of vowels used by each country or region.

[0020] In an embodiment, the persona information may include nationality or place of origin. The persona generator 211 may input existing items including the persona's nationality or place of origin as persona information into a large-scale language model, and create a persona generation prompt based on the output of the large-scale language model. The element data may include element data for generating a voice, and the element data for generating a voice may include synthesized voice data corresponding to elements including a speaking style or tone corresponding to the nationality or place of origin of the persona. This allows the voice generation unit 132 to generate voice data in a language corresponding to the nationality or place of origin based on the persona generation prompt and persona information generated by the persona generation unit 211. This allows the voice generation unit 132 to control the voice (tone, pitch, tone, etc.) using synthetic voice data that corresponds to the speaking style and tone.

[0021] The element data for generating the voice may include a plurality of element data corresponding to each of a plurality of languages. The voice generation unit 132 can select one of the plurality of element data corresponding to each of a plurality of languages ​​stored in the storage unit 140 based on the nationality or place of origin included in the persona information. As a result, the voice generation unit 132 generates voice data using synthesized voice data corresponding to each of the multiple languages. Furthermore, the voice generation unit 132 can control the voice to reflect the general speaking style and culture specific to a country or region where the nationality or place of origin of the user and the conversation partner differ. For example, the voice generation unit 132 may generate voice data by changing the voice to a voice specific to the country or region of the conversation partner. Furthermore, in the dialogue support system 1, speech may be instantly translated automatically into the language of a country or region different from the user's nationality or place of origin, and the speech generator 132 may generate speech data. Furthermore, the dialogue support system 1 may generate voice data by the voice generation unit 132 so that voices that reflect the general speaking style and culture specific to a specific country or region are spoken or responded to in a language selected from multiple languages.

[0022] The voice tone information processing unit 133 processes the voice data based on the element data, emotion parameters, or voice tone information corresponding to the content of the response text stored in the storage unit 140.

[0023] The motion processing unit 134 controls the motion of the avatar based on the element data, emotion parameters, or response text stored in the storage unit 140. The avatar motion represents, for example, the movement of the entire avatar or the movement of the avatar's hands. The element data may include element data for generating the motion. The element data for generating the motion is, for example, avatar images corresponding to various gestures. The various gestures include, for example, youthful gestures, arrogant gestures, etc. The motion processing unit 134 may control the avatar motion so that it reflects general gestures, hand movements, and culture specific to a specific country or region. The element data for generating the motion may represent the avatar motion corresponding to the nationality or place of origin, and the persona information may include the nationality or place of origin. The motion processing unit 134 can generate the avatar motion corresponding to the nationality or place of origin based on the persona generation prompt and persona information generated by the persona generation unit 211.

[0024] The emote processing unit 135 controls the facial expression of the avatar based on the element data, emotion parameters, and response text content stored in the storage unit 140. The emote processing unit 135 controls, for example, the movement of the avatar's eyes, eyebrows, mouth, etc. The element data for generating emotes is, for example, avatar images corresponding to various facial expressions of the avatar. The various facial expressions include, for example, a youthful expression, a calm expression, etc. Lip sync processor 136 controls the movement of the avatar's lips based on the emotion parameters and the content of the response text. The operation control unit 130 functions as an output unit that outputs an avatar corresponding to a persona based on the persona generation prompt generated by the persona generation unit 211.

[0025] The storage unit 140 stores, for example, customer information 141, response information 142, voice information 143, and action information 144. The customer information 141 includes, for example, persona information, utterance information, a persona generation prompt, and a persona designation prompt. The persona generation prompt is detailed information for generating a persona. The persona designation prompt is information indicating a persona that is designated when a user and an avatar actually have a dialogue, such as a role-play. The response information 142 includes, for example, a user voice text and a response text, and may also include an initial emotional parameter value and a current emotional parameter value. The voice information 143 includes, for example, voice data such as a user voice and a response voice, and tone information, and may also include an emotional parameter. The motion information 144 includes pool data containing a large number of element data. The element data includes element data related to the face or body, element data related to the voice, and element data related to the movement. The element data may also include element data related to clothing and element data related to facial expressions.

[0026] The element data of the avatar may include explanatory text information. The explanatory text information is text data for explaining the face and facial expressions of the avatar. The avatar generation unit 131 compares the persona generation prompt with the explanatory text information and selects or generates image information of an avatar for the persona based on the comparison result. The avatar generation unit 131 selects element data corresponding to the explanatory text as the degree of match between the persona generation prompt and the explanatory text information increases.

[0027] The generation server device 200 is, for example, a server device that performs processing in response to a request received from the processing server device 100 and transmits the processing results. The generation server device 200 includes, for example, a generation unit 210, a storage unit 220, and an LLM learning unit 240. The generation unit 210 and the LLM learning unit 240 are functional units realized by, for example, an information processing circuit that performs various processes by causing a CPU to execute a program. The storage unit 220 is realized by, for example, a recording device such as an HDD or SSD, or a hybrid storage device using a plurality of these, and may also be realized by an external storage device accessible via various networks, such as a NAS device.

[0028] The generation unit 210 includes, for example, a persona generation unit 211, a response text generation unit 212, and a unique information acquisition unit 214. The persona generation unit 211 inputs persona information acquired from the processing server device 100 into a first large-scale language model and generates a persona generation prompt based on the output of the first large-scale language model. The persona generation unit 211 may input persona information and information about a specific field into the first large-scale language model and create a persona generation prompt indicating the characteristics of a persona corresponding to the specific field based on the output of the first large-scale language model. The information about the specific field is various information about a field that becomes a topic of conversation. The information about the specific field is, for example, customer characteristic information such as customer issues related to product purchases that are empirically assumed for a specific industry, a specific generation, a specific nationality, or a specific place of origin. The information about the specific field is acquired as unique information by the unique information acquisition unit 214. The response text generation unit 212 generates a response text from the response prompt generated by the response prompt generation unit 123, the conversation history generated by the conversation history generation unit 125, and the unique information acquired by the unique information acquisition unit 214. The response text generation unit 212, for example, inputs the response prompt, the conversation history between the user and the avatar, and the unique information into a second large-scale language model, and generates a response text based on the second large-scale language model. Note that the response text generation unit 212 may extract contextual information of the conversation and generate a response text based on the contextual information in addition to the response prompt, the conversation history, and the unique information. The first large-scale language model is, for example, a large-scale language model (LLM) using a neural network. The second large-scale language model may be the same LLM as the first large-scale language model, or they may be different LLMs. Emotion parameter generation section 213 generates or updates emotion parameters according to the content of the generated response text. The unique information acquisition unit 214 acquires unique information that is information unique to a dialogue such as a role play. The unique information is acquired, for example, from a storage device that stores customer characteristic information, specific field information, specific industry information, specific generation information, specific country information, or specific region information (not shown).

[0029] The storage unit 220 stores, for example, unique information 221, element data 222, and LLM information 223. The unique information 221 includes customer characteristic information, specific field information, specific industry information, specific generation information, specific country information, and specific region (including within Japan) information. Customer characteristic information is information indicating the characteristics of the customer who is interacting with the user. Customer characteristic information is information such as age, gender, occupation, speaking style, personality, nationality, and place of origin. Specific field information is information indicating the field of interaction between the user and the customer. Specific industry information is information indicating the industry of interaction between the user and the customer. Specific generation information is information indicating the generation of the customer. Specific country information is information indicating the country to which the customer belongs (nationality, country of residence) or the country of origin of the customer. Specific region (including within Japan) information is information indicating the region to which the customer belongs or the region from which the customer is born, and a region may be a part of a specific country or a collective term for multiple specific countries. The element data 222 is element data to be stored in the processing server device 100. The element data 222 stored in the storage unit 220 may be transmitted to the processing server device 100 together with a persona generation prompt, together with response text information, or together with unique information, for example. The LLM information 223 is parameter information of an LLM (first large-scale language model) for generating a persona-generating prompt. The LLM information 223 may include parameter information of an LLM for generating a persona-specific prompt based on the persona-generating prompt. The LLM information 223 may include parameter information of an LLM (second large-scale language model) for generating response text based on the persona-specific prompt. The LLM information 223 may include parameter information of an LLM for generating an evaluation prompt based on a conversation history. Note that the LLM for generating the persona-generating prompt, the LLM for generating the persona-specific prompt, the LLM for generating response text, and the LLM for generating the evaluation prompt may be a single LLM or may be different LLMs.

[0030] The LLM training unit 240 performs a process of training an LLM (first large-scale language model) for generating persona-generating prompts and an LLM (second language model) for generating response texts. The LLM training unit 240 may also train an LLM for generating persona-specifying prompts and an LLM for generating evaluation prompts.

[0031] 1, the dialogue support system 1 in the embodiment has its functional configuration (functional units) distributed between the processing server device 100 and the generation server device 200, but is not limited to this and the functional units may be distributed in other configurations, the functional units of the processing server device 100 and the generation server device 200 may be integrated into one device, multiple functional units may be combined into one functional unit, or one function may be distributed among multiple functional units. The reception unit 111, the persona generation unit 211, the memory unit 220 or 140, the avatar generation unit 131, and the function of outputting an avatar by the operation control unit 130 correspond to an avatar generation system.

[0032] FIG. 2 is a diagram illustrating an outline of the processing performed by the dialogue support system 1 according to the embodiment. The reception unit 111 and the customer definition unit 112 generate customer definition information D10 and transmit it to the generation server device 200. The persona generation unit 211 inputs the customer definition information D10 into a persona generation LLM (P10) and generates a persona generation prompt D12 based on the output of the persona generation LLM (P10). The avatar generation unit 131 performs an avatar generation process (P12) based on the persona generation prompt D12 and element data D16. The persona generation unit 211 generates a persona specification prompt D14 when the user performs role-playing. The persona specification prompt D14 is output to the response text generation LLM.

[0033] The generation server device 200 acquires unique information D20, such as a specific field for role-playing, and processes the unique information D20 in the following order: text extraction processing P20, chunk division processing P21, and vectorization processing P22. The processing server device 100 then performs speech recognition processing P40 on the user's speech acquired from the user terminal device 300, and vectorization processing P41 on the text information processed by the speech recognition processing P40. A reference result to the vector database is extracted from the vector database using the vector corresponding to the speech as a query. The vectorized unique information D20 and the text information processed by the speech recognition processing P40 are output to the response text generation LLM (P30) along with a persona-specifying prompt D14.

[0034] The generation server device 200 inputs the persona-specifying prompt D14, unique information D20, and text information processed by the speech recognition process P40 into the response text generation LLM (P30), performs speech synthesis process P31 on the response text output from the response text generation LLM (P30), and performs avatar control process P32 based on the emotion parameters output from the response text generation LLM (P30) and the avatar generated by the avatar generation process (P12), thereby transmitting avatar content D30 to the user terminal device 300. This allows the user terminal device 300 to display or output sound using avatar content D30.

[0035] FIG. 3 is a flowchart showing an example of a processing procedure of the dialogue support system 1 according to the embodiment. First, the processing server device 100 inputs user information about the user who will be interacting with the avatar (step S100). The user information is, for example, character string data that characterizes the user, such as a new employee, a sales manager, a specific nationality, or a place of origin. Next, the processing server device 100 defines the customer that the user is considering (step S102). At this time, the processing server device 100 receives persona information indicating a persona that characterizes the user's interaction partner from the user terminal device 300 via the reception unit 111, and stores the persona information as a variable, for example, as shown in FIG. 4. 4 is a diagram showing an example of customer-defined information including a variable name and a character string in an embodiment. The receiving unit 111 determines whether there is further input (step S104), and if there is input, repeats the processing of step S102, or if there is no input, confirms the customer-defined information. The processing server device 100 transmits the customer-defined information to the generation server device 200.

[0036] The persona generation unit 211 inputs the customer-defined information and the unique information stored in the storage unit 220 into the persona generation LLM, and generates a persona generation prompt based on the output of the persona generation LLM (step S106). FIG. 5 is a diagram illustrating an example of a persona generation prompt according to an embodiment. The persona generation prompt includes, for example, text data indicating the customer's prerequisites, the customer's speaking style, and the customer's personality traits. The customer prerequisites include, for example, items corresponding to role-playing, such as age, gender, occupation, family structure, residential area, nationality, birthplace, and insurance information. If the customer prerequisites include, for example, a birthplace of Osaka, a prompt corresponding to the Kansai dialect can be created as a persona generation prompt. It is also possible to control the tone of voice or dialect depending on the residential area. Furthermore, by setting, for example, a birthplace of California as a prerequisite for the customer, a persona generation prompt can be created that can recreate a person who is strongly influenced by California culture. The customer's speaking style may be, for example, first person, second person, catchphrases, dialect, etc. The customer's personality traits may be, for example, neuroticism, extroversion, openness, conscientiousness, cooperativeness, etc. The persona generation unit 211 determines whether or not there is input for other items from the user terminal device 300 (step S108), and if input is received, repeats the processing of step S106, or if no input is received, confirms the persona generation prompt.

[0037] The persona generation unit 211 stores the created persona generation prompt in the storage unit 220 as a variable. FIG. 6 is a diagram illustrating an example of variables generated from a persona generation prompt according to an embodiment. The variables generated from the persona generation prompt are information predicted based on the output of the persona generation LLM after inputting customer-defined information into the persona generation LLM. The variables generated from the persona generation prompt include variables corresponding to customer-defined information such as age, gender, nationality, place of origin (including within Japan), speaking style, tone, and dialect; variables predicted from customer-defined information such as occupation and family structure; and element data variables corresponding to the variables predicted from the customer-defined information. The element data variables include, for example, a variable indicating which preset data among the avatar element data, a variable indicating which preset data among the synthetic voice element data, a variable indicating which preset data among the motion element data, and a variable indicating which preset data among the emote element data. 7 is a diagram showing an example of preset data included in element data in the embodiment. For example, multiple pieces of preset data are set for each of an avatar pool, a synthetic voice pool, a motion pool, and an emote pool.

[0038] The persona generation prompt is transmitted from the generation server device 200 to the processing server device 100. The avatar generation unit 131 selects preset data of element data for the variables created from the persona generation prompt, and generates an avatar using the selected preset data (step S110). The avatar generation unit 131 may input a persona generation prompt and multiple pieces of element data into a machine learning model trained with data about a real person as training data, and generate an avatar that imitates a real person based on the output from the machine learning model. The machine learning model may, for example, be trained with data about a real person associated with the persona generation prompt and multiple pieces of element data as training data. As a result, when data about a real person is input, the machine learning model can output a persona generation prompt and multiple pieces of element data corresponding to the data about the real person as training data if the input data matches the data about the real person as training data. As a result, the avatar generation unit 131 can generate an avatar that imitates a real person by generating an avatar based on the persona generation prompt and multiple pieces of element data output from the machine learning model.

[0039] The avatar generation unit 131 may construct a virtual three-dimensional space and generate and move an avatar within the three-dimensional space based on the persona generation prompt. For example, when an avatar movement pattern is set as element data of the avatar, the avatar generation unit 131 may select element data of the avatar movement pattern based on the avatar movement pattern in the persona generation prompt, and move the avatar within the three-dimensional space based on the selected element data. The avatar generation unit 131 may generate an avatar using a technology known as deep fake. The avatar generation unit 131 may also use a technology known as 3DCG (Three-dimensional Computer Graphics) to construct a three-dimensional space by computer processing, and generate and move an avatar within the three-dimensional space.

[0040] Next, the persona generation unit 211 creates a persona specification prompt (step S112). FIG. 8 is a diagram showing an example of a persona specification prompt in an embodiment. The persona generation unit 211 may input a variable corresponding to the persona generation prompt into the LLM and create the persona specification prompt based on the output of the LLM. The variable corresponding to the persona generation prompt may be a variable selected based on a user operation, or may be a variable extracted from the persona generation prompt randomly or according to a predetermined rule. The persona generation unit 211 transmits persona specification information such as that shown in FIG. 8 to the processing server device 100 based on the persona specification prompt. FIG. 9 is a diagram showing an example of persona specification information in an embodiment.

[0041] Next, the dialogue control unit 120 controls the dialogue between the avatar and the user (step S114). At this time, the movement control unit 130 reads element data based on the variables of the element data and controls the avatar corresponding to the persona. The element data is stored as pool data, as shown in FIG. 7, for example. The movement control unit 130 reads data corresponding to the variables from, for example, the avatar pool data, the synthetic voice pool data, the motion pool data, and the emote pool. The movement control unit 130 reads image data representing various avatar body and clothing from the avatar pool data, reads element data for generating various avatar voices from the synthetic voice pool data, reads element data for generating motions corresponding to various facial expressions and emotions from the motion pool data, and reads element data for generating emotes corresponding to various facial expressions and emotions from the emote pool data, and controls the avatar using the multiple types of element data read. The movement information 144 may include other information for operating the avatar. The other information may be a default value for controlling the avatar's motion, a default value for controlling the avatar's emotes, or a default value for controlling the avatar's lip-sync. Thus, the dialogue control unit 120 performs role-playing through dialogue between the user and the avatar. The utterance acquisition unit 121 acquires utterance information indicating the user's utterances, and the conversation history generation unit 125 stores the conversation history (step S116).

[0042] Next, the processing server device 100 determines whether to evaluate the user (step S118). If the processing server device 100 does not evaluate the user, it repeats the processes of steps S114 and S116. For example, when the processing server device 100 detects a user utterance such as "evaluate the role-play," the processing server device 100 determines to evaluate the user and transmits the conversation history to the generation server device 200. The generation server device 200 evaluates the user based on the utterance information acquired by the utterance acquisition unit 121 (step S120). The generation server device 200 generates an evaluation prompt based on the conversation history acquired from the processing server device 100. The generation server device 200 may input the conversation history to an LLM that generates an evaluation prompt, and generate the evaluation prompt based on the output of the LLM. For example, when a user explains a product in a role-play, the generation server device 200 may input the conversation history and information stored in a product information database to an LLM that generates an evaluation prompt, and generate the evaluation prompt based on the output of the LLM that generates the evaluation prompt. The generation server device 200 outputs evaluation information to the processing server device 100 as a result of evaluating the user's conversation based on the evaluation prompt (step S122). As a result, the processing server device 100 transmits the evaluation information to the user terminal device 300, and the user terminal device 300 can present the evaluation to the user.

[0043] The timing for evaluating the role-play may be at the end of the role-play (e.g., business negotiation), where the evaluation is performed as a whole. However, the evaluation may be performed for each conversation rally (each round of conversation) during the role-play. The processing server device 100 prompts the user to select either a comprehensive evaluation (end evaluation) or an in-game evaluation, for example, at the start of the role-play. The comprehensive evaluation is a process of evaluating the entire conversation history, while the in-game evaluation is a process of evaluating each conversation rally (including a pair of a receiver and a reply) during the conversation. When the comprehensive evaluation is selected, the processing server device 100 transmits the conversation history to the generation server device 200 upon detecting, for example, a user utterance such as "evaluate the role-play." When the in-game evaluation is selected, the processing server device 100 transmits the conversation history for one rally to the generation server device 200 upon detecting a break in the conversation rally. The generation server device 200 stores the results of the in-game evaluation based on the conversation history for one rally in the storage unit 220, and transmits one or more in-game evaluation results to the processing server device 100 upon the end of the role-play. This allows the user to be presented with the results of the evaluation each time.

[0044] Furthermore, both evaluation of the entire role-play (such as a business negotiation) and evaluation of each conversation rally (each round of conversation) during the role-play may be performed simultaneously. For example, at the start of a role-play, the processing server device 100 allows the user to select whether to perform both an overall evaluation and an individual evaluation. The processing server device 100 transmits a conversation history for one rally to the generation server device 200 upon detecting a break in a conversation, and further transmits the conversation history to the generation server device 200 upon detecting a user utterance of "evaluate the role-play." The generation server device 200 stores the results of the individual evaluation based on the conversation history for one rally in the storage unit 220. When the role-play ends, the generation server device 200 transmits the results of the overall evaluation and one or more individual evaluations to the processing server device 100. This allows the overall evaluation result and the individual evaluation results to be presented to the user on the same screen.

[0045] As described above, the avatar generation system according to the embodiment stores a plurality of element data indicating the components of an avatar as a conversation partner, receives persona information indicating a persona characterizing the user's conversation partner, creates a persona generation prompt indicating the characteristics of the persona based on the persona information, selects element data corresponding to the persona generation prompt, and generates an avatar using the selected element data. The avatar generation system makes it possible to easily create an avatar according to the attributes of the conversation partner. The avatar generation system makes it possible to easily increase the variety of attributes of conversation partners for role-playing, for example, and realize role-playing according to the attributes of the conversation partner.

[0046] The functions of the processing server device 100, the generation server device 200, and the user terminal device 300 in the above-described embodiments may be implemented by a computer. In this case, a program for implementing the functions may be recorded on a computer-readable recording medium, and the program may be loaded into a computer system and executed. Note that the term "computer system" as used herein includes hardware such as an operating system and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into a computer system. Furthermore, the term "computer-readable recording medium" may also include devices that dynamically store programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or telephone lines, or devices that store programs for a fixed period of time, such as volatile memory within the computer system serving as the server or client. The program may be designed to implement only a portion of the above-described functions, or may be capable of implementing the above-described functions in combination with a program already stored in the computer system, or may be implemented using a programmable logic device such as an FPGA (Field Programmable Gate Array).

[0047] Although each embodiment and variant example has been described, these are merely examples and are not intended to limit the scope of the present invention. For example, one of the embodiments or variant examples, or a part of each embodiment or a part of each variant example, may be combined with one or more other embodiments or one or more other variant examples to realize one aspect of the present invention. [Explanation of symbols]

[0048] 1...dialogue support system, 100...processing server device, 110...customer generation unit, 111...reception unit, 112...customer definition unit, 120...dialogue control unit, 121...utterance acquisition unit, 122...emotion parameter processing unit, 123...response prompt generation unit, 124...response text conversion unit, 125...conversation history generation unit, 130...motion control unit, 131...avatar generation unit, 132...voice generation unit, 133...voice tone information processing unit, 134...motion Processing unit, 135...emote processing unit, 136...lip sync processing unit, 140...storage unit, 141...customer information, 142...response information, 143...voice information, 144...action information, 200...generation server device, 210...generation unit, 211...persona generation unit, 212...response text generation unit, 214...unique information acquisition unit, 220...storage unit, 221...unique information, 222...element data, 223...LLM information, 300...user terminal device

Claims

1. a reception unit that receives persona information indicating a persona that characterizes a conversation partner of a user; a persona generation unit that generates a persona generation prompt indicating characteristics of the persona using a large-scale language model based on the persona information received by the reception unit; a storage unit that stores a plurality of element data representing components of an avatar as a conversation partner; an avatar generation unit that selects the element data stored in the storage unit in response to the persona generation prompt generated by the persona generation unit, and generates an avatar using the selected element data; an output unit that outputs an avatar corresponding to the persona based on the persona generation prompt generated by the persona generation unit; An avatar generation system comprising:

2. The avatar generation system according to claim 1 , wherein the element data includes element data relating to a face or a body, element data relating to a voice, and element data relating to a movement.

3. The persona information includes nationality or place of origin, the persona generation unit inputs existing items including the nationality or place of origin of the persona into the large-scale language model as the persona information, and creates a persona generation prompt based on an output of the large-scale language model; the element data includes element data for generating sound, the element data for generating the voice includes synthesized voice data corresponding to elements including a speaking style or tone corresponding to the nationality or place of origin of the persona; a voice generation unit that generates voice data in a language corresponding to the nationality or place of origin based on the persona generation prompt generated by the persona generation unit and the persona information; The avatar generation system according to claim 1 .

4. the element data for generating the voice includes a plurality of element data corresponding to each of a plurality of languages; The avatar generation system according to claim 3 , wherein the voice generation unit selects one of a plurality of element data corresponding to each of a plurality of languages ​​based on the nationality or place of origin included in the persona information.

5. The element data for generating the motion represents the motion of the avatar corresponding to the nationality or place of origin, a motion processing unit that generates a motion of an avatar corresponding to a nationality or place of origin based on the persona generation prompt generated by the persona generation unit and the persona information; The avatar generation system according to claim 3 .

6. The element data includes image information and explanatory text information for each element of the avatar, the avatar generation unit compares the persona generation prompt with the description information, and selects or generates image information of the avatar for the persona based on a comparison result; The avatar generation system according to claim 1 .

7. 2. The avatar generation system of claim 1, wherein the avatar generation unit inputs the persona generation prompt and the plurality of element data into a machine learning model trained on data related to real people as training data, and generates an avatar that imitates a real person based on output from the machine learning model.

8. The avatar generation system according to claim 1 , wherein the avatar generation unit constructs a virtual three-dimensional space and generates and moves an avatar within the three-dimensional space based on the persona generation prompt.

9. a step in which the avatar generation system stores, in a storage unit, a plurality of element data representing components of an avatar as a conversation partner of a user; The avatar generation system receives persona information indicating a persona that characterizes a user's interaction partner; generating, by the avatar generation system, a persona generation prompt that indicates characteristics of the persona based on the received persona information using a large-scale language model; a step in which the avatar generation system selects the element data stored in the storage unit for the generated persona generation prompt, and generates an avatar using the selected element data; The avatar generation system outputs an avatar corresponding to the persona based on the generated persona generation prompt; A method for generating an avatar, comprising:

10. The computer in the avatar generation system a step in which the avatar generation system stores, in a storage unit, a plurality of element data representing components of an avatar as a conversation partner of a user; The avatar generation system receives persona information indicating a persona that characterizes a user's interaction partner; generating, by the avatar generation system, a persona generation prompt that indicates characteristics of the persona based on the received persona information using a large-scale language model; a step in which the avatar generation system selects the element data stored in the storage unit for the generated persona generation prompt, and generates an avatar using the selected element data; The avatar generation system outputs an avatar corresponding to the persona based on the generated persona generation prompt; A program that executes.

Images