Interactive processing system, interactive processing method, interactive processing device, and program

Abstract

One aspect of the present disclosure is an interactive processing system having a processor that performs processing in response to a request from a user terminal device, the interactive processing system comprising: a personality information generation unit that generates personality-defining text information indicating the personality of a subject on the basis of a plurality of personality factor parameters that affect the personality of the subject; and a response processing unit that generates response text information on the basis of utterance information received from the user terminal device and the personality-defining text information generated by the personality information generation unit.

Description

Dialogue processing system, dialogue processing method, dialogue processing device, and program

[0001] The present invention relates to a dialogue processing system, a dialogue processing method, a dialogue processing device, and a program.

[0002] Conventional technologies using language models include those described in Patent Documents 1 and 2. The data generation method described in Patent Document 1 uses original data to construct a prompt that serves as an input sentence for the language model, inputs the prompt into a language model, and generates new data and label information for the new data from the language model, thereby generating sentences with high grammaticality and naturalness. Patent Document 1 also describes that the prompt includes a text type (e.g., review, article, etc.) and a label type (e.g., emotion, classification, etc.). Patent Document 2 describes an information processing device in which, while a user and an operator are speaking in real time, the user's terminal estimates the operator's emotion, displays an avatar image expressing the estimated emotion within a range of individual emotional expression levels set using a slider bar, and outputs the received audio data. The emotional expression level is determined by the overall emotional expression level set using a slider bar.

[0003] JP 2023-18624 A JP 2023-72110 A

[0004] Patent document 1 inputs prompts that express emotions as label types (emotion ratios) into a language model, and patent document 2 displays the degree of emotional expression of an avatar, but it is difficult to provide an interactive avatar that operates taking into account the personality and dialogue content of a specific person.

[0005] The present disclosure has been made in light of the above circumstances, and aims to provide an interactive avatar that operates taking into account the personality and dialogue content of a specific person.

[0006] The present disclosure has been made to solve the above-mentioned problems, and one aspect of the present disclosure is an interaction processing system having a processor that performs processing in response to a request from a user terminal device, the interaction processing system including: a personality information generation unit that generates personality definition text information indicating the personality of a subject based on a plurality of personality factor parameters that influence the personality of the subject; and a response processing unit that generates response text information based on utterance information received from the user terminal device and the personality definition text information generated by the personality information generation unit.

[0007] Another aspect of the present disclosure is an interaction processing method executed by a processor that performs processing in response to a request from a user terminal device, the interaction processing method including the steps of: generating personality definition text information indicating the personality of a subject based on a plurality of personality factor parameters that influence the personality of the subject; and generating response text information based on utterance information received from the user terminal device and the personality definition text information.

[0008] Another aspect of the present disclosure is a dialogue processing device having a processor that performs processing in response to a request from a user terminal device, the dialogue processing device comprising: a personality information generation unit that generates personality definition text information indicating the personality of a subject based on a plurality of personality factor parameters that affect the personality of the subject; and a response processing unit that generates response text information based on utterance information received from the user terminal device and the personality definition text information generated by the personality information generation unit.

[0009] Another aspect of the present disclosure is a program that causes a computer of an information processing device to execute the steps of generating personality definition text information indicating the personality of a subject based on a plurality of personality factor parameters that influence the personality of the subject, and generating response text information based on utterance information received from a user terminal device and the personality definition text information.

[0010] According to one aspect of the present invention, it is possible to provide an interactive avatar that operates taking into account the personality and dialogue content of a specific person.

[0011] Fig. 1 is a block diagram showing an example of a configuration of a dialogue processing system 1 according to an embodiment. Fig. 2 is a diagram illustrating an overview of processing by the dialogue processing system 1 according to an embodiment. Fig. 3 is a diagram showing an example of processing for emotion parameters by the dialogue processing system 1 according to an embodiment. Fig. 4 is a flowchart showing an example of a processing procedure for the dialogue processing system 1 according to an embodiment. Fig. 5 is a flowchart showing an example of a processing procedure for generating a response text by the dialogue processing system 1 according to an embodiment. Fig. 6 is a flowchart showing an example of processing for controlling an avatar by the dialogue processing system 1 according to an embodiment.

[0012] A dialogue processing system, a dialogue processing method, a dialogue processing device, and a program to which the present invention is applied will be described below with reference to the drawings.

[0013] 1 is a block diagram showing an example of the configuration of a dialogue processing system 1 according to an embodiment. The dialogue processing system 1 according to the embodiment provides an interactive avatar that operates taking into consideration the personality and dialogue content of a specific person. For example, the dialogue processing system 1 sets a specific person as a target person and controls the target person's avatar using information representing the target person's personality, knowledge, voice, appearance, episodes, past comments, profile, etc., thereby providing a content provision service that makes it seem as if the target person and a user are having a conversation.

[0014] The dialogue processing system 1 includes, for example, a processing server device 100, a generation server device 200, and a user terminal device 300. The processing server device 100, the generation server device 200, and the user terminal device 300 are communicatively connected via a network NW such as the Internet. The processing server device 100, the generation server device 200, and the user terminal device 300 may be connected via either wired or wireless communication, and may include a general-purpose network such as the Internet, or a private network such as local 5G or Wi-Fi (registered trademark). The processing server device 100, the generation server device 200, and the user terminal device 300 may have a communication interface, such as a network interface card (NIC) or a wireless communication module, for connecting to a network, and may exchange information with each other.

[0015] The user terminal device 300 is, for example, an information processing device operated by a user who interacts with an avatar. The user terminal device 300 includes, for example, a speaker, a microphone, a display device, an operation unit, and a processing unit such as a CPU.

[0016] The processing server device 100 is a server device that includes, for example, a processor that performs processing in response to requests received from the generation server device 200 and the user terminal device 300 and transmits the processing results to the processing server device 100. The processing server device 100 includes, for example, a personality information generation unit 110, a response processing unit 120, an operation control unit 130, and a storage unit 140. The personality information generation unit 110, the response processing unit 120, and the operation control unit 130 are functional units realized by, for example, an information processing circuit that performs various processes by causing a central processing unit (CPU) to execute a program. Furthermore, some or all of these functional units may be realized by hardware such as a large-scale integration (LSI), an application-specific integrated circuit (ASIC), or a field-programmable gate array (FPGA), or may be realized by a combination of software and hardware. The storage unit 140 is realized by, for example, a hard disk drive (HDD), a solid state drive (SSD), a flash memory, an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a random access memory (RAM), or a hybrid storage device that uses a combination of these. Part or all of the storage unit 140 may be realized by an external storage device accessible via various networks. An example of an external storage device is a network attached storage (NAS) device.

[0017] The personality information generation unit 110 receives a plurality of personality factor parameters that influence the personality of the subject and generates personality definition text information that indicates the personality of the subject. The personality information generation unit 110 includes, for example, a personality information extraction unit 111 and a prohibited matter confirmation unit 112.

[0018] The personality information extraction unit 111 collects information about the target person from external devices such as a terminal device of a service provider or an information providing server device connected via a communication network NW. The personality information extraction unit 111 extracts, from the collected information, personal information about the target person, such as the target person's profile information, speech information, conversation history, SNS information posted by the target person, and episode information. The personality information extraction unit 111 generates a personality extraction prompt using the various extracted information. The personality extraction prompt is text data including information such as the target person's personality.

[0019] The personality information generation unit 110 receives a plurality of personality factor parameters, for example, from a terminal device of a service provider or a terminal device related to the subject. The plurality of personality factor parameters are parameters indicating factors that represent the subject's personality. Factors that represent the subject's personality include, for example, neuroticism, extroversion, openness, conscientiousness, and agreeableness, and the parameters are numerical values ​​that indicate the degree of these factors. The personality information generation unit 110 includes the plurality of personality factor parameters in a personality extraction prompt. The personality information generation unit 110 transmits the personality extraction prompt to the generation server device 200 and obtains a personality definition text from the generation server device 200.

[0020] The prohibited item confirmation unit 112 confirms prohibited items included in the personality definition text information. Prohibited items include, for example, information that is not permitted to be leaked outside the target person or information that violates public order and morals. Information that violates public order and morals may include, for example, information that violates public order and morals specific to a specific country. If the personality definition text information includes prohibited items, the prohibited item confirmation unit 112 adds the prohibited items included in the personality definition text to a list.

[0021] The response processing unit 120 performs processing related to responses to be output to the user terminal device 300 during a dialogue between a user and a target person's avatar. The response processing unit 120 includes, for example, a speech-to-text conversion unit 121, an emotion parameter processing unit 122, a response prompt conversion unit 123, a response prompt generation unit 124, a response text conversion unit 124, and a conversation history generation unit 125. The speech-to-text conversion unit 121 converts user voice input from the user terminal device 300 into text data. The emotion parameter processing unit 122 performs processing to set and update emotion parameters. The emotion parameters are numerical values ​​that indicate the emotions of the avatar. The emotion parameters are information that expresses emotions such as joy, anger, sadness, enjoyment, confidence, confusion, and fear on a five-point scale from 1 to 5. The response prompt generation unit 123 generates a response prompt including text data of the user voice, personality definition text, and emotion parameters, and transmits the generated response prompt to the generation server device 200. The response text conversion unit 124 converts the response text acquired from the generation server device 200 into voice data. The conversation history generation unit 125 generates a history of the conversation between the user and the avatar of the target person.

[0022] The response processing unit 120 may generate response text information reflecting a speaking style unique to a specific country based on information indicating a specific country included in the profile information of the subject. The specific country may be the subject's nationality, country of origin, region, etc. included in the profile information stored in the storage unit 140. The specific country may also be interpreted as a specific region such as North America, South America, Africa, Europe, Asia, or Oceania. The response prompt generation unit 123 may, for example, include text data corresponding to the speaking style and culture of the specific country in the response prompt. The response prompt generation unit 123 may also generate a response prompt including information such as a tone of voice, pitch, and tone that reflects the speaking style and culture unique to the specific country. As a result, the response processing unit 120 performs processing to generate a response that reflects the speaking style and culture unique to the specific country.

[0023] Furthermore, the response processing unit 120 may generate response text information in a language corresponding to the specific country and in a language corresponding to a country other than the specific country based on information indicating multiple countries including the specific country included in the subject's profile information. For example, the response prompt generation unit 123 includes text data indicating that the subject speaks in the language of the specific country and the languages ​​of countries other than the specific country (multilingual) in the response prompt. This allows the response processing unit 120 to generate response texts spoken by the subject in multiple languages. The response processing unit 120 can also generate response text information reflecting the speaking style of the specific country and the speaking style of the country other than the specific country.

[0024] The movement control unit 130 performs processing to control the movement of the target person's avatar. The movement control unit 130 includes, for example, an avatar generation unit 131, a voice generation unit 132, a voice tone information processing unit 133, a motion processing unit 134, an emote processing unit 135, and a lip sync processing unit 136. The avatar generation unit 131 generates an avatar of the target person. The avatar generation unit 131 creates component information representing content that displays an avatar based on, for example, an image of the target person's appearance, and edits the content based on operations on the service provider's terminal device. The voice generation unit 132 generates voice data based on the response text. The voice generation unit 132 generates, for example, voice data that reproduces the target person's actual voice reading. The voice tone information processing unit 133 processes the voice data based on voice tone information corresponding to emotion parameters. The motion processing unit 134 controls the motion of the target person's avatar based on the emotion parameters and the content of the response text. The motion of the target person's avatar represents, for example, the movement of the entire avatar or the movement of the avatar's hands. The emote processor 135 controls the facial expression of the target person's avatar based on the emotion parameters and the content of the response text. The emote processor 135 controls, for example, the movement of the avatar's eyes, eyebrows, mouth, etc. The lip sync processor 136 controls the movement of the avatar's lips based on the emotion parameters and the content of the response text.

[0025] The storage unit 140 stores, for example, person information 141, response information 142, voice information 143, and movement information 144. The person information 141 includes, for example, profile information, speech information, episode information, personality extraction prompts, and personality definition text. The response information 142 includes, for example, user voice text, response text, emotional parameter initial values, and emotional parameter current values. The voice information 143 includes, for example, voice data such as user voice and response voice, emotional parameters, and tone of voice information. The movement information 144 includes, for example, emotional parameters, component information, motion information, emote information, and lip-sync information. The motion information is a default value representing the avatar's motion, the emote information is a default value representing the avatar's emote, and the lip-sync information is a default value representing the avatar's lip-sync.

[0026] The movement control unit 130 may control the movement of the subject's avatar to be unique to a specific country based on information indicating a specific country included in the subject's profile information. The movement control unit 130 controls the avatar's movement to reflect, for example, gestures, hand movements, and culture unique to a specific country. The movement control unit 130 may, for example, generate an avatar with an appearance corresponding to the subject's nationality and country of origin using the avatar generation unit 131. The audio generation unit 132 may generate audio data corresponding to the subject's nationality and country of origin. The motion processing unit 134 may control the overall movement of the avatar or the hand movement of the avatar according to the subject's nationality and country of origin. The emote processing unit 135 may control the movement of the avatar's eyes, eyebrows, mouth, etc. according to the subject's nationality and country of origin. The lip sync processing unit 136 may control the movement of the avatar's lips according to the subject's nationality and country of origin.

[0027] The generation server device 200 is, for example, a server device that performs processing in response to a request received from the processing server device 100 and transmits the processing results. The generation server device 200 includes, for example, a generation unit 210, a storage unit 220, and an LLM learning unit 230. The generation unit 210 and the LLM learning unit 230 are functional units realized by, for example, an information processing circuit that performs various processes by causing a CPU to execute a program. The storage unit 220 is realized, for example, by a recording device such as an HDD or SSD, or a hybrid storage device using a plurality of these, and may also be realized by an external storage device accessible via various networks, such as a NAS device.

[0028] The generation unit 210 includes, for example, a personality definition text generation unit 211, a response text generation unit 212, an emotion parameter processing unit 213, and a knowledge information acquisition unit 214. The personality definition text generation unit 211 inputs a personality extraction prompt acquired from the processing server device 100 into a first language model and generates personality definition text information based on the output of the first language model. The response text generation unit 212 generates a response text from the response prompt (including the personality definition text and the speech text) generated by the response prompt generation unit 123, the conversation history with the user generated by the conversation history generation unit 125, and context information acquired by the knowledge information acquisition unit 214. The response text generation unit 212 inputs, for example, the response prompt, the conversation history with the user, and the context information into a second language model and generates a response text based on the second language model. The first language model is, for example, a large language model (LLM) using a neural network. The second language model is an LLM different from the first language model.

[0029] The emotion parameter processing unit 213 generates or updates emotion parameters according to the content of the generated response text. The knowledge information acquisition unit 214 analyzes knowledge information indicating the unique knowledge possessed by the subject, and generates knowledge information of the subject.

[0030] The storage unit 220 includes, for example, subject information 221, LLM information 222, and dialogue AI model information 223. The subject information 221 is a database that associates subjects with their knowledge information. The knowledge information is not limited to the subject's knowledge and may supplement necessary knowledge. Furthermore, the knowledge information may be knowledge required according to the subject's role or the user's purpose. For example, the knowledge information may include product information and customer service information required for various purposes, such as the installation location of the user terminal device 300, selling products to the subject, or conducting training. The LLM information 222 includes parameter information of an LLM (first language model) for generating personality definition text and parameter information of an LLM (second language model) for generating response text. The dialogue AI model information 223 includes parameter information of a dialogue AI model that inputs response prompts and knowledge information and outputs dialogue information.

[0031] The LLM learning unit 230 performs processing to learn an LLM (first language model) for generating a personality definition text and an LLM (second language model) for generating a response text.

[0032] When the response prompt generated by the response prompt generation unit 123 includes text data corresponding to the speaking style and culture of a specific country, the response text generation unit 212 generates a response text that reflects the speaking style and culture of the specific country. The second language model learns, for example, information specifying a specific country, a conversation history with the user, context information, and a response text, and can output a response text that reflects the speaking style of the specific country when information specifying a specific country is input. Speaking styles and voice pitches may differ from country to country because the number of vowels varies from country to country. Furthermore, depending on the culture and topic of a country, for example, the timing at which the speaking speed increases or decreases, the timing at which the speech volume (intonation) increases or decreases, the timing at which speech becomes clear or unclear, and so on may differ. By including such characteristics of the speaking style of the specific country in the response prompt, the response text generation unit 212 can reflect the differences in the speaking style of the specific country in the response text.

[0033] In the embodiment, as shown in Figure 1, the dialogue processing system 1 distributes the functional configuration (functional units) between the processing server device 100 and the generation server device 200, but this is not limited to this, and the functional units may be distributed in other configurations, the functional units of the processing server device 100 and the generation server device 200 may be consolidated into one device, multiple functional units may be combined into one functional unit, or one function may be distributed among multiple functional units.

[0034] 2 is a diagram illustrating an outline of the processing performed by the dialogue processing system 1 according to an embodiment. The processing server device 100 acquires the target person's utterance record and dialogue record D10, profile information D11, and personality factor parameters D12 from an external device, and generates a personality extraction prompt D13 based on the acquired information D10, D11, and D12. The processing server device 100 transmits the personality extraction prompt D13 to the generation server device 200, which inputs the personality extraction prompt D13 into a personality definition text generation LLM (P10) and performs prohibition processing P11 on the output from the personality definition text LLM (P10). The personality definition text D13, to which a command statement defining a prohibition has been added by the prohibition processing P11, is output to the response text generation LLM.

[0035] The generation server device 200 acquires the subject's unique knowledge information D20, processes the unique knowledge information D20 in the following order: text extraction processing P20, chunk division processing P21, and vectorization processing P22, and stores vectors corresponding to the unique knowledge information D20 in a vector database. The processing server device 100 performs speech recognition processing P40 on the user's speech acquired from the user terminal device 300, and performs vectorization processing P41 on the text information processed by the speech recognition processing P40. A reference result to the vector database is extracted from the vector database using the vector corresponding to the speech as a query. The vectorized unique knowledge information D20 and the text information processed by the speech recognition processing P40 are output to the response text generation LLM (P30) together with the personality definition text D14.

[0036] The generation server device 200 inputs the personality definition text D14, the specific knowledge information D20, and the text information processed by the speech recognition process P40 into the response text generation LLM (P30), performs speech synthesis process P31 on the response text output from the response text generation LLM (P30), and performs avatar control process P32 based on the emotion parameters output from the response text generation LLM (P30), thereby transmitting avatar content D30 to the user terminal device 300. This allows the user terminal device 300 to display or output sound using avatar content D30.

[0037] 3 is a diagram showing an example of processing of emotion parameters in the dialogue processing system 1 in the embodiment. Based on the personality factor parameters (A) of subject A, initial values ​​(a1 [%], b1 [%], c1 [%], d1 [%], e1 [%]) of emotion parameters (A) of subject A are set. Based on the personality factor parameters (B) of subject B, initial values ​​(a2 [%], b2 [%], c2 [%], d2 [%], e2 [%]) of emotion parameters (B) of subject B are set. There is a difference (A-B) between the personality factor parameters (A) and the personality factor parameters (B), and a difference occurs between the initial values ​​of emotion parameters (A) and (B) depending on this difference (A-B). A difference (A-B) between the personality factor parameter (A) and the personality factor parameter (B) causes a difference between the range of fluctuation (or rate of fluctuation) of the emotion parameter (A) and the range of fluctuation (or rate of fluctuation) of the emotion parameter (B). The processing server device 100 controls the movement of the avatar of the subject A based on the emotion parameter (A), and controls the movement of the avatar of the subject B based on the emotion parameter (B).

[0038] The initial value of the emotion parameter (A) varies to the emotion parameter (A)# depending on the content of the conversation between the user and the avatar and the personality factor parameter (A). The initial value of the emotion parameter (B) varies to the emotion parameter (B)# depending on the content of the conversation between the user and the avatar and the personality factor parameter (B). The generation server device 200 may adjust the range of variation of the emotion parameter (A)# based on the response text information between the user and the avatar of target person A and the multiple personality factor parameters (A), and may adjust the range of variation of the emotion parameter (B)# based on the response text information between the user and the avatar of target person B and the multiple personality factor parameters (B).

[0039] This allows the processing server device 100 to change the voice data and movements of the avatar based on the emotion parameters that have changed based on the personality of the target person. For example, for the avatar of target person A, the emote information can be adjusted based on the emotion parameter (A) to have a happy expression, the voice tone information can be adjusted to have a gentle voice tone, and the motion information can be adjusted to have a lively motion. For example, for the avatar of target person B, the emote information can be adjusted based on the emotion parameter (B) to have an angry expression, the voice tone information can be adjusted to have an irritated voice tone, and the motion information can be adjusted to have an irritated motion.

[0040] FIG. 4 is a flowchart showing an example of a processing procedure of the dialogue processing system 1 according to the embodiment. First, the processing server device 100 acquires profile information, speech information, utterance information, episode information, etc. of the target person (step S100). The profile information is, for example, text information such as the target person's name, age, nationality, country of origin, occupation, family structure, social status, medical history, and physical characteristics. The speech information and utterance information is, for example, text information such as the target person's utterances and conversation information, information posted on the target's SNS, information on articles in which the target person has been featured, and text information transcribed from audio data or video data. The episode information is text information that represents the target person's past experiences, behavior, etc. Note that the target person may be a real person or a fictional character. If the target person is a fictional character, the profile information, speech information, etc. may be entirely fictional information, or some of it may be information about a real person or information about a deceased person.

[0041] Next, the processing server device 100 generates a personality extraction prompt based on the information acquired in step S100 and transmits it to the generation server device 200 (step S102). The personality extraction prompt is text information including multiple personality factor parameters of the subject. The personality extraction prompt may include information that represents the subject's personality profile, factors that determine behavior, factors that avoid behavior, and the subject's character, and may be information such as the subject's questionnaire results or diagnostic results. The processing server device 100 may use a machine learning model that can input information representing the subject's personality and output personality factor parameters.

[0042] The generation server device 200 inputs the personality extraction prompt acquired from the processing server device 100 into the personality definition text LLM to generate a personality definition text (step S104). The generation server device 200 transmits the generated personality definition text to the processing server device 100.

[0043] The processing server device 100 determines whether the personality definition text acquired from the generation server device 200 contains any prohibited items (step S106), and if the personality definition text contains any prohibited items, adds the prohibited items to a list (step S106: YES, step S108). This allows the generation server device 200 to add prohibited items for specific actions or statements to the personality definition text. The processing server device 100 stores the personality definition text in the storage unit 140 (step S110).

[0044] 5 is a flowchart showing an example of a processing procedure for generating a response text by the dialogue processing system 1 according to the embodiment. The user terminal device 300 transmits voice data to the processing server device 100 in response to a user's voice input (step S200). The processing server device 100 converts the voice data into voice text (step S202) and stores the voice text (step S204). The processing server device 100 retrieves the personality definition text and voice text (step S206), generates a response prompt (step S208), and transmits the response prompt to the generation server device 200. The response prompt includes the personality definition text including personality factor parameters and prohibited items, the voice text, and the current values ​​of the emotion parameters.

[0045] When the generation server device 200 receives the response prompt, it extracts context information from the speech text by referring to the subject's knowledge information (step S210), and generates a response text based on the personality definition text, the speech text, the context information, and the conversation history (step S212). The generation server device 200 generates the response text so that the prohibited items contained in the personality definition text are not included. The generation server device 200 updates the emotion parameters based on the content of the generated response text (step S214), and transmits the response text and emotion parameters to 100.

[0046] The processing server device 100 stores the response text and emotion parameters acquired from the generation server device 200 (step S216), generates a conversation history using the conversation history, voice text, and response text up to the previous time (step S218), and stores the stored emotion parameters as information to be reused (step S220).

[0047] 6 is a flowchart showing an example of a process for controlling an avatar by the dialogue processing system 1 according to the embodiment. First, the processing server device 100 retrieves a stored response text (step S300) and converts the response text into voice data (step S302). The processing server device 100 retrieves a stored emotion parameter (step S304) and changes the avatar's tone of voice, motion, emotes, and lip-sync based on the emotion parameter (step S306). The processing server device 100 transmits avatar content reflecting the changed tone of voice, motion, emotes, and lip-sync to the user terminal device 300. The user terminal device 300 receives the avatar content from the processing server device 100, displays the avatar based on the avatar content, and outputs voice (step S308).

[0048] As described above, according to the embodiment, it is possible to realize a dialogue processing system 1 including a personality information generation unit (110, 111, 211) that receives input of a plurality of personality factor parameters that influence the personality of a target person and generates personality definition text information that indicates the personality of the target person, and a response processing unit (120) that receives input of utterance information received from a user terminal device 300 and the personality definition text information generated by the personality information generation unit and generates response text information. According to the embodiment, it is possible to provide an interactive avatar that operates taking into account the personality and dialogue content of a specific person.

[0049] According to the dialogue processing system 1 of the embodiment, a personality extraction prompt can be automatically generated by extracting a target person's profile information, the target person's speech or conversation history information, and personality factor parameters from information acquired from an external device, and personality definition text information can be generated based on the generated personality extraction prompt. This dialogue processing system 1 can provide an avatar that reproduces the personality of a specific person by adding personality factor parameters to the information used for existing avatars.

[0050] According to the dialogue processing system 1 of the embodiment, a personality extraction prompt including knowledge information is generated, so that an avatar that reproduces not only the personality of a specific person but also the knowledge possessed by the specific person can be provided.

[0051] According to the embodiment of the dialogue processing system 1, profile information, speech or conversation history information, and personality factor parameters can be input into a first language model, and personality definition text information can be generated based on the output of the first language model, thereby reducing the effort required to create personality definition text.

[0052] According to the embodiment of the dialogue processing system 1, prohibited items to be excluded from the response text output by the first language model are extracted, so that the actions or utterances that the avatar may take can be controlled based on the prohibited items described in the personality definition text.

[0053] According to the dialogue processing system 1 of the embodiment, utterance information and personality definition text information are input into a second language model, and response text information is generated based on the output of the second language model, so that the response content of the avatar can be changed taking into account the personality factor parameters contained in the personality definition text information.

[0054] According to the dialogue processing system 1 of the embodiment, emotion parameters of an avatar are generated based on personality definition text information and response text information. Furthermore, according to the dialogue processing system 1 of the embodiment, utterance information, personality definition text information, and response text information are input to a second language model, and response text information and emotion parameters of the avatar are generated based on the output of the second language model. In this way, according to the dialogue processing system 1, it is possible to change the emotion of the avatar in consideration of the personality factor parameters included in the personality definition text information.

[0055] According to the dialogue processing system 1 of the embodiment, the response processing unit 120 can generate response text information that reflects a speaking style unique to a specific country, based on information indicating a specific country included in the profile information of the target person. Furthermore, according to the dialogue processing system 1 of the embodiment, the operation control unit can control the behavior of the target person's avatar so that it behaves in a way unique to a specific country, based on information indicating a specific country included in the profile information of the target person. As a result, according to the dialogue processing system 1, by setting a specific country for the target person, it is possible to realize an avatar that speaks and acts in a way unique to a specific country.

[0056] According to the dialogue processing system 1 of the embodiment, the response processing unit 120 can generate response text information in a language corresponding to the specific country and in a language corresponding to a country other than the specific country, based on information indicating multiple countries including the specific country included in the target person's profile information. As a result, the dialogue processing system 1 can realize avatars that are compatible with the languages ​​of many countries.

[0057] According to the dialogue processing system 1 of the embodiment, the movement of the target person's avatar can be controlled based on the response text information by the movement control unit 130. As a result, according to the dialogue processing system 1, the avatar can be controlled based on not only the personality and dialogue content of a specific person but also the movement of the specific person.

[0058] According to the dialogue processing system 1 of the embodiment, the initial values ​​of the emotion parameters are set based on a plurality of personality factor parameters, and the emotion parameters are changed based on response text information, so that an avatar can be provided in which the emotion parameters have been changed in accordance with the personality of a specific person. Furthermore, according to the dialogue processing system 1, the range of variation of the emotion parameters is adjusted based on the response text information and a plurality of personality factor parameters, so that the emotion can be changed in accordance with the personality of a specific person. Furthermore, according to the dialogue processing system 1, the voice data of the avatar and the movement of the avatar are changed based on the emotion parameters, so that an avatar can be provided in accordance with the personality of a specific person.

[0059] The functions of the processing server device 100, the generation server device 200, and the user terminal device 300 in the above-described embodiments may be implemented by a computer. In this case, a program for implementing this function may be recorded on a computer-readable recording medium, and the program may be loaded into a computer system and executed. Note that the term "computer system" as used herein includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as floppy disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into a computer system. Furthermore, the term "computer-readable recording medium" may also include devices that dynamically store programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or telephone lines, or devices that store programs for a fixed period of time, such as volatile memory within the computer system that serves as the server or client. The program may be designed to implement only a portion of the above-described functions, or may be capable of implementing the above-described functions in combination with programs already stored in the computer system, or may be implemented using a programmable logic device such as an FPGA (Field Programmable Gate Array).

[0060] Although each embodiment and variant has been described, these are merely examples and are not intended to be limiting. For example, one aspect of the present invention may be realized by combining any of the embodiments or variants, or a part of each embodiment or a part of each variant, with one or more other embodiments or one or more other variants.

[0061] 1...Dialogue processing system, 100...Processing server device, 110...Personality information generation unit, 111...Personality information extraction unit, 112...Prohibition item confirmation unit, 120...Response processing unit, 121...Speech to text conversion unit, 122...Emotion parameter processing unit, 123...Response prompt generation unit, 124...Response text conversion unit, 125...Conversation history generation unit, 130...Movement control unit, 131...Avatar generation unit, 132...Speech generation unit, 133...Voice tone information processing unit, 134...Motion processing unit, 135...Emote processing unit, 136... Lip sync processing unit, 140... Memory unit, 141... Person information, 142... Response information, 143... Voice information, 144... Action information, 200... Generation server device, 210... Generation unit, 211... Personality definition text generation unit, 212... Response text generation unit, 212... Response processing unit, 213... Emotion parameter processing unit, 214... Knowledge information acquisition unit, 220... Memory unit, 221... Target person information, 222... LLM information, 223... Dialogue AI model information, 230... LLM learning unit, 300... User terminal device

Claims

1. A dialogue processing system having a processor that performs processing in response to a request from a user terminal device, comprising: a personality information generation unit that generates personality definition text information indicating the personality of a subject based on a plurality of personality factor parameters that influence the personality of the subject; and a response processing unit that generates response text information based on utterance information received from the user terminal device and the personality definition text information generated by the personality information generation unit.

2. The dialogue processing system of claim 1, wherein the personality information generation unit generates a personality extraction prompt by extracting profile information of the subject, speech or conversation history information of the subject, and personality factor parameters from information acquired from an external device, and generates the personality definition text information based on the generated personality extraction prompt.

3. The dialogue processing system of claim 2, further comprising a knowledge information acquisition unit that acquires knowledge information indicating the unique knowledge possessed by the subject, and the personality information generation unit generates the personality extraction prompt including the knowledge information acquired by the knowledge information acquisition unit.

4. The dialogue processing system of claim 1, wherein the personality information generation unit inputs the subject's profile information, the subject's speech or conversation history information, and the plurality of personality factor parameters into a first language model, and generates the personality definition text information based on the output of the first language model.

5. The dialogue processing system according to claim 4, wherein said personality information generating section extracts prohibited matters to be excluded from the response text output by said first language model.

6. The dialogue processing system of claim 1, wherein the response processing unit inputs the utterance information and the personality definition text information generated by the personality information generation unit into a second language model, and generates the response text information based on the output of the second language model.

7. The dialogue processing system according to claim 1, wherein said response processing unit generates emotion parameters for an avatar based on said personality definition text information and said response text information generated by said personality information generation unit.

8. The dialogue processing system of claim 6, wherein the response processing unit inputs the utterance information and the personality definition text information generated by the personality information generation unit into the second language model, and generates the response text information and avatar emotion parameters based on the output of the second language model.

9. The dialogue processing system according to claim 1, wherein the response processing unit generates the response text information reflecting a speaking style peculiar to a specific country based on information indicating the specific country included in the target person's profile information.

10. The dialogue processing system of claim 1, wherein the response processing unit generates the response text information in a language corresponding to a specific country and a language corresponding to a country other than the specific country based on information indicating a plurality of countries including the specific country included in the target person's profile information.

11. The dialogue processing system according to claim 1, further comprising a motion control unit that controls motion of an avatar of the target person based on the response text information generated by the response processing unit.

12. The dialogue processing system of claim 11, wherein the response processing unit sets an initial value of an emotion parameter based on the plurality of personality factor parameters and changes the emotion parameter based on the response text information, and the behavior control unit controls the behavior of an avatar based on the emotion parameter.

13. The dialogue processing system according to claim 12, wherein said response processing unit adjusts a range of variation of said emotion parameter based on said response text information generated by said response processing unit and said plurality of personality factor parameters.

14. The dialogue processing system of claim 11, wherein the response processing unit generates emotion parameters for the avatar based on the personality definition text information and the response text information generated by the personality information generation unit, and the movement control unit changes the voice data of the avatar and the movement of the avatar based on the emotion parameters.

15. The dialogue processing system of claim 11, wherein the movement control unit controls the movement of the subject's avatar so that the movement is specific to a specific country based on information indicating the specific country included in the subject's profile information.

16. A dialogue processing method executed by a processor that performs processing in response to a request from a user terminal device, the dialogue processing method comprising: a step of generating personality definition text information indicating a subject's personality based on a plurality of personality factor parameters that influence the subject's personality; and a step of generating response text information based on utterance information received from the user terminal device and the personality definition text information.

17. A dialogue processing device having a processor that performs processing in response to a request from a user terminal device, comprising: a personality information generation unit that generates personality definition text information indicating a subject's personality based on a plurality of personality factor parameters that influence the subject's personality; and a response processing unit that generates response text information based on utterance information received from the user terminal device and the personality definition text information generated by the personality information generation unit.

18. A program that causes a computer of an information processing device to execute the steps of: generating personality definition text information indicating the personality of a subject based on a plurality of personality factor parameters that influence the personality of the subject; and generating response text information based on utterance information received from a user terminal device and the personality definition text information.

Images