Response control device, response control method, and response control program

The response control device addresses the challenge of generating natural and personality-accurate NPC dialogue by translating utterances and using character-specific models, resulting in immersive and cost-effective dialogue experiences in virtual spaces.

WO2024247704A1PCT designated stage expired Publication Date: 2024-12-05SONY GROUP CORP

Patent Information

Application Number
PCT/JP2024/017713
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-26
Filing Date
2024-05-14
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Existing dialogue systems in virtual spaces struggle to generate high-quality response sentences that accurately reflect the personality of Non-Player Characters (NPCs), often resulting in unnatural phrasing and requiring significant time and effort to prepare character strings or train accurate dialogue models, which is costly and impractical.

Method used

A response control device that acquires utterances and character information, translates them into a second language to exclude individuality, and then uses a character translation model to generate responses that match the NPC's personality, separating the process of content generation from tone expression to achieve precise personality reflection without extensive model learning.

Benefits of technology

This approach allows for natural and immersive dialogue experiences by generating responses that accurately reflect NPC personalities with reduced preparation burden and learning costs, providing a more realistic and engaging virtual environment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024017713_05122024_PF_FP_ABST
    Figure JP2024017713_05122024_PF_FP_ABST
Patent Text Reader

Abstract

A response control device according to an embodiment of the present disclosure comprises: an acquisition unit that acquires speech in a first language, and character information, which is information indicating the personality of a speaker related to the speech; a response unit that generates a response to the speech in the first language by translating the speech in the first language into a second language and then generating a response sentence to the speech in the second language, and translating the generated response sentence into the first language using a translation model corresponding to the character information; and an output control unit that outputs the response generated by the response unit.
Need to check novelty before this filing date? Find Prior Art

Description

Response control device, response control method, and response control program

[0001] The present disclosure relates to a response control device, a response control method, and a response control program for providing a dialogue experience that reflects the personality of a character.

[0002] Various technologies have been put into practical use to present users with three-dimensional virtual spaces constructed within computers or networks. For example, three-dimensional virtual spaces are constructed as game spaces running on computers, or as virtual worlds that mimic real worlds, such as the Metaverse.

[0003] These virtual spaces may contain virtual characters called NPCs (Non-Player Characters). A user can interact with an NPC in the virtual space using a character or avatar that the user controls. For example, in a game, a user can obtain information about gameplay from an NPC with whom the user has a predetermined interaction.

[0004] A dialogue model using artificial intelligence (AI) may be used for dialogue with NPCs. For example, a known dialogue-related technique is to define semantics in the utterances of a voice agent realized by AI and replace the semantics with terms that match the character attributes, thereby expressing dialogue that is in line with the character.

[0005] Japanese Patent Application Laid-Open No. 2022-29869

[0006] According to the conventional technology, it is possible to realize a dialogue that expresses character, as if two unique people were having a conversation.

[0007] However, the method of partially replacing character strings for each character attribute can lead to unnaturalness in the overall sentence or phrasing. Furthermore, preparing replacement character strings one by one to realize various personalities is extremely time-consuming and unrealistic. One approach to address this issue is to train a dialogue model itself to generate responses that reflect character attributes. However, due to issues such as the cost of the learning environment required to tune a highly accurate dialogue model and the time required for training, this approach is difficult to implement.

[0008] Therefore, this disclosure proposes a response control device, a response control method, and a response control program that can provide a dialogue experience that reflects the character's individuality with high accuracy without imposing an excessive preparation burden.

[0009] In order to solve the above problems, one form of response control device according to the present disclosure includes an acquisition unit that acquires an utterance in a first language and character information that is information indicating the personality of the speaker related to the utterance; a response unit that generates a response to the utterance in the first language by translating the utterance in the first language into a second language and then generating a response sentence to the utterance in the second language and translating the generated response sentence into the first language using a translation model corresponding to the character information; and an output control unit that outputs the response generated by the response unit.

[0010] FIG. 1 is a diagram (1) showing an overview of the response control processing according to the embodiment. FIG. 2 is a block diagram showing an overview of the response control processing according to the embodiment. FIG. 3 is a diagram showing an example of response sentence generation according to character information. FIG. 4 is a diagram (1) showing an example of character information according to the embodiment. FIG. 5 is a diagram (2) showing an example of character information according to the embodiment. A diagram showing an example of the configuration of a response control device according to the embodiment. A diagram showing an example of a character translation model storage unit according to the embodiment. A flowchart showing the flow of the response control processing according to the embodiment. A diagram showing an example of a character translation model storage unit according to a modified example. A hardware configuration diagram showing an example of a computer that realizes the functions of a response control device.

[0011] Hereinafter, embodiments will be described in detail with reference to the drawings. In the following embodiments, the same components are designated by the same reference numerals, and redundant description will be omitted.

[0012] The present disclosure will be described in the following order: 1. Embodiment 1-1. Overview of response control processing according to embodiment 1-2. Configuration of response control device according to embodiment 1-3. Processing procedure according to embodiment 1-4. Modification 2. Other embodiments 3. Effects of response control device according to the present disclosure 4. Hardware configuration

[0013] (1. Embodiment) (1-1. Overview of response control processing according to embodiment) An example of response control processing according to an embodiment will be described using Fig. 1. Fig. 1 is a diagram (1) showing an overview of response control processing according to an embodiment.

[0014] The response control process according to the embodiment is executed by a response control device 100 (not shown in FIG. 1 ). The response control device 100 is an information processing terminal for executing content such as games and implementing VR and AR technologies. The response control device 100 is implemented by any information processing device, such as a game console, a personal computer (PC), a tablet terminal, or a smartphone. The response control device 100 may be, for example, a head-mounted display (HMD) or AR glasses.

[0015] In an embodiment, the response control device 100 displays a three-dimensional virtual space (hereinafter simply referred to as "virtual space") constructed within a computer or network, such as a metaverse, on a display unit (display, etc.).

[0016] The response control device 100 displays a character 10 in a virtual space that performs various behaviors in response to user operations. The character 10 is a character that can be played by a user and is referred to as a playable character, player character, etc. The response control device 100 also displays an NPC (Non-Player Character) 20 in the virtual space, which is a character whose behavior is automatically performed by the game system.

[0017] For example, the response control device 100 reflects the gestures and facial expressions of the user in the virtual space based on input from an input device such as a game controller, or information captured by an image sensor (such as a camera) capturing an image of the user in real space. The response control device 100 also reflects speech by the character 10 in the virtual space by accepting text input from a controller or voice input via a microphone. This allows the user to experience the sensation of having an alter ego exist in the virtual space.

[0018] A large number of NPCs are placed in the virtual space. For example, NPCs have the role of buying and selling game items that users can use in the game, or of providing users with useful information in the game. Furthermore, in order to virtually represent towns and cities, NPCs are placed in a manner that simulates various roles, such as office workers on their way to work, shoppers in a store, ordinary passersby, and families with children. These NPCs are set to, for example, automatically wander the virtual space, travel a set course, or move in a loop around a set course.

[0019] These NPCs can be individually assigned personalities and attributes, thereby creating a more realistic virtual space. For example, by effectively reflecting the personal expressions, endings, and phrases contained in the speech of each character, particularly those that express their character, NPCs can create a sense of realism for the user. For example, just like in the real world, NPCs can include characters that speak in a friendly manner, characters that speak politely, and characters that speak rudely. This allows the user to become more immersed in the worldview of the virtual space.

[0020] However, in a dialogue system that adapts to each of the numerous NPCs present in a virtual space, it is difficult to create high-quality responses that match the character's tone of voice. For example, a method that replaces partial character strings for each character attribute can result in unnatural overall sentences and phrasing. Furthermore, a method that trains a dialogue model to generate responses that reflect character attributes would require training numerous models given the diverse personalities of the characters, making it difficult to implement from a cost perspective.

[0021] Therefore, the response control device 100 according to the present disclosure solves the above problem by the following configuration. First, the response control device 100 acquires an utterance in a first language and character information, which is information indicating the personality of the speaker who uttered the utterance. Next, the response control device 100 translates the utterance in the first language into a second language different from the first language. Furthermore, the response control device 100 generates a response sentence to the utterance by inputting the utterance in the second language into a model that generates a response sentence. After that, the response control device 100 generates a response to the initially acquired utterance by translating the generated response sentence into the first language using a translation model corresponding to the character information. Finally, the response control device 100 outputs the generated response to the user by displaying it on a screen or playing it as audio.

[0022] That is, when generating a response sentence, the response control device 100 first extracts only the content of the response using a second language that excludes expressions such as personality from the language. Then, the response control device 100 generates the final response sentence to be output using the extracted content and a translation model that has been trained to express the content in accordance with personality. In this way, the response control device 100 performs the process of generating the response content itself and the process of imparting personality to the response content separately, thereby making it possible to generate a response that reflects personality with a high degree of accuracy without having to train a model that generates a response sentence for each personality.

[0023] The response control process will be described in order using Figure 1 and subsequent figures. For example, Figure 1 shows a situation in which a character 10 controlled by a user is having a conversation with an NPC 20 in a virtual space that is one screen of game content. In the example of Figure 1, the NPC 20 is given a personality that uses somewhat rude language.

[0024] In the example shown in FIG. 1 , the contents of the utterance 15 of the character 10 and the response 25 of the NPC 20 are as follows: Character 10: "Do you usually do housework?" NPC 20: "I only do the bare minimum." Character 10: "So you do the three things properly: cooking, laundry, and cleaning?" NPC 20: "Well, laundry and cleaning, I guess. I can't wear the same clothes every day." Character 10: "I guess it's surprising that you usually clean." NPC 20: "Apart from my workroom... I keep the rest of my room pretty clean." Character 10: "There seem to be a lot of energy drink cans lying around." NPC 20: "How did you know?! Are you a psychic?!" As described above, the response control device 100 reflects the words uttered by the user in the character 10 and outputs the utterance of the NPC 20 in response to the words uttered by the character 10. In this way, the response control device 100 outputs natural responses that reflect the personality of the NPC 20, as if they were actual people having a conversation.

[0025] The response sentence generation process shown in Fig. 1 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an overview of the response control process according to the embodiment.

[0026] The response control device 100 acquires the voice uttered by the user and the text input by the user as input text 40. The input text 40 corresponds to the speech of the character 10. The response control device 100 also acquires character information 30, which is setting information related to the personality, etc., of the NPC 20 with whom the user is talking. Note that the character information 30 may include not only the personality, etc. set for the NPC 20, but also any information related to the NPC 20, such as the dialogue history with the character 10.

[0027] First, the response control device 100 performs a language determination process on the input text 40 (step S10). For example, the response control device 100 determines whether the input text 40 needs to be converted into a language that can be input into a response sentence generation model used in the dialogue system. Furthermore, if the input text 40 is not in a language that can be input into a response sentence generation model, the response control device 100 determines from which language to which translation is required. Note that if the dialogue is between NPCs that does not involve a user, the response control device 100 may determine that conversion processing is not required because there is no need to output the language being processed.

[0028] Next, the response control device 100 uses a translation model of the language corresponding to the acquired input text 40 to translate the input text 40 into the language used in the response sentence generation model (step S11). As an example, the response control device 100 selects an input translation model 42, which is a model for translating the content of the input text 40, which is in Japanese, into English, and translates the input text 40. Note that a known model may be used as such a language translation model.

[0029] Next, the response control device 100 generates a response sentence in response to the input text 40. For example, if the input text 40 is "Do you usually do housework?", an utterance by the character 10 shown in FIG. 1 , the response control device 100 translates the utterance into English and generates a response sentence in response to the utterance, "Only the bare minimum." That is, in order to generate a response to the input text 40 that excludes individuality, the response control device 100 first generates only the content of the response in a second language (English in this example) in which individuality is less likely to be reflected. Note that a known model may be used as such a response sentence generation model. Furthermore, by acquiring character information 30 of the conversation partner, the response control device 100 also manages the dialogue history with the NPC 20, which is necessary for the dialogue, at this stage. That is, by acquiring the character information 30 of the NPC 20, the response control device 100 can read the dialogue history linked to the NPC 20 and create a response sentence that can be a continuation of the dialogue.

[0030] Thereafter, the response control device 100 executes translation for output so that the user can understand the dialogue with the NPC 20 (step S13). Here, the response control device 100 selects a translation model based on the character information 30 of the NPC 20. For example, the response control device 100 selects a character translation model 44, which is a model for translating from English to Japanese and which has been trained in accordance with the personality of the NPC 20, who speaks in a somewhat rough manner.

[0031] That is, the response control device 100 translates the response sentence generated in step S12 into a language that matches the tone of speech of the NPC 20 and is the same as the language of the initially input input text 40. The character translation model includes a variety of models, such as models that match the tone of speech of each character that has been set in detail, and models that only have simple personality settings (such as "polite" or "rough").

[0032] An example of combining the process of Fig. 2 with the dialogue of Fig. 1 will be described with reference to Fig. 3. Fig. 3 is a diagram (2) showing an outline of the response control process according to the embodiment.

[0033] 3, each time an utterance 15 is input, the response control device 100 generates a response sentence to be uttered by the NPC 20 via the input translation model 42 (step S12). Then, for each generated response sentence, the response control device 100 outputs a response 25 of the NPC 20 using the character translation model 44. Specifically, the response control device 100 outputs the response 25 by translating the response 35 in the second language (English) obtained in generating the response sentence using the character translation model 44 trained in accordance with the personality of the NPC 20.

[0034] In this way, the response control device 100 separately generates a response sentence using a response sentence generation model and generates an expression using a translation model that produces an expression that is in line with the character. This allows the response control device 100 to generate responses by NPCs in a variety of tones from a single common response sentence. This point will be explained using Figure 4. Figure 4 is a diagram showing an example of response sentence generation based on character information.

[0035] 4, NPC 21 is a character set to speak in a slightly more polite tone than NPC 20. In this case, the response control device 100 translates the response 35 into the first language (Japanese in this example) via a character translation model 45 that has been trained in accordance with the tone (personality) corresponding to NPC 21. This allows the response control device 100 to output a response 26 by NPC 21.

[0036] 4, the NPC 22 is a character who speaks in a somewhat more childish manner, giving the impression of being somewhat more approachable than the NPC 20. In this case, the response control device 100 translates the response 35 into the first language via a character translation model 46 that has been trained in accordance with the tone of voice (personality) corresponding to the NPC 22. This allows the response control device 100 to output a response 27 by the NPC 22.

[0037] In addition, NPCs placed in a game are assigned personalities, and each NPC can also have a conversation independently of the user. In this case, the conversation between NPCs may be executed as an internal process if it is not displayed on the screen or output as audio. When the conversation between NPCs is not output, the response control device 100 can omit translation from the second language to the first language and execute a conversation based only on the content, without using any personal expressions.

[0038] As a specific example, consider a conversation between NPC23 (not shown), who has a "serious personality and speaks in a friendly manner," and NPC24 (not shown), who has a "calm personality and speaks in a taciturn manner."

[0039] NPC 23 utters to NPC 24, "It's a nice day today, isn't it?" The response control device 100 acquires "It's a nice day today" as input text and converts it into a second language (English in this example) that does not express any character. The response control device 100 also generates a response to the translated utterance. This utterance is internally processed as a dialogue such as the following: NPC 23: "It's a nice day today, isn't it?" NPC 24: "Well"

[0040] If the speech of the NPC 24 is to be output in the form of a screen display or the like, the response control device 100 converts it into a response sentence that suits the NPC 24 by using the language information and character information of the original input text. Since the NPC 24 has been given the personality of being taciturn, the output of the character translation model is output in the form of a short reply after a period of silence, for example, as follows: NPC 23: "It's a nice day today, isn't it?" NPC 24: "...I guess so."

[0041] When it is not necessary to output a dialogue between NPCs, the response control device 100 can use the generated response sentence in the second language for dialogue processing, thereby enabling more direct processing within the system. For example, the dialogue between NPC 23 and NPC 24 continues in the second language as follows:

[0042] NPC23: "It's a nice day today, isn't it?" NPC24: "Well" NPC23: "It's getting warmer and warmer, and spring may be around the corner." NPC24: "That's true." NPC23: "Are you okay with hay fever?" NPC24: "I don't have hay fever, so there's no problem." NPC23: "I see. I envy you. It's pretty hard for me." NPC24: "OK."

[0043] When outputting the above dialogue, the response control device 100 can use a character translation model based on the information on each character and output it as follows: NPC23: "It's a nice day today." NPC24: "...I guess so." NPC23: "It's gradually getting warmer, maybe spring is near." NPC24: "That's true...maybe." NPC23: "Are you okay with hay fever?" NPC24: "I don't have hay fever. It's no problem." NPC23: "I see. That's great. I have a pretty hard time." NPC24: "I see..."

[0044] Even if the content of the above dialogue is the same, when an NPC 23 who is "serious and speaks in a friendly manner" and an NPC 28 who is "rough and speaks in a humorous manner" dialogue with each other, the response control device 100 can output the following dialogue: NPC 23: "It's a nice day today." NPC 28: "Yeah, I guess so." NPC 23: "It's getting warmer and warmer, maybe spring is near." NPC 28: "Yeah, I think so." NPC 23: "Are you okay with hay fever?" NPC 28: "I don't have hay fever. I've never been bothered by it." NPC 23: "I see. That's great to hear. It's quite tough for me." NPC 28: "Really?"

[0045] In other words, even if the response sentence (in English) output from the response sentence generation model is the same, the response control device 100 can then output a dialogue with different expressions depending on the character translation model selected.

[0046] 2. It should be noted that when a user (character 10) is involved in a dialogue between NPCs, the response control device 100 appropriately performs language determination and translation of text input by the user. The response control device 100 also generates a response sentence in response to the user's input, translates the generated response sentence using a character translation model, and outputs a response. For example, the response control device 100 outputs the following dialogue: NPC 23: "It's a nice day today." NPC 24: "...I guess so." NPC 23: "It's gradually getting warmer, maybe spring is near." NPC 24: "That's true...maybe so." NPC 23: "Are you okay with hay fever?" NPC 24: "I don't have hay fever. It's no problem." Character 10: "Oh, NPC 24 doesn't have hay fever. I'm jealous." (Language determination and translation are performed on this user input.) NPC 23: "You're right. I have a pretty hard time." (This response is generated in response to the user's speech.) NPC 24: "I see..."

[0047] Since the user can join the dialogue at any time, the response control device 100 generates a response sentence for the NPC at the time the user joins. Therefore, it is expected that the content of the above dialogue will change depending on whether the user speaks to NPC 23 or NPC 24.

[0048] The relationship between the character information set for an NPC and the translation result will now be illustrated using Figures 5 and 6. Figure 5 is a diagram (1) showing an example of character information according to an embodiment. Figure 6 is a diagram (2) showing an example of character information according to an embodiment.

[0049] The data table 60 in Figure 5 shows examples of response sentences that are output when the gender of an NPC is set to "male" and the personality is set to "polite," "friendly," "cool," "wild," "gentle," etc.

[0050] Similarly, data table 61 in Figure 6 shows examples of response sentences to be output when the gender of an NPC is set to "female" and the personality is set to "polite," "friendly," "cool," "wild," "gentle," etc.

[0051] When learning a character translation model, the response control device 100 can generate a character translation model that suits each individual's personality by collecting and learning text data (corpus) that is expressed as shown in Figures 5 and 6.

[0052] As described above with reference to FIGS. 1 to 6 , the response control process according to the embodiment operates separately between a model that generates a response sentence without using expressions and a model that translates into a language that includes expressions tailored to the individual's personality. Generally, a dialogue generation model must generate an accurate response based on semantic content, which requires significant training costs. On the other hand, the response control device 100 according to the embodiment generates a response sentence using a response sentence generation model and generates expressions using a translation model that provides expressions tailored to the character. This allows the response control device 100 to generate responses from NPCs with various speaking tones without the burden of learning a response sentence generation model for each character. In other words, the response control device 100 can accurately output dialogue expressions tailored to various personalities without the effort of generating a response sentence generation model tailored to every personality.

[0053] (1-2. Configuration of response control device according to embodiment) Next, a description will be given of the configuration of the response control device 100. Fig. 7 is a diagram showing an example of the configuration of the response control device 100 according to the embodiment.

[0054] 7, the response control device 100 includes a communication unit 110, a storage unit 120, a control unit 130, a sensor unit 140, and a display unit 150. The response control device 100 may also include an input unit (such as a keyboard or a touch panel) that accepts various operations from a user operating the response control device 100.

[0055] The communication unit 110 is realized by, for example, a network interface card (NIC) or a network interface controller. The communication unit 110 is connected to a network N by wire or wirelessly, and transmits and receives information to and from external devices, etc. via the network N. The network N is realized by, for example, a wireless communication standard or method such as Bluetooth (registered trademark), the Internet, Wi-Fi (registered trademark), UWB (Ultra Wide Band), or LPWA (Low Power Wide Area).

[0056] The storage unit 120 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk.

[0057] The storage unit 120 stores various information related to the response control process according to the embodiment. In the embodiment, the storage unit 120 includes an input translation model storage unit 121, a response sentence generation model storage unit 122, and a character translation model storage unit 123.

[0058] The input translation model storage unit 121 stores a translation model for translating text in a first language input via voice or key input into a second language. For example, the input translation model storage unit 121 has a model for translating text input in Japanese into English text.

[0059] The response sentence generation model storage unit 122 stores models used to generate response sentences and dialogues.

[0060] The character translation model storage unit 123 stores a model for translating text in a second language into text that includes expressions that reflect individuality such as gender and personality.

[0061] An example of a model stored in the character translation model storage unit 123 will be described using Fig. 8. Fig. 8 is a diagram showing an example of the character translation model storage unit 123 according to the embodiment. As shown in Fig. 8, the character translation model storage unit 123 has items such as "character translation model ID," "gender," and "personality."

[0062] "Character translation model ID" is identification information for identifying a character translation model. "Gender" indicates the gender set for the NPC with which you are having a conversation. "Personality" indicates the personality set for the NPC with which you are having a conversation. In other words, the character translation model storage unit 123 has different character translation models for each gender and personality of the NPC.

[0063] 7, the sensor unit 140 is a sensor that detects various types of environmental information. For example, the sensor unit 140 includes a camera 141 for sensing the appearance and behavior of the user, a microphone 142 for sensing voice input by the user, and the like.

[0064] The camera 141, for example, recognizes a subject (for example, a user located in real space) located in front of the response control device 100. The camera 141 may acquire an image of the subject, the user, and calculate the distance from the camera 141 to the subject based on the parallax between images captured by a stereo camera or an analysis of the captured image data. The sensor unit 140 may also detect the distance in real space using a depth sensor capable of detecting the distance to an arbitrary subject, such as a user.

[0065] The microphone 142 measures the user's voice and surrounding sounds. For example, when a user engages in voice chat, the user can input voice using the microphone 142 provided in the response control device 100.

[0066] Furthermore, for example, when the response control device 100 is a wearable device such as an HMD, the sensor unit 140 may have a function of detecting various information related to the user's movements, such as the orientation, tilt, movement, and movement speed of the user's body, in addition to the camera 141 and microphone 142. Specifically, the sensor unit 140 detects information related to the user's movements, such as information related to the user's head and posture, the movements of the user's head and body (acceleration and angular velocity), the direction of the field of view, and the speed of the viewpoint movement. For example, the sensor unit 140 functions as various motion sensors, such as a three-axis acceleration sensor, a gyro sensor, and a speed sensor, to detect information related to the user's movements. More specifically, the sensor unit 140 detects the components of the yaw direction, pitch direction, and roll direction as the movement of the user's head, thereby detecting a change in at least one of the position and posture of the user's head.

[0067] The sensor unit 140 does not necessarily have to be provided in the response control device 100, and may be, for example, an external sensor connected by wire or wirelessly to the response control device 100. For example, the camera 141 does not necessarily have to be provided inside the response control device 100, and may be an imaging device installed outside the response control device 100.

[0068] The response control device 100 can reflect in the virtual space the user's behavior detected by the sensor unit 140. For example, the response control device 100 can reflect the user's hand-raising movement or the user's line of sight on the character 10, or reflect the words uttered by the user as the character 10's speech.

[0069] The display unit 150 displays various information output from the control unit 130. For example, the display unit 150 is a display that outputs video to the user. The display unit 150 may also include an audio output unit (such as a speaker) that outputs audio.

[0070] The control unit 130 is realized by, for example, a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), etc. executing a program (for example, a response control program according to the present disclosure) stored inside the response control device 100 using a RAM, etc. as a working area. The control unit 130 is a controller, and may be realized by, for example, an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0071] 7 , the control unit 130 includes an acquisition unit 131, a response unit 132, and an output control unit 136. The response unit 132 includes a first translation unit 133, a response sentence generation unit 134, and a second translation unit 135.

[0072] The acquisition unit 131 acquires various types of information. For example, the acquisition unit 131 acquires input information to be reflected on the character 10 by sensing the user using the sensor unit 140. Specifically, the acquisition unit 131 acquires the user's movements in real space and text included in the voice as input information.

[0073] Furthermore, the acquisition unit 131 acquires character information set for each NPC. For example, the acquisition unit 131 recognizes an NPC with which the user is about to interact or an NPC displayed on the screen, and acquires character information of the recognized NPC. Furthermore, if the user and the NPC have previously interacted or are currently interacting, the acquisition unit 131 may acquire an interaction history between the user and the NPC as character information.

[0074] That is, the acquisition unit 131 acquires an utterance in the first language and character information that indicates the personality of a speaker related to the utterance (the speaker who uttered the utterance or the person in the conversation). The acquisition unit 131 passes the acquired information to the response unit 132.

[0075] The response unit 132 translates the utterance in the first language into the second language, and then generates a response sentence in response to the utterance in the second language. The response unit 132 also generates a response to the utterance in the first language by translating the generated response sentence into the first language using a character translation model corresponding to the character information.

[0076] Specifically, the first translation unit 133 of the response unit 132 determines the input first language and selects a translation model corresponding to the determined first language to translate the utterance into the second language. In this way, the response unit 132 obtains text that excludes character characteristics and includes only the semantic content of the utterance. The response sentence generation unit 134 of the response unit 132 then generates a response sentence in the second language in response to the text translated into the second language. The response sentence generation unit 134 may generate a response sentence in response to the utterance in the second language by referencing a dialogue history included in the character information. The second translation unit 135 of the response unit 132 then translates the response sentence in the second language using a character translation model that is consistent with the character information of the NPC with which the user is interacting, thereby generating a response sentence in the first language that reflects the personality of the NPC.

[0077] As described above, the response unit 132 generates a response using a translation model trained for each gender and personality set for the speaker. At this time, if the response unit 132 determines that the response will not be output in a format that the user can recognize, the response unit 132 may use a response sentence in the second language as the response without translating the response sentence into the first language.

[0078] For example, if it is determined that the user is not involved in a conversation, such as a conversation between NPCs in a game, the response unit 132 determines that there is no need to output a response in a format that the user can understand, and the conversation can continue in the second language.

[0079] In the embodiment, the response unit 132 acquires an utterance in, for example, Japanese, which is a first language, and generates a response sentence by translating the utterance in Japanese into English, which is a second language; however, the language is not limited to this example and may be any language.

[0080] The output control unit 136 controls the output of the response generated by the response unit 132. For example, the output control unit 136 outputs the response from the NPC in any format, such as by displaying the response sentence on the display unit 150 or playing audio corresponding to the response sentence from a speaker.

[0081] More specifically, the output control unit 136 outputs the response generated by the response unit 132 as text in the first language or as audio in the first language. For example, in a virtual space such as game content, the output control unit 136 outputs the response as part of a dialogue (such as a speech bubble) between the user and an NPC in the virtual space.

[0082] The output control unit 136 may control the display of the behavior of NPCs, etc., to give the virtual space a sense of realism. For example, the output control unit 136 may display NPCs as if they were wandering freely in the virtual space, or display NPCs having natural conversations with each other. When allowing the user to recognize the content of a conversation between NPCs, the output control unit 136 may display the conversation between NPCs translated into the first language and reflecting their personalities. By controlling the display in this way, the output control unit 136 can present the virtual space as a more realistic world, thereby giving the user a deeper sense of immersion.

[0083] (1-3. Processing Procedure According to the Embodiment) Next, the processing procedure according to the embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the flow of response control processing according to the embodiment.

[0084] 9, the response control device 100 acquires a dialogue (input text) (step S101). At this point, the response control device 100 may also acquire character information of the speaker involved in the dialogue.

[0085] The response control device 100 determines whether or not the acquired dialogue needs to be translated for input (step S102). If translation for input is needed (step S102; Yes), for example, if the acquired dialogue is in a first language and needs to be translated into a language (second language) that can be input to the response sentence generation model, the response control device 100 executes translation for input (step S103).

[0086] Thereafter, the response control device 100 inputs the text in the second language of the translated word into the response sentence generation model to generate a response sentence (step S104). Note that if translation for input is not required (step S102; No), the response control device 100 can input the acquired text directly into the response sentence generation model.

[0087] Next, the response control device 100 determines whether a dialogue with the user will occur in the dialogue (step S105). If a dialogue with the user will occur (step S105; Yes), the response control device 100 needs to output a response using an expression that reflects the individuality of the character, and therefore, after referencing the character information of the responding NPC, determines whether there is a translation model for the character that corresponds to the character information (step S106).

[0088] If there is a translation model for the corresponding character (step S106; Yes), the response control device 100 uses the model for that character to perform translation for output (step S107).

[0089] On the other hand, if there is no translation model for the corresponding character (step S106; No), for example, if a model corresponding to the NPC's personality is not available, or if the NPC is set as a general-purpose character rather than an important character, the response control device 100 performs the output translation using the general-purpose model (step S108).

[0090] Thereafter, the response control device 100 outputs a response sentence from the NPC, who is the other party in the dialogue (step S109). Note that if no dialogue with the user occurs (step S105; No) and the dialogue between NPCs continues, translation is not necessary, and the response control device 100 may generate the response sentence in the second language as is.

[0091] (1-4. Modifications) The processing according to the above embodiment may be modified in various ways.

[0092] (1-4-1. Responses Including Emotions) In the above embodiment, an example was shown in which the response control device 100 generates a response sentence using a model for each NPC's personality. Here, the response control device 100 may further generate a response using a model trained according to the NPC's emotion.

[0093] That is, the response control device 100 can also express an emotional tone of voice in a dialogue with an NPC, similar to the character setting.

[0094] In this case, the response control device 100 collects example sentences that correspond to the emotions that are to be reflected in the dialogue in order to pre-train a translation model for characters that includes emotions. Specifically, the response control device 100 clusters a large collection of example sentences and acquires feature quantities for a collection of example sentences that are thought to include emotions. In this case, the feature quantity is set to a representative value, such as the average of vectorized values ​​of the example sentences. The response control device 100 then uses the vector information as a reference for discrimination using similarity to further collect example sentences that include emotions, and trains a translation model using the collected data.

[0095] For example, when a character's emotions change as the dialogue progresses, the response control device 100 can generate a response that is more suited to the situation by applying a translation model that includes the learned emotions. That is, since the response control device 100 generates responses using the dialogue history, it can estimate changes in the speaker's emotions as the dialogue progresses using a known emotion determination process. In other words, the response control device 100 estimates the emotion (e.g., "joy," "anger," "sadness," "enjoyment," "confusion," "fear," etc.) that will be applied to the next response sentence to be generated in the dialogue. Then, the response control device 100 applies a translation model that corresponds to the estimated emotion.

[0096] An example of a translation model for each emotion is shown in Fig. 10. Fig. 10 is a diagram showing an example of a character translation model storage unit 123A according to a modified example. As shown in Fig. 10, the character translation model storage unit 123A has an item such as "emotion" in addition to the character translation model storage unit 123.

[0097] The response control device 100 can output a response that is in line with the emotion by selecting a model that corresponds to the emotion estimated by a model that estimates emotions based on the content of the dialogue and translating the response sentence.

[0098] As an example, let's say the following dialogue takes place between Speaker A and Speaker B: Speaker A: "I'm sorry, but I can't join you today." Speaker B: "Oh, that's sudden." Speaker A: "I have an urgent matter to attend to." Speaker B: "I see, that can't be helped." Speaker A: "I'm so sorry." Speaker B: "Please let me know sooner next time." Speaker A: "I'll make it up to you." Speaker B: "Okay, see you next time."

[0099] In this case, the response control device 100 generates the following responses in the second language based on the input text: Speaker A: "Sorry, I can't go with you today." Speaker B: "Yeah, it's a short notice." Speaker A: "Something urgent came up." Speaker B: "I see, it can't be helped." Speaker A: "I'm really sorry." Speaker B: "Tell me early next time." Speaker A: "I'll make it up to you." Speaker B: "Yeah, maybe next time."

[0100] At this time, if speaker A is the NPC 23 who has a "serious personality and speaks in a friendly manner" as described above, and speaker B is the NPC 24 who has a "calm personality and speaks in a taciturn manner", the response control device 100 can translate using expressions that match the character information of each. For example, the response control device 100 outputs the following dialogue: NPC 23: "Sorry, I can't make it to tomorrow's plans." NPC 24: "Oh, that's sudden." NPC 23: "I have some urgent business to attend to." NPC 24: "I see..." NPC 23: "I'm really sorry." NPC 24: "Please let me know earlier next time." NPC 23: "I'll make it up to you." NPC 24: "Ah, ... see you next time."

[0101] Here, the response control device 100 can perform an emotion estimation process in a dialogue and apply a model suited to the emotion of each dialogue, thereby producing the following output. It is assumed that in such a dialogue, the NPC 24 felt the emotion of "anger" during the dialogue. NPC 23: "Sorry, I can't make it to tomorrow's plans." NPC 24: "Oh, that's sudden." NPC 23: "I have an urgent matter to attend to." NPC 24: "I see..." (Response including the emotion of "anger") NPC 23: "I'm really sorry." NPC 24: "Next time, please tell me earlier..." (Response including the emotion of "anger") NPC 23: "I'll make it up to you." NPC 24: "Ah, ... see you next time" (Response including the emotion of "anger")

[0102] As another example, the speaker A is an NPC 23 who has a "serious personality and speaks in a friendly manner" and the speaker B is an NPC 28 who has a "rough personality and speaks in a humorous manner". When emotions are not included, the response control device 100 outputs the following dialogue: NPC 23: "Sorry, I can't make it to tomorrow's plans." NPC 28: "Oh, that's sudden." NPC 23: "I have some urgent business to attend to." NPC 28: "Oh, I see, I guess there's nothing I can do about it." NPC 23: "I'm really sorry." NPC 28: "Please let me know earlier next time." NPC 23: "I'll make it up to you." NPC 28: "Okay, see you next time!"

[0103] When emotion estimation is applied to the above dialogue, the response control device 100 outputs, for example, the following dialogue: NPC 23: "Sorry, I can't make it to tomorrow's plan." NPC 28: "Oh, that was sudden." NPC 23: "I had to take care of something urgently." NPC 28: "Oh, I'm sorry." (Response including emotion of "anger") NPC 23: "I'm really sorry." NPC 28: "Next time, you should have said it sooner." (Response including emotion of "anger") NPC 23: "I'll make it up to you." NPC 28: "Damn, maybe next time." (Response including emotion of "anger")

[0104] Depending on the personality of the speaker, even if a response is generated using a model that includes emotional expressions, it may be difficult to distinguish between normal speech and emotional speech. In this case, the response control device 100 may perform processing such as converting the intonation of the voice into one that includes the corresponding emotion, in addition to the text representation.

[0105] Furthermore, the response control device 100 can forcibly set emotions in the dialogue, rather than using the estimated emotions in the dialogue. That is, the response control device 100 can include any emotion in the expression of a response by applying a translation model that includes emotions instead of a normal translation model. The following is an example of a dialogue that does not include emotions. NPC23: "It's a nice day today." NPC24: "...I guess so." NPC23: "It's getting warmer and warmer, maybe spring is near." NPC24: "That's true...maybe." NPC23: "Are you okay with hay fever?" NPC24: "I don't have hay fever. It's not a problem." NPC23: "I see. That's great. I have a pretty hard time." NPC24: "I see..."

[0106] The response control device 100 can output the following dialogue by applying a model that has been trained to express emotions such as "joy" and "sadness" to the above dialogue: NPC23: "It's a nice day today." NPC24: "...I guess so." NPC23: "It's gradually getting warmer, spring might be near." NPC24: "That's true...maybe." NPC23: "Are you okay with hay fever?" NPC24: "Fortunately, I don't have hay fever. It's not a problem." (Example of translation using a model that is forcibly adapted to the emotion of "joy") NPC23: "I see. That's great. I have a really hard time." NPC24: "Is that so..." (Example of translation using a model that is forcibly adapted to the emotion of "sadness")

[0107] As another example, the response control device 100 can output the following dialogue: NPC 23: "It's a nice day today." NPC 28: "Yeah, I guess so." NPC 23: "It's getting warmer, maybe spring is near." NPC 28: "Yeah, I think so." NPC 23: "Are you okay with hay fever?" NPC 28: "I don't have hay fever, after all. I'm fine." (Example of translation forcibly using a model that corresponds to the emotion of "joy.") NPC 23: "I see. That's great. I have a pretty hard time." NPC 28: "Wow, is that so?" (Example of translation forcibly using a model that corresponds to the emotion of "sadness.")

[0108] In this way, the response control device 100 can generate a response using a translation model trained for each emotion set for the speaker. The response control device 100 may estimate the emotion of the speaker regarding the utterance and generate a response using a translation model corresponding to the estimated emotion, or may forcibly set an emotion for the speaker regarding the utterance and generate a response using a translation model corresponding to the set emotion. In this way, the response control device 100 can provide the user with a dialogue using a wider variety of expressions by outputting a dialogue that includes emotions.

[0109] (1-4-2. System Configuration) The configuration of the response control device 100 shown in FIG. 7 conceptually illustrates the functions, and may take various forms depending on the embodiment. For example, in the above embodiment, an example was shown in which the response control device 100 performs the response control process and the output control process. However, the process according to the embodiment may be executed by a server device that performs the response control process and an information processing terminal or the like that performs the output control process in cooperation with each other. In other words, the response control device 100 may be a device in which the display unit and the information processing unit are configured separately. In this case, the information processing unit of the response control device 100 may be any information processing device such as a server or a PC.

[0110] In the embodiment, an example in which the response control process is executed in a virtual space has been described. However, the response control process does not necessarily have to be executed in a virtual space. For example, the response control process may be applied to a dialogue between robots or smart speakers in a real space, or to a dialogue between a user and a robot or the like in a real space.

[0111] In the embodiment, an example has been shown in which the response control device 100, when acquiring an utterance in Japanese, which is a first language, translates the utterance into English, which is a second language, in order to eliminate individuality. However, the translation is not limited to these languages. Furthermore, the response control device 100 may translate the first language into any abstract expression that eliminates individuality, rather than translating the first language into the second language.

[0112] In addition, when generating a response sentence in a second language, the response control device 100 may generate the response sentence using a dialogue AI model provided by an external service, rather than using a response sentence generation model stored in the memory unit 120.

[0113] (2. Other Embodiments) The processing according to each of the above-described embodiments may be implemented in various different forms other than the above-described embodiments.

[0114] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. Furthermore, the information, including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings, can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

[0115] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0116] Furthermore, the above-described embodiments and modifications can be combined as appropriate within the scope of not causing any contradiction in the processing content.

[0117] Furthermore, the effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0118] (3. Effects of the Response Control Device According to the Present Disclosure) As described above, the response control device according to the present disclosure (the response control device 100 in the embodiment) includes an acquisition unit (the acquisition unit 131 in the embodiment), a response unit (the response unit 132 in the embodiment), and an output control unit (the output control unit 136 in the embodiment). The acquisition unit acquires an utterance in a first language and character information that is information indicating the personality of the speaker related to the utterance. The response unit translates the utterance in the first language into a second language, and then generates a response sentence in response to the utterance in the second language, and translates the generated response sentence into the first language using a translation model (a character translation model in the embodiment) corresponding to the character information, thereby generating a response to the utterance in the first language. The output control unit outputs the response generated by the response unit.

[0119] In this way, the response control device according to the present disclosure uses separate models to generate response sentences related to the dialogue content and to translate the generated responses in a tone of voice that matches the character's voice. This allows the response control device to provide the user with a dialogue experience that accurately reflects the character's personality, without imposing an excessive preparation burden such as training numerous response generation models.

[0120] The response unit also generates a response using a translation model that has been trained for each gender or personality type set for the speaker.

[0121] In this way, the response control device can appropriately express a tone of voice that matches the character of the speaker by preparing models for translating response sentences according to gender and personality.

[0122] The response unit also references the dialogue history included in the character information to generate a response sentence to the utterance in the second language.

[0123] Furthermore, when the output control unit determines that the response will not be output in a format recognizable by the user, the response unit may use the response sentence in the second language as a response without translating the response sentence into the first language. Note that when the output control unit determines that the user is not involved in the dialogue regarding the utterance, the output control unit may determine not to output the response in a format recognizable by the user.

[0124] In this way, the response control device can reduce the load of dialogue processing by internally processing responses that do not require output in the second language.

[0125] Furthermore, the response unit generates a response using a translation model trained for each emotion set for the speaker. For example, the response unit estimates the emotion of the speaker regarding the utterance and generates a response using a translation model corresponding to the estimated emotion. Alternatively, the response unit may set an emotion for the speaker regarding the utterance and generate a response using a translation model corresponding to the set emotion.

[0126] In this way, the response control device can provide the user with a wider variety of dialogue expressions by expressing a tone of voice that includes emotion.

[0127] For example, the acquisition unit acquires an utterance in Japanese, which is a first language. The response unit translates the utterance in Japanese into English, which is a second language. The output control unit outputs the response as text display in the first language or as audio in the first language. The output control unit may also output the response as part of a dialogue between the user and an NPC in the virtual space.

[0128] In this way, the response control device can provide the user with an immersive virtual space experience, such as dialogue using a variety of expressions, by outputting the generated response as text or voice.

[0129] (4. Hardware Configuration) Information devices such as the response control device 100 according to each of the above-described embodiments are realized by a computer 1000 configured as shown in FIG. 11 . The following description will be given using the response control device 100 as an example. FIG. 11 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of the response control device 100. The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. The components of the computer 1000 are connected by a bus 1050.

[0130] The CPU 1100 operates and controls each component based on programs stored in the ROM 1300 or the HDD 1400. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to the various programs.

[0131] The ROM 1300 stores boot programs such as a Basic Input Output System (BIOS) that is executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000 .

[0132] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records a response control program according to the present disclosure, which is an example of program data 1450.

[0133] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0134] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from input devices such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to output devices such as a display, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs and the like recorded on a predetermined recording medium. Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase Change Rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, and semiconductor memories.

[0135] For example, when the computer 1000 functions as the response control device 100 according to the embodiment, the CPU 1100 of the computer 1000 executes a response control program loaded onto the RAM 1200 to realize the functions of the control unit 130, etc. The response control program according to the present disclosure and data in the storage unit 120 are stored in the HDD 1400. The CPU 1100 reads and executes program data 1450 from the HDD 1400, but as another example, the CPU 1100 may obtain these programs from another device via an external network 1550.

[0136] The present technology can also be configured as follows. (1) A response control device comprising: an acquisition unit that acquires an utterance in a first language and character information that is information indicating a speaker's personality related to the utterance; a response unit that translates the utterance in the first language into a second language, and then generates a response to the utterance in the second language, and translates the generated response into the first language using a translation model corresponding to the character information, thereby generating a response to the utterance in the first language; and an output control unit that outputs the response generated by the response unit. (2) The response control device according to (1), wherein the response unit generates the response using the translation model trained for each gender set for the speaker. (3) The response control device according to (1) or (2), wherein the response unit generates the response using the translation model trained for each personality set for the speaker. (4) The response control device according to any one of (1) to (3), wherein the response unit generates the response to the utterance in the second language by referring to a dialogue history included in the character information. (5) The response control device according to any one of (1) to (4), wherein, when the output control unit determines that the response will not be output in a format recognizable by the user, the response unit does not translate the response sentence into the first language and sets the response sentence in the second language as the response. (6) The response control device according to (5), wherein, when it is determined that the user is not involved in a dialogue related to the utterance, the output control unit determines not to output the response in a format recognizable by the user. (7) The response control device according to (3), wherein the response unit generates the response using the translation model trained for each emotion set for a speaker. (8) The response control device according to (7), wherein the response unit estimates an emotion of the speaker with respect to the utterance and generates the response using the translation model corresponding to the estimated emotion. (9) The response control device according to (7) or (8), wherein the response unit sets an emotion for the speaker with respect to the utterance and generates the response using the translation model corresponding to the set emotion.(10) The response control device according to any one of (1) to (9), wherein the acquisition unit acquires an utterance in Japanese, which is the first language, and the response unit translates the utterance in Japanese into English, which is the second language. (11) The response control device according to any one of (1) to (10), wherein the output control unit outputs the response by displaying text in the first language or by speech in the first language. (12) The response control device according to (11), wherein the output control unit outputs the response as part of a dialogue between a user and an NPC (Non Player Character) in a virtual space. (13) A response control method comprising: a computer acquiring an utterance in a first language and character information that is information that indicates a speaker's personality regarding the utterance, translating the utterance in the first language into a second language, generating a response sentence to the utterance in the second language, and translating the generated response sentence into the first language using a translation model corresponding to the character information, thereby generating a response to the utterance in the first language, and outputting the generated response. (14) A response control program for causing a computer to function as a response control device comprising: an acquisition unit that acquires an utterance in the first language and character information that is information that indicates a speaker's personality regarding the utterance, a response unit that translates the utterance in the first language into a second language, generating a response sentence to the utterance in the second language, and translating the generated response sentence into the first language using a translation model corresponding to the character information, thereby generating a response to the utterance in the first language, and an output control unit that outputs the response generated by the response unit.

[0137] 10 Character 100 Response control device 110 Communication unit 120 Storage unit 121 Input translation model storage unit 122 Response sentence generation model storage unit 123 Character translation model storage unit 130 Control unit 131 Acquisition unit 132 Response unit 133 First translation unit 134 Response sentence generation unit 135 Second translation unit 136 Output control unit 140 Sensor unit 141 Camera 142 Microphone 150 Display unit

Claims

1. A response control device comprising: an acquisition unit that acquires an utterance in a first language and character information, which is information indicating the personality of a speaker regarding the utterance; a response unit that generates a response to the utterance in the first language by translating the utterance in the first language into a second language and then generating a response sentence to the utterance in the second language and translating the generated response sentence into the first language using a translation model corresponding to the character information; and an output control unit that outputs the response generated by the response unit.

2. The response control device according to claim 1, wherein the response unit generates the response using the translation model trained for each gender set for the speaker.

3. The response control device according to claim 2, wherein the response unit generates the response using the translation model trained for each personality set for a speaker.

4. The response control device according to claim 1, wherein the response unit generates a response sentence to the utterance in the second language by referring to a dialogue history included in the character information.

5. A response control device as described in claim 1, wherein when the output control unit determines that the response is not output in a format recognizable by the user, the response unit does not translate the response sentence into the first language, but instead uses the response sentence in the second language as the response.

6. The response control device according to claim 5, wherein the output control unit determines not to output the response in a format recognizable by the user when it determines that the user is not involved in the dialogue regarding the utterance.

7. The response control device according to claim 3, wherein the response unit generates the response using the translation model trained for each emotion set for a speaker.

8. The response control device according to claim 7, wherein the response unit estimates the emotion of the speaker regarding the utterance, and generates the response using the translation model corresponding to the estimated emotion.

9. The response control device according to claim 7, wherein the response unit sets an emotion for the speaker regarding the utterance, and generates the response using the translation model corresponding to the set emotion.

10. The response control device according to claim 1, wherein the acquisition unit acquires an utterance in Japanese, which is the first language, and the response unit translates the utterance in Japanese into English, which is the second language.

11. The response control device according to claim 1, wherein the output control unit outputs the response by displaying a text in the first language or by speech in the first language.

12. The response control device according to claim 11, wherein the output control unit outputs the response as part of a dialogue between the user and an NPC (Non Player Character) in a virtual space.

13. A response control method comprising: a computer acquiring an utterance in a first language and character information which is information indicating a speaker's personality with respect to the utterance; translating the utterance in the first language into a second language, generating a response sentence to the utterance in the second language, and translating the generated response sentence into the first language using a translation model corresponding to the character information, thereby generating a response to the utterance in the first language; and outputting the generated response.

14. A response control program for causing a computer to function as a response control device comprising: an acquisition unit that acquires an utterance in a first language and character information, which is information indicating the speaker's personality regarding the utterance; a response unit that generates a response to the utterance in the first language by translating the utterance in the first language into a second language and then generating a response sentence to the utterance in the second language and translating the generated response sentence into the first language using a translation model corresponding to the character information; and an output control unit that outputs the response generated by the response unit.

Citation Information

Patent Citations

  • Method and device for generating interactive models

    JP2021108094A

  • Display program, display method and information processing device

    JP2021111190A

  • input / output system, input / output program, information processing device, chat system

    JP6243072B1

Cited By

  • Information processing device and translation method

    JP7802902B1