Information processing method, program, and information processing system

Abstract

You can have a more natural conversation with your avatar. [Solution] According to an information processing method of one embodiment, the information processing method is executed by an information processing device and includes a storage step of vectorizing sentences in a conversation and storing the sentences of the conversation in correspondence with the sentence vectors, an acquisition step of acquiring a user's voice, a conversion step of converting the acquired voice into text and vector converting it, an extraction step of extracting a sentence vector corresponding to the converted vector, a generation step of generating an answer to the acquired voice based on the sentences of the conversation corresponding to the extracted sentence vector, and a speaking step of speaking the generated answer.

Description

[Technical field]

[0001] The present invention relates to an information processing method, a program, and an information processing system. [Background technology]

[0002] Patent Document 1 discloses a training system that incorporates a communication AI to support the education of professionals who require conversation skills. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-14188 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the case of the above-mentioned prior art, it is necessary to convert the user's voice into text, perform language processing on the converted text to generate an answer, and convert the generated answer into voice. Due to these processes, it takes time for the avatar to respond after the user speaks, making it difficult to have a natural conversation. In addition, since the time it takes for the avatar to speak changes depending on the complexity of the user's speech, a more natural conversation is required by using transition words or the like for the avatar according to that time.

[0005] In consideration of the above, an object of the present invention is to provide an information processing method, a program, and an information processing system that enable a user to have a more natural conversation with an avatar. [Means for solving the problem]

[0006] According to an information processing method of one embodiment, the information processing method is executed by an information processing device, and includes a storage step of vectorizing sentences in a conversation and storing the sentences of the conversation in correspondence with the vectors of the sentences, an acquisition step of acquiring a user's voice, a conversion step of converting the acquired voice into text and vector converting it, an extraction step of extracting vectors of the sentences corresponding to the converted vectors, a generation step of generating an answer to the acquired voice based on the sentences of the conversation corresponding to the extracted vectors of the sentences, and a speaking step of speaking the generated answer.

[0007] According to a program of one embodiment, an information processing device is caused to execute an information processing method which includes a storage step of vectorizing sentences in a conversation and storing the sentences of the conversation in correspondence with the vectors of the sentences, an acquisition step of acquiring a user's voice, a conversion step of converting the acquired voice into text and vector converting it, an extraction step of extracting vectors of the text corresponding to the converted vectors, a generation step of generating an answer to the acquired voice based on the sentences of the conversation corresponding to the extracted vectors of the text, and a speaking step of speaking the generated answer.

[0008] According to one embodiment of an information processing system, the information processing system is executed by an information processing device, and the information processing method is executed by the information processing device, and includes a storage step of vectorizing sentences in a conversation and storing the sentences of the conversation in correspondence with the vectors of the sentences, an acquisition step of acquiring a user's voice, a conversion step of converting the acquired voice into text and vector converting it, an extraction step of extracting vectors of the sentences corresponding to the converted vectors, a generation step of generating an answer to the acquired voice based on the sentences of the conversation corresponding to the extracted vectors of the sentences, and a speaking step of speaking the generated answer. Effect of the Invention

[0009] According to one embodiment, the time from when a user speaks until an avatar responds can be shortened, allowing a user to have a more natural conversation with the avatar. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a server according to an embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a user terminal according to an embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a functional configuration of a server according to an embodiment. [Diagram 5] FIG. 2 is a diagram illustrating an example of a functional configuration of a user terminal according to an embodiment. [Figure 6] 11 is a flowchart illustrating an example of a process performed by the information processing system according to the embodiment until an avatar speaks. [Figure 7] 11 is a flowchart illustrating an example of a process until an avatar speaks when a new answer is generated, which is executed by the information processing system according to the embodiment. [Figure 8] 1 is a diagram showing an overview of a role-playing system according to an embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] An embodiment of an information processing system according to the present invention will be described below with reference to Figures 1 to 5. In each drawing, the same or equivalent components and parts are given the same reference numerals. Also, the dimensional ratios in the drawings are exaggerated for the convenience of explanation and may differ from the actual ratios.

[0012] (System Overview) First, an overview of an information processing system 10 according to this embodiment will be described. The information processing system 10 according to this embodiment is a system that provides a role-playing environment and supports human resource training for jobs that require conversation skills. In this embodiment, the information processing system 10 is a system that can shorten the time from when a user speaks to when an avatar responds in role-playing using an avatar. This allows a user to have a more natural conversation with an avatar.

[0013] (System Configuration) Fig. 1 is a diagram showing an example of a configuration of an information processing system 10 according to the present embodiment. As shown in Fig. 1, the information processing system 10 according to the present embodiment includes a server device 1 and a user terminal 2, which are communicably connected to each other via a network N. The network N is, for example, a wired LAN (Local Area Network), a wireless LAN, the Internet, a public line network, a mobile data communication network, or a combination of these. In the example of Fig. 1, the information processing system 10 includes one server device 1 and one user terminal 2, but may include multiple of each.

[0014] The server device 1 is an example of an information processing device that provides a role-playing environment, generates an answer from a user's voice inputted at the user terminal 2, and causes an avatar to speak the generated answer. The server device 1 may be a PC (Personal Computer), a smartphone, a tablet terminal, a server device, a microcomputer, or a combination of these. The specific configuration and operation of the server device 1 will be described later.

[0015] The user terminal 2 is an example of an information processing device that performs operations for inputting and displaying various information. The user terminal 2 may be a PC (Personal Computer), a smartphone, a tablet terminal, a server device, a microcomputer, a wearable device, or a combination of these. In this embodiment, an information processing device equipped with a voice input function and a photographing function is used as an example, but other functions are not limited as long as the information processing device has the voice input function.

[0016] (Hardware configuration - Server) 2 is a block diagram showing a hardware configuration of the server device 1. The server device 1 includes a processor 101, a memory 102, a storage 103, and a communication I / F 104, which are communicatively connected to each other via a bus B.

[0017] The processor 101 controls each component of the server device 1 and realizes the functions of the server device 1 by expanding various programs stored in the storage 103 into the memory 102 and executing them. The programs executed by the processor 101 include, but are not limited to, an OS (Operating System) and various programs described below. The execution of these programs by the processor 101 realizes a part of the state visualization method according to this embodiment. The processor 101 is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), or a combination of these.

[0018] The memory 102 is, for example, a Read Only Memory (ROM), a Random Access Memory (RAM), or a combination thereof. The ROM is, for example, a Programmable ROM (PROM), an Erasable Programmable ROM (EPROM), an Electrically Erasable Programmable ROM (EEPROM), or a combination thereof. The RAM is, for example, a Dynamic RAM (DRAM), a Static RAM (SRAM), a Magnetoresistive RAM (MRAM), or a combination thereof.

[0019] The storage 103 stores an OS, various programs described below, and various data. The storage 103 is, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), a storage class memory (SCM), or a combination of these.

[0020] The communication I / F 104 is an interface for connecting the server device 1 to an external device including the user terminal 2 via the network N and controlling communication. The communication I / F 104 is, for example, an adapter conforming to Bluetooth (registered trademark), Wi-Fi (registered trademark), ZigBee (registered trademark), Ethernet (registered trademark), or optical communication (for example, Fibre Channel), but is not limited thereto.

[0021] (Hardware configuration - User terminal) 3 is a block diagram showing a hardware configuration of the user terminal 2. The user terminal 2 includes a processor 201, a memory 202, a storage 203, a communication I / F 204, an input / output I / F 205, an input device 206, and an output device 207, which are communicatively connected to each other via a bus B. The communication I / F 204 is an interface for connecting the user terminal 2 to an external device including the server device 1 via a network N and controlling communication. The communication I / F 204 is, for example, an adapter conforming to Bluetooth (registered trademark), Wi-Fi (registered trademark), ZigBee (registered trademark), Ethernet (registered trademark), or optical communication (for example, Fibre Channel), but is not limited thereto.

[0022] The input / output I / F 205 is an interface for connecting an input device 206 and an output device 207 to the user terminal 2. The input device 206 is, for example, a mouse, a keyboard, a touch panel, a microphone, a scanner, a camera, various sensors, an operation button, or a combination of these. The output device 207 as a user interface is, for example, a display, a projector, a printer, a speaker, a vibrator, or a combination of these. In this embodiment, as an example, the output device 207 and the input device 206 are an integrally configured touch panel display.

[0023] In this embodiment, the program may be written in the memory 202 or the storage 203 during the manufacturing stage of the server device 1, or may be provided to the server device 1 via the network N. Also, the program may be provided to the server device 1 via a non-transitory computer-readable recording medium such as a disk medium (not shown).

[0024] (Functional configuration - Server) Next, the functional configuration of the server device 1 will be described. FIG. 4 is a diagram showing an example of the functional configuration of the server device 1. When executing various programs, the server device 1 realizes various functions using the above hardware resources. The server device 1 has a communication unit 11, a storage unit 12, and a control unit 13 as functional configurations realized by the server device 1. Each functional configuration is realized by the processor 101 reading and executing a program stored in the memory 102 or the storage 103. The storage unit 12 also vectorizes sentences of conversation that can be used in role-playing, and stores a conversation sentence 121 and a vector 122 of the sentence in correspondence with each other. Furthermore, the storage unit 12 stores a time adjustment sentence 123 that causes an avatar to speak when a predetermined time is required to generate an answer from the user's voice. The conversation sentence 121 is a set of exchanges in a conversation, and also stores the voice of the beginning part of the answer part that is the set. The conversation sentences to be stored may be set for each theme, for example, ice-breaking, closing, product explanation, and other various themes are set, and conversation sentences 121 corresponding to the theme are stored together with the theme. The time adjustment sentences 123 are sentences that can buy time, such as "Please wait a moment" and "I'll think about it for a moment," and are stored in a number of patterns. The time adjustment sentences for adjusting the time are not stored in the storage unit 12, but are dynamically generated.

[0025] The control unit 13 includes an acquisition unit 131 that acquires a voice sent from the user terminal 2, a conversion unit 132 that converts the voice acquired by the acquisition unit 131 into text and vectorizes it, and an extraction unit 133 that extracts a sentence vector 122 that is close to (corresponding to) the vector converted by the conversion unit 132. The control unit 13 also includes a generation unit 134 that generates a corresponding conversation sentence from the sentence vector 122 extracted by the extraction unit 133, and a speech unit 135 that causes an avatar to speak the generated answer. Details of these will be described later. The generation unit 134 generates an answer using, for example, LLM (Large Language Models).

[0026] (Functional configuration - User terminal) Next, the functional configuration of the user terminal 2 will be described. Fig. 5 is a diagram showing an example of the functional configuration of the user terminal 2. When executing various programs, the user terminal 2 realizes various functions using the above hardware resources. The user terminal 2 has a communication unit 21, a storage unit 22 in which a program 221 is stored, and a control unit 23 as functional configurations realized by the user terminal 2. Each functional configuration is realized by the processor 201 reading and executing the program 221 stored in the memory 202 or the storage 203. The control unit 23 includes an information acquisition control unit 231 that acquires information sent from the server device 1, and a display unit 232 that displays the acquired information on the output device 207.

[0027] Next, the flow of processing of the information processing system according to this embodiment will be described with reference to Fig. 6 to Fig. 8. Fig. 6 is a flowchart showing an example of a process executed by the information processing system 10 according to this embodiment until an avatar speaks. Fig. 7 is a flowchart showing an example of a process executed by the information processing system 10 according to this embodiment until an avatar speaks when a new answer is generated. Fig. 8 is a diagram showing an overview of the role-playing system according to this embodiment.

[0028] First, the process from when the user speaks until when the avatar speaks will be described with reference to FIG. 6. Specifically, the process from when the user speaks until when the avatar responds in a role-playing game using an avatar will be described. In step S101, the acquisition unit 131 acquires the voice of the user's speech input to the user terminal 2. Next, in step S102, the conversion unit 132 converts the user's voice acquired in step S101 into text. Next, in step S103, the conversion unit 132 vectorizes the converted text. Here, when converting voice into text, for example, any software that can convert voice into text, such as voice recognition software, is sufficient, and is not limited to a specific software. In addition, the vectorization of text may be, for example, vectorization of natural language processing, and other methods may be used without being limited thereto.

[0029] Next, in step S104, the extraction unit 133 determines a theme stored in the storage unit 12 from the vector converted in step S103. Next, in step S105, the extraction unit 133 extracts a sentence vector 122 that is close (corresponding) to the vector converted in step S103 in the theme determined in step S104. Next, in step S106, the speech unit 135 causes the speech unit 135 to speak the voice of the beginning part of the answer of the conversation sentence 121 that is paired with the sentence vector 122 in the conversation sentence 121 corresponding to the extracted sentence vector 122. Next, in step S107, the generation unit 134 generates the remaining answer to the user's voice while the beginning part of the answer is spoken in step S106. At this time, the generation unit 134 generates an answer using, for example, LLM (Large Language Models), but is not limited to this and may use another method. Next, in step S108, speaking unit 135 causes the avatar to speak the remaining sentences of the answer generated in step S107.

[0030] In this embodiment, the conversation theme and the exchange in the conversation (for example, a question and an answer, etc.) are stored as a pair, and the vector of the conversation sentence is also stored in advance, so that the time until an answer to the voice uttered by the user is generated can be shortened. At this time, for the answer sentence to be stored, the voice of the first part of the sentence is also stored, so that the answer can be immediately uttered in response to the user's voice. And, by having the first part of the sentence be uttered first, the time from when the user speaks to when the avatar speaks can be shortened. In addition, by storing the theme and the conversation corresponding to the theme in advance, when extracting the vector of the text of the user's voice, the corresponding vector is extracted from the theme that the user is currently talking about, so that the time until extraction can be shortened. In other words, in the past, it was necessary to convert the user's voice into text, perform language processing on the converted text to generate an answer, and convert the generated answer into voice. In contrast, according to this embodiment, the time from when the user speaks to when the avatar answers can be shortened, so that a more natural conversation with the avatar can be held.

[0031] Next, a case where the sentence vector 122 extracted in step S105 in Fig. 6 is not within a predetermined range from the vector of the user's voice will be described with reference to the flowchart in Fig. 7. The processes from step S101 to step S105 in Fig. 7 are similar to those from step S101 to step S104 in Fig. 6, and therefore their description will be omitted. After step S105, in step S109, the extraction unit 133 determines whether or not the sentence vector 122 extracted in step S105 is within a predetermined range from the vector of the user's voice. If it is within the predetermined range (YES), the processes are similar to those from step S106 to step S108 in Fig. 6, and therefore their description will be omitted.

[0032] On the other hand, if it is not within the predetermined range (NO), the process proceeds to step S110, where the utterance unit 135 utters the time adjustment sentence 123. Then, in step S111, the generation unit 134 newly generates a response to the voice. At this time, for example, the response is generated based on the text converted in step S102 using LLM (large-scale language model). If the vector 122 of the extracted sentence is not within the predetermined range, it may not be appropriate as a response to the user's voice, so a new response needs to be generated. Note that since the theme of the conversation is determined in step S104, a response that is in line with the theme can be generated for the user's voice. When generating a new response, in step S106, the utterance unit 135 utters the beginning part first, even if the new response is in the middle of being generated. Then, in step S107, the generation unit 134 generates the remaining response.

[0033] In this way, even if the vector 122 of the sentence extracted in step S105 is not included within a predetermined range from the vector of the user's voice, a new appropriate answer can be generated and spoken by the avatar without giving an inappropriate (off-put) answer. Even if it takes time to generate a new answer, a more natural conversation can be made by storing the time adjustment sentence 123 in advance and having the avatar speak during that time. In other words, if it takes time to generate a new answer, silence continues and it is difficult for the user to know whether the role-playing is progressing properly or not. Therefore, by preparing the time adjustment sentence 123 in advance, even if it takes time to generate a new answer, it is possible to know that the role-playing is not stopping midway by speaking a sentence that can buy time. Also, it is possible to prevent the user from feeling that it is taking time to generate a new answer, and a more natural conversation can be made.

[0034] Next, an example of a screen of the role-playing system according to this embodiment is shown with reference to FIG. 8. The screen of the role-playing system has an area A1 and an area A2, and an avatar screen im01, a user screen im02, and a progress screen im03 are arranged in the area A1. A retry button B1, a pause button B2, and a completion button B3 are arranged in the area A2. When the role-playing starts, the screen shown in FIG. 8 appears, and a conversation with the avatar displayed on the avatar screen im01 begins. During the conversation, the user himself is displayed on the user screen im02. The progress screen im03 may display information on the progress, such as what kind of conversation is being held with the avatar, or a real-time evaluation of the conversation. The type of conversation may be determined by converting the voice of the conversation between the user and the avatar into text, and extracting a vector obtained by vector-converting the converted text from the vector 122 of the text in the storage unit 12 that is close to the vector.

[0035] Furthermore, if the player wishes to restart the conversation with the avatar from the beginning, the player can do so by pressing the retry button B1, and if the player wishes to pause the conversation, the player can do so by pressing the pause button B2. When the role-playing is complete, the player presses the complete button B3. Although not shown, after the role-playing is completed by pressing the complete button B3, a screen may be presented that displays the entire role-playing, including necessary skills, conversation omissions, good points, and areas for improvement, as an evaluation of the entire role-playing.

[0036] In this embodiment, the conversation theme and the conversational approach are stored in advance, and the vector of the conversational text is also stored in advance, so that the time from when the user speaks to when the avatar responds can be shortened in role-playing using an avatar. Also, by storing the voice of the first part of the text of the stored answer text of the conversation, it becomes possible to immediately respond to the user's voice. Furthermore, by storing the time adjustment text 123, even if it takes time to generate an answer, the avatar's speech does not stop unnaturally, and a more natural conversation with the avatar can be held.

[0037] As described above, according to this embodiment, in role-playing using an avatar, it is possible to reduce the time from when a user speaks until the avatar responds.

[0038] Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within the scope that can achieve the object of the present invention are included in the present invention.

[0039] Also, for example, the above-mentioned series of processes can be executed by hardware or software. In other words, the functional configuration is merely an example and is not particularly limited. That is, it is sufficient that the information processing system is provided with a function capable of executing the above-mentioned series of processes as a whole, and there is no particular limit to what kind of functional block is used to realize this function. In addition, the location of the functional block is not particularly limited and may be arbitrary. For example, the functional block of the server may be transferred to a user terminal or the like. Conversely, the functional block of the user terminal may be transferred to a server or the like. In addition, one functional block may be configured by hardware alone, software alone, or a combination thereof.

[0040] For example, when a series of processes is executed by software, a program constituting the software is installed in a computer or the like from a network or a recording medium. The computer may be a computer incorporated in dedicated hardware. The computer may also be a computer capable of executing various functions by installing various programs, such as a server, a general-purpose smartphone, or a personal computer.

[0041] Furthermore, for example, a recording medium containing such a program may not only be constituted by a removable medium (not shown) that is distributed separately from the device main body in order to provide the program to the user, but may also be constituted by a recording medium that is provided to the user in a state in which it is pre-installed in the device main body.

[0042] In this specification, the steps of describing a program to be recorded on a recording medium include not only processes that are performed chronologically according to the order, but also processes that are not necessarily performed chronologically but are executed in parallel or individually. In addition, in this specification, the term "system" refers to an overall device that is composed of a plurality of devices, a plurality of means, etc. [Explanation of symbols]

[0043] 10 Information Processing Systems 1 server 2. User terminal 131 Acquisition Department 132 conversion unit 133 Extraction part 134 generation part 135 speech unit

Claims

1. An information processing method executed by an information processing device, a storage step of vectorizing sentences in a conversation and storing the sentences of the conversation and the vectors of the sentences in correspondence with each other; An acquisition step of acquiring a user's voice; A conversion step of converting the acquired voice into text and vectors; an extraction step of extracting vectors of the text corresponding to the transformed vectors; a generating step of generating an answer to the acquired voice based on a sentence of the conversation corresponding to the extracted sentence vector; a speaking step of speaking the generated answer; Including, In the storing step, a voice of a beginning part of a sentence of the answer in the conversation and a sentence for time adjustment are further stored, generating a remaining answer sentence while uttering a speech of a beginning part of the answer sentence in the conversation corresponding to the vector extracted in the extraction step; If the vector of the extracted sentence in the extraction step is not within a predetermined range from the converted vector, the sentence for time adjustment is spoken in the utterance step, and a new answer is generated using natural language processing in the generation step. Information processing methods.

2. In the storing step, the sentences of the conversation are stored according to a theme; In the extraction step, the theme is determined from the converted vector, and the vector of the sentence is extracted from the determined theme. The information processing method according to claim 1 .

3. When the new answer is generated in the generating step, the converted vector and the vector of the new answer are associated with each other and further stored in the storing step. The information processing method according to claim 1 .

4. In the information processing device, An information processing method executed by an information processing device, a storage step of vectorizing sentences in a conversation and storing the sentences of the conversation and the vectors of the sentences in correspondence with each other; An acquisition step of acquiring a user's voice; A conversion step of converting the acquired voice into text and vectors; an extraction step of extracting vectors of the text corresponding to the transformed vectors; a generating step of generating an answer to the acquired voice based on a sentence of the conversation corresponding to the extracted sentence vector; a speaking step of speaking the generated answer; Including, In the storing step, a voice of a beginning part of a sentence of the answer in the conversation and a sentence for time adjustment are further stored, generating a remaining answer sentence while uttering a speech of a beginning part of the answer sentence in the conversation corresponding to the vector extracted in the extraction step; If the vector of the extracted sentence in the extraction step is not within a predetermined range from the converted vector, the sentence for time adjustment is spoken in the utterance step, and a new answer is generated using natural language processing in the generation step. A program for executing an information processing method.

5. An information processing system executed by an information processing device, An information processing method executed by an information processing device, a storage step of vectorizing sentences in a conversation and storing the sentences of the conversation and the vectors of the sentences in correspondence with each other; An acquisition step of acquiring a user's voice; A conversion step of converting the acquired voice into text and vectors; an extraction step of extracting vectors of the text corresponding to the transformed vectors; a generating step of generating an answer to the acquired voice based on a sentence of the conversation corresponding to the extracted sentence vector; a speaking step of speaking the generated answer; Do the following: In the storing step, a voice of a beginning part of a sentence of the answer in the conversation and a sentence for time adjustment are further stored, generating a remaining answer sentence while uttering a speech of a beginning part of the answer sentence in the conversation corresponding to the vector extracted in the extraction step; If the vector of the extracted sentence in the extraction step is not within a predetermined range from the converted vector, the sentence for time adjustment is spoken in the utterance step, and a new answer is generated using natural language processing in the generation step. Information processing system.

Images