Information processing method, program, and information processing system

Abstract

Role-playing can be evaluated in real time. [Solution] According to one embodiment of an information processing method, the method includes a setting step of setting a scenario, a plurality of stages in the scenario, and conversation sentences in each stage in association with each other, an acquisition step of acquiring a user's voice, a determination step of converting the acquired voice into text and determining whether the text of the voice corresponds to the conversation sentences in the stage set in the setting step, and an evaluation step of evaluating the user's conversation in the stage based on the determination in the determination step.

Description

Technical Field

[0001] The present invention relates to an information processing method, a program, and an information processing system.

Background Art

[0002] Patent Document 1 discloses a training system for supporting the education of professionals who require conversation skills, in which communication AI is implemented.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the case of the above prior art, since evaluation data is generated according to the matching result between the voice analysis result of the user and the evaluation criterion information after the role-playing ends, it cannot be confirmed during the role-playing.

[0005] In consideration of the above facts, an object of the present invention is to obtain an information processing method, a program, and an information processing system capable of evaluating role-playing in real time.

Means for Solving the Problems

[0006] According to one embodiment of an information processing method, the method includes a setting step of setting a scenario, a plurality of stages in the scenario, and conversation sentences in each stage in association with each other, an acquisition step of acquiring a user's voice, a determination step of converting the acquired voice into text and determining whether the text of the voice corresponds to the conversation sentences in the stage set in the setting step, and an evaluation step of evaluating the user's conversation in the stage based on the determination in the determination step.

[0007] According to a program of one embodiment, an information processing device is caused to execute an information processing method including a setting step of setting a scenario, a plurality of stages in the scenario, and conversation sentences in each stage in correspondence with each other, an acquisition step of acquiring a user's voice, a determination step of converting the acquired voice into text and determining whether the text of the voice corresponds to the conversation sentences in the stage set in the setting step, and an evaluation step of evaluating the user's conversation in the stage based on the determination in the determination step.

[0008] According to one embodiment, the information processing system is executed by an information processing device and performs the following steps: a setting step of setting a scenario, a plurality of stages in the scenario, and conversation sentences in each stage in correspondence with each other; an acquisition step of acquiring a user's voice; a determination step of converting the acquired voice into text and determining whether the text of the voice corresponds to the conversation sentences in the stage set in the setting step; and an evaluation step of evaluating the user's conversation in the stage based on the determination in the determination step. Effect of the Invention

[0009] According to one embodiment, role-playing can be assessed in real time. [Brief description of the drawings]

[0010] [Figure 1]FIG. 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a server according to an embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a user terminal according to an embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a functional configuration of a server according to an embodiment. [Diagram 5] FIG. 2 is a diagram illustrating an example of a functional configuration of a user terminal according to an embodiment. [Figure 6] 11 is a flowchart illustrating an example of a process for evaluating a conversation of a user, which is executed by the information processing system according to the embodiment. [Figure 7] 1 is a diagram showing an overview of a role-playing system according to an embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] An embodiment of an information processing system according to the present invention will be described below with reference to Figures 1 to 5. In each drawing, the same or equivalent components and parts are given the same reference numerals. Also, the dimensional ratios in the drawings are exaggerated for the convenience of explanation and may differ from the actual ratios.

[0012] (System Overview) First, an overview of the information processing system 10 according to this embodiment will be described. The information processing system 10 according to this embodiment is a system that provides a role-playing environment and supports human resource training for occupations that require conversation skills. In this embodiment, the information processing system 10 is a system that can evaluate a user's conversation in real time for role-playing using an avatar. This allows the user to check for himself or herself what items the user is talking about and the flow of the conversation.

[0013] (System Configuration) Fig. 1 is a diagram showing an example of a configuration of an information processing system 10 according to the present embodiment. As shown in Fig. 1, the information processing system 10 according to the present embodiment includes a server device 1 and a user terminal 2, which are communicably connected to each other via a network N. The network N is, for example, a wired LAN (Local Area Network), a wireless LAN, the Internet, a public line network, a mobile data communication network, or a combination of these. In the example of Fig. 1, the information processing system 10 includes one server device 1 and one user terminal 2, but may include multiple of each.

[0014] The server device 1 is an example of an information processing device that provides a role-playing environment, generates answers from the user's voice inputted at the user terminal 2, makes an avatar speak the generated answers, and evaluates the user's conversation. The server device 1 may be a PC (Personal Computer), a smartphone, a tablet terminal, a server device, a microcomputer, or a combination of these. The specific configuration and operation of the server device 1 will be described later.

[0015] The user terminal 2 is an example of an information processing device that performs operations for inputting and displaying various information. The user terminal 2 may be a PC (Personal Computer), a smartphone, a tablet terminal, a server device, a microcomputer, a wearable device, or a combination of these. In this embodiment, an information processing device equipped with a voice input function and a photographing function is used as an example, but other functions are not limited as long as the information processing device has the voice input function.

[0016] (Hardware configuration - Server) 2 is a block diagram showing a hardware configuration of the server device 1. The server device 1 includes a processor 101, a memory 102, a storage 103, and a communication I / F 104, which are communicatively connected to each other via a bus B.

[0017] The processor 101 controls each component of the server device 1 and realizes the functions of the server device 1 by expanding various programs stored in the storage 103 into the memory 102 and executing them. The programs executed by the processor 101 include, but are not limited to, an OS (Operating System) and various programs described below. The execution of these programs by the processor 101 realizes a part of the state visualization method according to this embodiment. The processor 101 is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), or a combination of these.

[0018] The memory 102 is, for example, a Read Only Memory (ROM), a Random Access Memory (RAM), or a combination thereof. The ROM is, for example, a Programmable ROM (PROM), an Erasable Programmable ROM (EPROM), an Electrically Erasable Programmable ROM (EEPROM), or a combination thereof. The RAM is, for example, a Dynamic RAM (DRAM), a Static RAM (SRAM), a Magnetoresistive RAM (MRAM), or a combination thereof.

[0019] The storage 103 stores an OS, various programs described below, and various data. The storage 103 is, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), a storage class memory (SCM), or a combination of these.

[0020] The communication I / F 104 is an interface for connecting the server device 1 to an external device including the user terminal 2 via the network N and controlling communication. The communication I / F 104 is, for example, an adapter conforming to Bluetooth (registered trademark), Wi-Fi (registered trademark), ZigBee (registered trademark), Ethernet (registered trademark), or optical communication (for example, Fibre Channel), but is not limited thereto.

[0021] (Hardware configuration - User terminal) 3 is a block diagram showing a hardware configuration of the user terminal 2. The user terminal 2 includes a processor 201, a memory 202, a storage 203, a communication I / F 204, an input / output I / F 205, an input device 206, and an output device 207, which are communicatively connected to each other via a bus B. The communication I / F 204 is an interface for connecting the user terminal 2 to an external device including the server device 1 via a network N and controlling communication. The communication I / F 204 is, for example, an adapter conforming to Bluetooth (registered trademark), Wi-Fi (registered trademark), ZigBee (registered trademark), Ethernet (registered trademark), or optical communication (for example, Fibre Channel), but is not limited thereto.

[0022] The input / output I / F 205 is an interface for connecting an input device 206 and an output device 207 to the user terminal 2. The input device 206 is, for example, a mouse, a keyboard, a touch panel, a microphone, a scanner, a camera, various sensors, an operation button, or a combination of these. The output device 207 as a user interface is, for example, a display, a projector, a printer, a speaker, a vibrator, or a combination of these. In this embodiment, as an example, the output device 207 and the input device 206 are an integrally configured touch panel display.

[0023] In this embodiment, the program may be written to the memory 202 or the storage 203 at the manufacturing stage of the server device 1, or may be provided to the server device 1 via the network N. Further, it may be provided to the server device 1 via a non-transitory computer-readable recording medium such as a disk medium (not shown).

[0024] (Functional Configuration - Server) Next, the functional configuration of the server device 1 will be described. FIG. 4 is a diagram showing an example of the functional configuration of the server device 1. When executing various programs, the server device 1 uses the above-mentioned hardware resources to realize various functions. The server device 1 has a communication unit 11, a storage unit 12, and a control unit 13 as functional configurations realized by the server device 1. Each functional configuration is realized by the processor 101 reading and executing a program stored in the memory 102 or the storage 103. Further, the storage unit 12 stores the role-playing scenario 121, the stage 122 in the scenario, and the evaluation items 123 for each stage in correspondence. A plurality of stages are set along the scenario 121 for the stage 122, and the conversation text and achievement conditions for each stage are also stored. Regarding the achievement conditions for each stage, for example, they may be the content of the conversation text in a specific theme for each stage. The evaluation items 123 include not only the evaluation of whether the stage has been achieved, but also other evaluations (for example, evaluations such as intonation of the voice, timing of speech, expression, etc.).

[0025] The control unit 13 includes a setting unit 131 that sets the stage 122 corresponding to the scenario 121 and the evaluation items 123 in association with each other, and an acquisition unit 132 that acquires the voice sent from the user terminal 2. Further, the control unit 13 includes a conversion unit 133 that converts the voice acquired by the acquisition unit 132 into text, an extraction unit 134 that extracts the text corresponding to the stage 122 from the text converted by the conversion unit 133, and an evaluation unit 135 that evaluates the conversation from the extracted text. Details of these will be described later.

[0026] (Functional Configuration - User Terminal) Next, the functional configuration of the user terminal 2 will be described. Fig. 5 is a diagram showing an example of the functional configuration of the user terminal 2. When executing various programs, the user terminal 2 realizes various functions using the above hardware resources. The user terminal 2 has a communication unit 21, a storage unit 22 in which a program 221 is stored, and a control unit 23 as functional configurations realized by the user terminal 2. Each functional configuration is realized by the processor 201 reading and executing the program 221 stored in the memory 202 or the storage 203. The control unit 23 includes an information acquisition control unit 231 that acquires information sent from the server device 1, and a display unit 232 that displays the acquired information on the output device 207.

[0027] Next, the flow of processing of the information processing system according to this embodiment will be described with reference to Fig. 6 and Fig. 7. Fig. 6 is a flow chart showing an example of a process executed by the information processing system 10 according to this embodiment until an avatar speaks. Fig. 7 is a diagram showing an overview of the role-playing system according to this embodiment.

[0028] First, a process of evaluating a user's conversation will be described with reference to FIG. 6. Specifically, a process of evaluating a user's conversation in real time for each evaluation item in a role-playing game using an avatar will be described. First, in step S101, the setting unit 131 sets a scenario 121, a stage 122 corresponding to the scenario 121, and an evaluation item 123 in association with each other. Next, in step S102, the acquisition unit 132 acquires the user's speech input to the user terminal 2. Next, in step S103, the conversion unit 133 converts the user's speech acquired in step S102 into text. Here, when converting speech into text, for example, any software that can convert speech into text, such as speech recognition software, is sufficient, and is not limited to a specific software. Next, in step S104, the extraction unit 134 analyzes the text converted in step S104, and in step S105, it is determined whether or not the current stage has been achieved among the stages 122 in the storage unit 12. In this case, the converted text may be analyzed using, for example, LLM (Large Language Models), and it is sufficient to analyze the text and determine whether it corresponds to any of the conversation sentences in stage 122 of the memory unit 12.

[0029] In step S105, if the extraction unit 134 judges that the converted text may correspond to any of the stages 122 (YES), the process proceeds to step S106, where the evaluation unit 135 judges that the current stage has been achieved. Then, in step S107, the evaluation unit 135 displays the evaluation result (in this case, the result that the current stage has been achieved) and the next achievement condition on the user terminal 2. Here, the evaluation result to be displayed is displayed so that it is clear whether the stage has been achieved or not. In addition, other evaluation items included in the evaluation item 123 may not be displayed in real time, but may be displayed together, for example, after the role-playing ends. On the other hand, if the analysis result of the converted text indicates that it does not correspond to the current stage and is another stage (i.e., the stage has not been achieved) (NO), the process proceeds to step S108, where the evaluation unit 135 judges that the current stage has not been achieved (not achieved). Then, in step S107, the evaluation unit 135 displays the evaluation result (in this case, the result that the current stage has not been achieved) and the next achievement condition on the user terminal 2.

[0030] In this embodiment, the scenario 121 of the entire role-playing, a plurality of stages 122 in the scenario 121, and the evaluation items 123 are stored in advance, so that the user's conversation can be evaluated in real time for each stage. That is, in the past, the conversation was evaluated by voice recognition after the role-playing was finished, so that the evaluation was not known until the role-playing was finished, but according to this embodiment, the user can be evaluated in real time during the role-playing. This allows the user to check for himself which stage of the entire role-playing scenario he is currently in, what the next stage is, what the achievement conditions are, and the flow of the story. The real-time evaluation is whether or not the stage has been achieved, and other evaluations (e.g., evaluation of the intonation of the voice, the timing of the speech, facial expressions, how long it took, etc.) are evaluated together after the role-playing is finished. However, the evaluation is not limited to this, and may be performed in real time.

[0031] Next, an example of a screen of the role-playing system according to this embodiment is shown with reference to FIG. 7. The screen of the role-playing system has areas A1 and A2, and an avatar screen im01, a user screen im02, and a progress screen im03 are arranged in area A1. A retry button B1, a pause button B2, and a completion button B3 are arranged in area A2. When role-playing starts, the screen shown in FIG. 7 appears, and a conversation with an avatar displayed on the avatar screen im01 begins. During the conversation, the user himself is displayed on the user screen im02. The progress screen im03 may display progress information such as what kind of conversation is being held with the avatar, or a real-time evaluation of the conversation.

[0032] In this embodiment, the progress screen im03 displays which stage the current conversation is at, the next stage, and the conditions for achieving it. There are stages 122 in the scenario 121 stored in advance in the storage unit 12, and in step S105 of FIG. 6, it is determined whether the stage has been achieved. If the current stage is achieved, this is displayed, and the conditions for achieving the next theme are displayed. The conditions for achieving may be set for each stage. In this embodiment, whether each stage has been achieved is evaluated and displayed in real time on the screen, but specific evaluations (e.g., evaluations of voice intonation, speech timing, facial expressions, etc.) are not displayed in real time. However, without being limited to this, for example, a configuration may be adopted in which detailed evaluations are also displayed in real time.

[0033] By presenting the evaluation in real time on the progress screen im03, if the user wants to start the conversation with the avatar over again, the user can do so by pressing the retry button B1. In addition, if the user wants to pause the conversation to check the presented evaluation, the user can pause the conversation by pressing the pause button B2. By presenting the evaluation in real time on the progress screen im03, the user can correct the conversation during the role-playing. Then, when the role-playing is completed, the user presses the complete button B3. Note that the type of button is not limited to this embodiment, and for example, the retry button B1 may not be present. Although not shown, after the completion of the role-playing by pressing the complete button B3, a screen may be presented that displays the entire role-playing, such as the presentation of necessary skills, omissions in the conversation, good points, and improvements, as an evaluation of the entire role-playing.

[0034] In this manner, in this embodiment, by storing evaluation items, the sentences of conversation associated with the evaluation items, and the vectors of the sentences of the conversation in advance, it is possible to evaluate the user's conversation in real time in role-playing using an avatar. This allows the user to check the evaluation in real time, and therefore to proceed with the conversation while making appropriate corrections during the role-playing.

[0035] In this embodiment, the evaluation items, the sentences of the conversation corresponding to the evaluation items, and the vectors of the sentences are stored in advance, and whether or not the evaluation items have been achieved is appropriately evaluated. However, without being limited to this, for example, evaluation may be performed taking into consideration not only the sentences, but also time factors such as the time interval between conversations and the way of taking a pause. In this case, the evaluation may be determined by measuring the time from one speech of the user to the next speech, or the time from one speech of the avatar to the user's speech. In addition, evaluation may be performed based on sentences for adjusting time, such as words for buying time (for example, please wait a moment, etc.).

[0036] As described above, according to this embodiment, the user's conversation can be evaluated in real time, and the user can check for himself / herself the flow of the conversation, such as what item (stage) the user is currently talking about and what the next item (stage) is.

[0037] Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within the scope that can achieve the object of the present invention are included in the present invention.

[0038] Also, for example, the above-mentioned series of processes can be executed by hardware or software. In other words, the functional configuration is merely an example and is not particularly limited. That is, it is sufficient that the information processing system is provided with a function capable of executing the above-mentioned series of processes as a whole, and there is no particular limit to what kind of functional block is used to realize this function. In addition, the location of the functional block is not particularly limited and may be arbitrary. For example, the functional block of the server may be transferred to a user terminal or the like. Conversely, the functional block of the user terminal may be transferred to a server or the like. In addition, one functional block may be configured by hardware alone, software alone, or a combination thereof.

[0039] For example, when a series of processes is executed by software, a program constituting the software is installed in a computer or the like from a network or a recording medium. The computer may be a computer incorporated in dedicated hardware. The computer may also be a computer capable of executing various functions by installing various programs, such as a server, a general-purpose smartphone, or a personal computer.

[0040] Furthermore, for example, a recording medium containing such a program may not only be constituted by a removable medium (not shown) that is distributed separately from the device main body in order to provide the program to the user, but may also be constituted by a recording medium that is provided to the user in a state in which it is pre-installed in the device main body.

[0041] In this specification, the steps of describing a program to be recorded on a recording medium include not only processes that are performed chronologically according to the order, but also processes that are not necessarily performed chronologically but are executed in parallel or individually. In addition, in this specification, the term "system" refers to an overall device that is composed of a plurality of devices, a plurality of means, etc. [Explanation of symbols]

[0042] 10 Information Processing Systems 1 server 2. User terminal 131 Setting Department 132 acquisition department 133 conversion unit 134 Extraction part 135 Evaluation Department

Claims

1. An information processing method executed by an information processing device, a setting step of setting a scenario, a plurality of stages in the scenario, and conversation sentences in each stage in association with each other; An acquisition step of acquiring a user's voice; a determination step of converting the acquired voice into text and determining whether or not the text of the voice corresponds to a sentence of the conversation in the stage set in the setting step; an evaluation step of evaluating the conversation of the user in the stage based on the determination in the determination step; a display step of displaying the result of the evaluation in the evaluation step to a user; Including, In the display step, a current stage of the user, an evaluation result in the evaluation step for the current stage, and an achievement condition for the next stage are displayed. Information processing methods.

2. In the evaluation step, when it is determined in the determination step that the text of the voice corresponds to a sentence of the conversation in the stage set in the setting step, the conversation of the user is evaluated as having achieved the stage; In the display step, the fact that the stage has been achieved is displayed. The information processing method according to claim 1 .

3. In the evaluation step, if the text of the voice does not correspond to any of the sentences of the conversation set in the setting step, the stage is evaluated as not having been reached; In the display step, a display is made to indicate that the stage has not been reached. The information processing method according to claim 1 .

4. In the display step, an evaluation is displayed each time a stage is completed, When the multiple stages in the scenario and the conversations in each stage are all completed, the results of the evaluations in the evaluation steps for all stages are displayed together. The information processing method according to any one of claims 1 to 3.

5. In the information processing device, a setting step of setting a scenario, a plurality of stages in the scenario, and conversation sentences in each stage in association with each other; An acquisition step of acquiring a user's voice; a determination step of converting the acquired voice into text and determining whether or not the text of the voice corresponds to a sentence of the conversation in the stage set in the setting step; an evaluation step of evaluating the conversation of the user in the stage based on the determination in the determination step; a display step of displaying the result of the evaluation in the evaluation step to a user; Including, In the display step, a current stage of the user, an evaluation result in the evaluation step for the current stage, and an achievement condition for the next stage are displayed. A program for executing an information processing method.

6. An information processing system executed by an information processing device, a setting step of setting a scenario, a plurality of stages in the scenario, and conversation sentences in each stage in association with each other; An acquisition step of acquiring a user's voice; a determination step of converting the acquired voice into text and determining whether or not the text of the voice corresponds to a sentence of the conversation in the stage set in the setting step; an evaluation step of evaluating the conversation of the user in the stage based on the determination in the determination step; a display step of displaying the result of the evaluation in the evaluation step to a user; Do In the display step, a current stage of the user, an evaluation result in the evaluation step for the current stage, and an achievement condition for the next stage are displayed. Information processing system.

Images