Information processing apparatus, information processing method, and computer program

Abstract

To enable interactions during a lecture in a virtual space.SOLUTION: An information processing apparatus which provides a lecture in a virtual space includes an interaction processing unit which enables two-way communications between a person who provides a lecture and an auditor. The interaction processing unit executes at least one of auditor question / answer processing and system question issuing processing. The auditor question / answer processing is a process of receiving an auditor question, referring to a QA database that specifies combinations of assumed questions and assumed answers related to the contents of a lecture, selecting an assumed answer having a high degree of association with the auditor question, the assumed answer being associated with an assumed question having a high degree of association with the auditor question, and outputting the selected assumed answer so as to be perceived by the auditor. The system question issuing processing is a process of creating a system question whose correct answer is a matter related to the contents of the lecture, and outputs the system question so as to be perceived by the auditor.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The technology disclosed in this specification relates to information processing for realizing lectures in a virtual space.

Background Art

[0002] In recent years, online lectures have been implemented in educational institutions such as universities. Also, in online lectures, in order to enable students to take lectures in a situation similar to face-to-face lectures, for example, technologies for realizing lectures in a virtual space using VR (Virtual Reality) have been proposed (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] While online lectures have the advantage that listeners can choose their learning environment, there is a problem in that it is difficult to conduct two-way interactions such as question-and-answer sessions and comprehension checks, and there is an issue in improving the learning motivation of the listeners. Such problems still exist even when lectures in a virtual space are adopted.

Means for Solving the Problems

[0005] The technology disclosed in this specification can be realized, for example, in the following forms.

[0006] (1) The information processing apparatus disclosed in this specification is an apparatus for realizing a lecture in a virtual space, and includes an interaction processing unit that realizes a two-way interaction between the provider of the lecture and the listener. The interaction processing unit executes at least one of a listener question-and-answer process and a system question issuance process. The listener question-and-answer process receives a listener question, which is a question from the listener, refers to a QA database that defines a combination of assumed questions and assumed answers regarding the content of the lecture, and selects an assumed answer that is associated with the assumed question having a high similarity to the listener question and has a high relevance to the listener question, and outputs the selected assumed answer in a perceptible manner to the listener. The system question issuance process is a process of creating a system question, which is a question having an item regarding the content of the lecture as the correct answer, and outputting the system question in a perceptible manner to the listener.

[0007] Thus, this information processing apparatus includes an interaction processing unit that executes at least one of a listener question-and-answer process that receives a question from the listener and outputs an answer to the listener, and a system question issuance process that creates a system question and outputs it to the listener. Therefore, during a lecture in a virtual space, a two-way interaction between the lecture provider and the listener can be realized, and an improvement in the learning motivation of the learner can be realized.

[0008] (2) In the above information processing apparatus, the interaction processing unit may execute at least the system question issuance process, and the system question issuance process may include a process of receiving a listener answer, which is an answer from the listener to the system question. By adopting this configuration, during a lecture in a virtual space, a more effective interaction of issuing a system question and receiving a listener answer can be realized, and a further improvement in the learning motivation of the learner can be realized.

[0009] (3) In the above information processing apparatus, the system question issuance process may be configured to include a process of outputting, in a manner perceptible to the listener, an evaluation based on the listener's answer and the correct answer. By adopting this configuration, during the lecture in the virtual space, it is possible to realize a more effective interaction including the issuance of system questions, the reception of the listener's answers, and the output of the evaluation of the listener's answers, and it is possible to further improve the learning motivation of the attendees.

[0010] (4) In the above information processing apparatus, the interaction processing unit executes at least the system question issuance process, and the interaction processing unit may be configured to select terms with high importance from the terms included in the lecture materials and create the system questions with the selected terms as the correct answers. By adopting this configuration, it is possible to issue system questions in accordance with the lecture content and realize an improvement in the understanding degree of the lecture.

[0011] (5) In the above information processing apparatus, the interaction processing unit may be configured to create a summary including the correct answer based on the lecture transcript and create the system questions using the correct answer and the summary. By adopting this configuration, it is possible to issue system questions that more conform to the lecture content and realize a further improvement in the understanding degree of the lecture.

[0012] (6) In the above information processing apparatus, the interaction processing unit may be configured to verify the validity of the system questions based on the degree of coincidence between a first predicted answer predicted based on the system questions and the transcript and a second predicted answer predicted based on the system questions and the summary, and output the system questions with high validity. By adopting this configuration, it is possible to issue system questions with higher validity and realize a further improvement in the understanding degree of the lecture.

[0013] (7) In the information processing apparatus, the interaction processing unit executes at least the lecture attendee question-and-answer processing. The interaction processing unit may be configured to select the assumed questions whose similarity to the lecture attendee questions is higher than a predetermined threshold value, and among the assumed answers associated with the selected assumed questions, select the assumed answers whose relevance to the lecture attendee questions is higher than a predetermined threshold value. By adopting this configuration, it is possible to select and output answers that more appropriately answer lecture attendee questions, realize smoother question-and-answer sessions, and further improve the understanding level of the lecture.

[0014] (8) In the information processing apparatus, the interaction processing unit executes at least the lecture attendee question-and-answer processing. The interaction processing unit may be configured to select the assumed answers based on a predetermined index value calculated from the similarity between the lecture attendee questions and the relevance between the lecture attendee questions. By adopting this configuration, it is possible to select and output answers that more appropriately answer lecture attendee questions, realize smoother question-and-answer sessions, and further improve the understanding level of the lecture.

[0015] Note that the technology disclosed in this specification can be realized in various forms. For example, it can be realized in the form of an information processing apparatus, an information processing method, a computer program for realizing the method, a non-transitory recording medium on which the computer program is recorded, and the like.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Mode for Carrying Out the Invention

[0017] A. Embodiment: A-1. Configuration of the VR lecture system 10: FIG. 1 is an explanatory diagram schematically showing the configuration of the VR lecture system 10 in this embodiment. The VR lecture system 10 of this embodiment is a system that enables a listener (user) U to take an online lecture in a virtual space using VR (virtual reality) technology. The VR lecture system 10 of this embodiment provides an on-demand VR lecture that can be taken by the listener U at any time and at any place. In this specification, the "lecture" is not limited to lectures in educational institutions such as universities, but means any communication in which a lecture provider (lecturer, speaker) and a listener exist and a two-way interaction is expected between the two, including, for example, seminars, training sessions, briefing sessions, symposiums, etc. Also, in this specification, the lecture provider in the VR lecture system 10 is also referred to as the "system" for convenience.

[0018] The VR lecture system 10 includes an information processing device 100 which is a device on the lecturer side, and a head-mounted display (hereinafter referred to as "HMD") 200 which is a device on the listener side. Each device constituting the VR lecture system 10 is communicably connected to each other via a network NET such as the Internet.

[0019] The HMD 200 is a device that allows the listener U to view an image while being worn on the head of the listener U. In the present embodiment, the HMD 200 is a non-transmissive HMD (so-called VR goggles) that completely covers both eyes of the listener U. Further, the HMD 200 has a microphone and a speaker (not shown) and can perform input and output of sound. Based on the data provided from the information processing device 100, the HMD 200 allows the listener U to view a VR lecture image VRI which is an image of the VR lecture and to perceive the lecture sound, thereby providing the listener U with a VR lecture experience via vision and hearing. As a result, the listener U can take an online lecture in a situation similar to a face-to-face lecture. In this specification, for convenience, it is also said that the HMD 200 displays an image when allowing the listener U to view an image.

[0020] FIG. 2 is an explanatory diagram showing an example of the VR lecture image VRI. In the example shown in FIG. 2, the VR lecture image VRI includes an image of a lecture slide SL, an image of a lecturer avatar AV which is an avatar of the lecture provider, and an image of a user interface UI. During the VR lecture, an image of the lecture slide SL showing the content of the lecture is displayed in the VR lecture image VRI, and the lecturer avatar AV explains in voice or points to the explanatory part in the lecture slide SL.

[0021] In addition, the listener U of the VR lecture can perform various operations (such as playing, pausing, changing the playback speed, fast-forwarding, rewinding, and displaying notes of the VR lecture) by operating the user interface UI. Also, as will be described later, when the listener U of the VR lecture has, for example, unclear points or questions, the listener U can ask the system a question by selecting the question button B1 included in the user interface UI.

[0022] Figure 3 is an explanatory diagram showing the schematic configuration of the information processing apparatus 100. The information processing apparatus 100 is an apparatus for providing a VR lecture to the listener U and is configured by a computer (such as a PC or a server). The information processing apparatus 100 includes a control unit 110, a storage unit 120, a display unit 130, an operation input unit 140, and an interface unit 150. These units are connected to each other via a bus 190 so as to be communicable with each other.

[0023] The display unit 130 of the information processing apparatus 100 is configured by, for example, a liquid crystal display or the like and displays various images and information. The operation input unit 140 is configured by, for example, a keyboard, a mouse, buttons, a microphone, a track pad, or the like and receives operations and instructions from the administrator. Note that the display unit 130 may function as the operation input unit 140 by including a touch panel. The interface unit 150 is configured by, for example, a LAN interface or a USB interface or the like and communicates with other devices by wire or wirelessly. Note that the information processing apparatus 100 may include a speaker as voice output means and a microphone as voice input means.

[0024] The storage unit 120 of the information processing apparatus 100 is composed of, for example, a ROM, a RAM, a hard disk drive (HDD), etc., and stores various programs and data, or is used as a working area or a temporary storage area for data when executing various programs. For example, the storage unit 120 stores a VR lecture program CP, which is a computer program for executing the listener question-and-answer process and the system question-issuing process described later. The VR lecture program CP is provided, for example, in a state stored in a computer-readable recording medium (not shown) such as a CD-ROM, a DVD-ROM, or a USB memory, or is provided in a state that can be acquired from an external device (for example, a server on the cloud or another terminal device) via the interface unit 150, and is stored in the storage unit 120 in an operable state on the information processing apparatus 100.

[0025] Also, the storage unit 120 of the information processing apparatus 100 stores lecture slide data LD, transcript data TD, and a QA database DB in advance. The lecture slide data LD is data of a lecture slide SL prepared in advance. The transcript data TD is text data and / or audio data representing the transcribed text of the VR lecture. The QA database DB will be described in accordance with the description of the listener question-and-answer process described later.

[0026] The control unit 110 of the information processing apparatus 100 is configured by, for example, a CPU or the like, and controls the operation of the information processing apparatus 100 by executing a computer program read from the storage unit 120. For example, the control unit 110 functions as a VR lecture processing unit 111 for realizing the above-described VR lecture by reading and executing the VR lecture program CP from the storage unit 120. For example, the VR lecture processing unit 111 causes the HMD 200 to display a VR lecture image VRI including an image of a lecture slide SL based on the lecture slide data LD or reproduce the voice of the lecture on the HMD 200 in response to a request from the HMD 200. In the present embodiment, the VR lecture processing unit 111 includes an interaction processing unit 112 including a listener question and answer processing unit 113 and a system question issuance processing unit 114. The functions of these units will be described in accordance with the description of various processes described later.

[0027] A-2. Listener Question and Answer Processing: Next, the listener question and answer processing executed in the information processing apparatus 100 of the present embodiment will be described. FIG. 4 is a flowchart showing the listener question and answer processing in the present embodiment. The listener question and answer processing is a process of presenting an answer to a question to the listener U on the system side when the listener U listening to the VR lecture asks a question to the system. The listener question and answer processing is started, for example, when the listener U selects a question button B1 (see FIG. 2) included in the user interface UI of the VR lecture image VRI at an arbitrary timing during the lecture.

[0028] First, the lecturer Q&A processing unit 113 (Fig. 3) of the information processing apparatus 100 receives a question from the lecturer U (hereinafter referred to as the "lecturer question") (S110). More specifically, when the lecturer Q&A processing unit 113 receives information indicating that the question button B1 has been selected by the lecturer U from the HMD 200, as shown in Fig. 5, a first dialogue box D1 including a message prompting the input of a question (for example, a message such as "Please ask a question") is displayed on the VR lecture image VRI. More specifically, the lecturer Q&A processing unit 113 updates the VR lecture image VRI to include the first dialogue box D1. This also applies to the following image display (image update). After the first dialogue box D1 is displayed, when the lecturer U inputs a question (for example, a question such as "What is self-training for conversation?") by voice, the lecturer Q&A processing unit 113 generates text data of the question by performing voice recognition and displays the content of the question in the first dialogue box D1. For convenience, Fig. 5 shows a state in which the first dialogue box D1 includes a message prompting question input and a question by the lecturer U. However, naturally, the question by the lecturer U is not included at the timing when the first dialogue box D1 is first displayed. This also applies to the fourth dialogue box D4 in Fig. 11 described later. Also, the input of a question by the lecturer U may be performed via other means such as a keyboard.

[0029] Next, the listener question and answer processing unit 113 refers to the QA database DB and calculates the similarity between the listener's question and the assumed questions registered in the QA database DB (hereinafter referred to as "question-question similarity") (S120). As shown in FIG. 6, the QA database DB is a database that defines a plurality of combinations of assumed questions and assumed answers regarding the content of the VR lecture. For example, in the example of FIG. 6, assumed answers A1, A2, A3, A4, A5, A6,... are associated with assumed questions Q1, Q2, Q3, Q4, Q5, Q6,... respectively. The QA database DB is created in advance by the lecture provider and stored in the storage unit 120. The listener question and answer processing unit 113 calculates the similarity between the listener's question and each of the plurality of assumed questions registered in the QA database DB. The calculation of the question-question similarity can be executed by any method. For example, a model obtained by fine-tuning RoBERTa, which is an encoder model used for text classification, etc., with similar question pairs in the QA database DB is created, and the listener's question and the assumed question are input to the model for calculation.

[0030] Next, the listener question and answer processing unit 113 selects the assumed answers associated with the assumed questions whose question-question similarity is equal to or greater than a predetermined threshold as candidate answers (S130). For example, in the example shown in FIG. 6, the assumed answers (A1, A3, A6) associated with the assumed questions (Q1, Q3, Q6) whose question-question similarity is 0.7 or more are selected as candidate answers.

[0031] Next, the listener question and answer processing unit 113 calculates the relevance between the candidate answers and the listener's question (hereinafter referred to as "question-answer relevance") (S140). The calculation of the question-answer relevance can be executed by any method. For example, a model obtained by fine-tuning RoBERTa with a large-scale QA corpus is created, and the listener's question and the candidate answers are input to the model for calculation.

[0032] Next, the listener question and answer processing unit 113 selects the candidate answer with the highest relevance between the question and the answer among the candidate answers, and outputs it in a perceptible manner to the listener U (S150). For example, in the example shown in FIG. 6, among the candidate assumed answers (A1, A3, A6), answer A1, which is the answer with the highest relevance between the question and the answer, is selected. As shown in FIG. 7, the listener question and answer processing unit 113 displays a second dialogue box D2 including an answer to the listener's question (for example, an answer such as "Practice interacting with the AI avatar for customer service, etc.") on the VR lecture image VRI. Thereby, the listener U can obtain an answer from the system to the question issued to the system.

[0033] In the example shown in FIG. 7, the second dialogue box D2 includes a close button B2 and a re-question button B3. When the listener U's doubts are resolved by the answer presented by the system, the listener U can select the close button B2 and return to watching the lecture. Also, when the listener U's doubts are not resolved by the answer presented by the system or new doubts arise, the listener U can select the re-question button B3 and input a new question.

[0034] If there is no assumed question whose question similarity calculated in S120 is equal to or higher than a predetermined threshold, and / or if there is no assumed answer whose relevance between the question and the answer calculated in S140 is equal to or higher than a predetermined threshold, the listener question and answer processing unit 113 may determine that it is impossible to give an appropriate answer to the listener's question and perform a preset unable-to-answer process. The unable-to-answer process may be, for example, a process of outputting a message such as "I can't answer here, so I will answer again at a later date."

[0035] Also, in S150, instead of the process of selecting the candidate answer with the highest relevance between the question and the answer, a process of selecting a candidate answer based on a predetermined index value calculated from the question similarity and the relevance between the question and the answer may be executed. Examples of such a process include a process of selecting the candidate answer with the highest weighted average of both.

[0036] A-3. System Question Issuance Process: Next, the system question issuance process executed by the information processing apparatus 100 of the present embodiment will be described. FIG. 8 is a flowchart showing the system question issuance process in the present embodiment. The system question issuance process is a process of creating a question regarding the content of a lecture (hereinafter referred to as a "system question") and outputting the system question to the listener U who is attending the VR lecture. The system question issuance process is executed, for example, to confirm to what extent the listener U understands the content of the lecture. Each step of the system question issuance process is executed in advance before the listener U listens to the VR lecture, or at a predetermined timing during the listener U's listening to the VR lecture.

[0037] First, the system question issuance processing unit 114 (FIG. 3) of the information processing apparatus 100 creates an answer candidate list in which candidates for answers (correct answers) to the system questions are listed (S210). The creation of the answer candidate list can be performed by any method. For example, keywords are extracted from the lecture slide SL using a keyword automatic extraction tool (e.g., termextract), the importance of each keyword is calculated based on the IDF (Inverse Document Frequency) dictionary of words created from the Japanese Wikipedia text, and keywords with high importance (high IDF) are selected as answer candidates. The upper part of FIG. 9 shows an example in which the keyword KW "educational content" in the lecture slide SL is selected as an answer candidate. By selecting answer candidates in this way, appropriate keywords KW for confirming the degree of understanding of the content of the VR lecture are selected as answer candidates.

[0038] Next, the system question issuance processing unit 114 creates system questions corresponding to the answer candidates included in the answer candidate list (S220). The creation of system questions can be performed by any method. For example, it is executed by the following method using GPT-3, which is a learning model that generates text based on descriptions (prompts) such as explanations and conditions regarding the text to be generated, and T5, which is an encoder-decoder model used for summarization, question generation, etc. First, a prompt describing to create a summary containing the words included in the answer candidate list obtained in S210 from the transcript is input to GPT-3 to generate a summary of the lecture content described for the answer candidates included in the answer candidate list. Then, each answer candidate is input to T5 together with the summary to generate a system question such that the input answer candidate becomes the correct answer. The middle to lower part of FIG. 9 shows an example in which a summary SU including the keyword KW "educational content" is generated based on the lecture transcript TR, and a system question QE with the keyword KW "educational content" as the correct answer is generated based on the summary SU.

[0039] Note that the system question issuance processing unit 114 may execute a process of verifying the validity of the generated system question. The verification of the validity of the system question can be performed by any method. For example, it is executed by the following method using BERT, which is an encoder model used for expression, relation extraction, etc. That is, the system question and the transcript are input to BERT to obtain a first predicted answer extracted from the transcript. Also, the system question and the summary are input to BERT to obtain a second predicted answer extracted from the summary. Then, when the first predicted answer obtained from the transcript and the second predicted answer obtained from the summary match, it is determined that the generated system question is valid, and when the two do not match, it is determined that the generated system question is not valid. When it is determined that the system question is not valid, the system question issuance processing unit 114 either creates a system question again or aborts the system question issuance process.

[0040] Next, the system question issuing processing unit 114 outputs a system question in a perceptible manner by the listener U at a predetermined timing (S230). For example, as shown in FIG. 10, the system question issuing processing unit 114 displays a third dialogue box D3 including a system question (for example, a question such as "What is the place where the digital campus creates and transmits something?") on the VR lecture image VRI. The third dialogue box D3 includes an answer button B4 for the listener U to give an answer. The output timing of the system question can be arbitrarily set. For example, it can be the timing of the break in the content of the lecture or the timing when the listener U executes some selection via the user interface UI.

[0041] Next, the system question issuing processing unit 114 receives an answer from the listener U (hereinafter referred to as "listener answer") (S240). More specifically, when the answer button B4 included in the above-described third dialogue box D3 is selected, the system question issuing processing unit 114, as shown in FIG. 11, displays a fourth dialogue box D4 including a message (for example, a message such as "Please speak the answer") prompting the input of the answer to the system question on the VR lecture image VRI. In response to this, when the listener U inputs an answer (for example, an answer such as "educational content") by voice, the system question issuing processing unit 114 generates text data of the answer by performing voice recognition and displays the answer in the fourth dialogue box D4.

[0042] Next, the system question issuing processing unit 114 evaluates the received answer (for example, determines whether it is a correct answer or an incorrect answer) and outputs the evaluation result (S250). For example, the system question issuing processing unit 114 determines whether the received answer matches the correct answer associated with the system question and outputs the result. As shown in FIG. 12, the system question issuing processing unit 114 displays a fifth dialogue box D5 including an evaluation (for example, an evaluation such as "Correct answer") on the VR lecture image VRI. Thereby, the listener U can recognize whether his / her answer to the system question is correct.

[0043] A-4. Effects of this embodiment: As described above, the information processing apparatus 100 of the present embodiment is an apparatus that realizes a lecture in a virtual space, and includes an interaction processing unit 112 that realizes a two-way interaction between the lecture provider and the listener U. The interaction processing unit 112 executes a listener question-and-answer process. The listener question-and-answer process receives a listener question, which is a question from the listener U, refers to a QA database DB that defines combinations of assumed questions and assumed answers regarding the content of the lecture, and selects an assumed answer that is associated with an assumed question having a high similarity to the listener question and has a high relevance to the listener question, and outputs the selected assumed answer in a perceptible manner to the listener U.

[0044] As described above, the information processing apparatus 100 of the present embodiment includes an interaction processing unit 112 that executes a listener question-and-answer process for receiving a listener question and outputting an answer. Therefore, during a lecture in a virtual space, it is possible to realize a question-and-answer session, which is one of the two-way interactions (interactions) between the lecture provider and the listener U, and it is possible to improve the learning motivation of the learners.

[0045] In addition, the interaction processing unit 112 selects an assumed question whose similarity to the listener question is higher than a predetermined threshold value, and among the assumed answers associated with the selected assumed question, selects an assumed answer whose relevance to the listener question is higher than a predetermined threshold value. Therefore, it is possible to select and output an answer that answers the listener question more appropriately, realize a smoother question-and-answer session, and further improve the understanding degree of the lecture.

[0046] In addition, the interaction processing unit 112 may select an assumed answer based on a predetermined index value calculated from the similarity between the listener question and the relevance to the listener question. Even in this way, it is possible to select and output an answer that answers the listener question more appropriately, realize a smoother question-and-answer session, and further improve the understanding degree of the lecture.

[0047] In addition, the interaction processing unit 112 executes system question issuance processing. The system question issuance processing is processing for creating a system question, which is a question with an answer regarding matters related to the content of the lecture, and outputting the system question in a perceptible manner to the listener U. As described above, the information processing apparatus 100 of the present embodiment includes the interaction processing unit 112 that executes the system question issuance processing for creating and outputting a system question. Therefore, during the lecture in the virtual space, it is possible to realize the issuance of a system question, which is one of the two-way exchanges (interactions) between the lecture provider and the listener U, and it is possible to improve the learning motivation of the attendees.

[0048] In addition, the system question issuance processing includes processing for receiving a listener answer, which is an answer from the listener U to the system question. Therefore, during the lecture in the virtual space, it is possible to realize a more effective interaction of issuing a system question and receiving a listener answer, and it is possible to further improve the learning motivation of the attendees.

[0049] In addition, the system question issuance processing includes processing for outputting, in a perceptible manner to the listener U, an evaluation based on the listener answer and the correct answer. Therefore, during the lecture in the virtual space, it is possible to realize a more effective interaction of issuing a system question, receiving a listener answer, and outputting an evaluation of the listener answer, and it is possible to further improve the learning motivation of the attendees.

[0050] In addition, the interaction processing unit 112 selects terms with high importance from the terms included in the lecture materials, and creates a system question with the selected terms as the correct answer. Therefore, it is possible to issue a system question in accordance with the lecture content, and it is possible to improve the understanding of the lecture.

[0051] In addition, the interaction processing unit 112 creates a summary including the correct answer based on the lecture transcript, and creates a system question using the correct answer and the summary. Therefore, it is possible to issue a system question that more conforms to the lecture content, and it is possible to further improve the understanding of the lecture.

[0052] In addition, based on the degree of coincidence between the first predicted answer predicted based on the system question and the transcript and the second predicted answer predicted based on the system question and the summary, the interaction processing unit 112 verifies the validity of the system question and outputs a highly valid system question. Therefore, a more valid system question can be issued, and a further improvement in the understanding degree of the lecture can be realized.

[0053] A-5. Evaluation: Evaluation was conducted on the above-described listener question-and-answer processing and system question issuance processing. FIG. 13 is an explanatory diagram showing the evaluation results. Specifically, the state of one listener U using each function of the VR lecture was recorded, and evaluations for the video were collected. In the evaluation, for the following four systems, four types of videos of the viewers' screens were prepared, and 12 workers evaluated each video using crowdsourcing. (1) First system S1 that can execute neither listener question-and-answer processing nor system question issuance processing (2) Second system S2 that can execute listener question-and-answer processing but cannot execute system question issuance processing (3) Third system S3 that can execute system question issuance processing but cannot execute listener question-and-answer processing (4) Fourth system S4 that can execute both listener question-and-answer processing and system question issuance processing

[0054] The evaluation items were the following seven items Q1 to Q7, and the evaluation was performed on a five-point scale from 1 to 5 (the higher the numerical value, the higher the evaluation). Also, for the statistical test, Steel-Dwass multiple comparison was used. Q1: Naturalness of the lecture Q2: Naturalness of the conversation Q3: Satisfaction with the conversation Q4: Concentration on the lecture Q5: Interest in the lecture content Q6: Understanding of the lecture content Q7: Overall satisfaction

[0055] As shown in FIG. 13, in the evaluation of Q1 regarding the naturalness of the lecture, since there was no significant difference in the results among any of the systems, it was confirmed that the system of the present embodiment does not impair the naturalness of the lecture. Regarding Q2 and Q3 regarding the interaction with the system, the evaluation of the fourth system S4 capable of executing both the listener question-and-answer process and the system question-issuing process was the highest, and a significant difference was recognized between this and the first system S1 that could not execute any of them. From this, it was confirmed that by making it possible to execute the listener question-and-answer process and the system question-issuing process, a more natural interaction is possible in the VR lecture. Regarding Q6 regarding the understanding of the lecture content, the evaluation of the fourth system S4 was the highest, and a significant difference was recognized between this and the first system S1. On the other hand, regarding Q4 and Q5 regarding the learning motivation, there was no significant difference among any of the systems. Referring to the above evaluations, it was confirmed that in the lecture in the virtual space, by making it possible to execute the listener question-and-answer process and / or the system question-issuing process, it is possible to expect an improvement in the understanding of the lecture content while maintaining the learning motivation.

[0056] B. Modification example: The technology disclosed in this specification is not limited to the above-described embodiment, and can be modified into various forms without departing from the gist thereof. For example, the following modifications are also possible.

[0057] The configurations of the VR lecture system 10 and the information processing apparatus 100 in the above-described embodiment are merely examples, and can be variously modified. For example, in the above-described embodiment, the interaction processing unit 112 includes the listener question-and-answer processing unit 113 and the system question-issuing processing unit 114, but the interaction processing unit 112 may not include one of them. When the interaction processing unit 112 includes the listener question-and-answer processing unit 113 and does not include the system question-issuing processing unit 114, the interaction processing unit 112 executes listener question-and-answer processing but does not execute system question-issuing processing. Conversely, when the interaction processing unit 112 includes the system question-issuing processing unit 114 and does not include the listener question-and-answer processing unit 113, the interaction processing unit 112 executes system question-issuing processing but does not execute listener question-and-answer processing.

[0058] In the above-described embodiment, the information processing apparatus 100 itself has the QA database DB, but the information processing apparatus 100 may not have the QA database DB and may refer to the QA database DB possessed by another apparatus.

[0059] A part of the components of the information processing apparatus 100 in the above-described embodiment may be provided in the HMD 200 instead of the information processing apparatus 100. Further, the apparatus used by the listener U is not limited to the HMD 200, and any other apparatus capable of attending a lecture in the virtual space may be used.

[0060] The content of the listener question-and-answer processing in the above-described embodiment is merely an example, and can be variously modified. For example, in the above-described embodiment, an answer is selected based on the similarity between questions and the relevance between question and answer, but an answer may be selected based on either the similarity between questions or the relevance between question and answer. Further, when selecting an answer, other indicators other than these two may be referred to. Further, the output form of the answer is not limited to the display of an image, and may be output in another form (for example, output by voice) as long as it is perceivable by the listener U.

[0061] The content of the system question issuance process in the above embodiment is merely an example and can be variously modified. For example, in the above embodiment, after the system question is output, reception of the listener's answer (S240 in FIG. 8) and output of the evaluation of the listener's answer (S250 in the same figure) are executed, but at least one of these processes may be omitted. Also, the output form of the system question and the evaluation result is not limited to the display of an image, and may be output in other forms (for example, output by voice) as long as it is perceivable by the listener U.

[0062] In the above embodiment, the VR lecture system 10 for realizing a VR lecture has been described. However, the technology disclosed in this specification is not limited to this, and is similarly applicable to a system and / or apparatus for realizing a lecture in a virtual space.

[0063] In the above embodiment, a part of the configuration realized by hardware may be replaced with software, and conversely, a part of the configuration realized by software may be replaced with hardware.

Explanation of Reference Numerals

[0064] 10: VR lecture system 100: Information processing apparatus 110: Control unit 111: VR lecture processing unit 112: Interaction processing unit 113: Listener question and answer processing unit 114: System question issuance processing unit 120: Storage unit 130: Display unit 140: Operation input unit 150: Interface unit 190: Bus 200: HMD

Claims

1. An information processing apparatus for realizing a lecture in a virtual space, comprising an interaction processing unit that realizes a two-way interaction between a provider and a listener of the lecture, wherein the interaction processing unit receives a listener question that is a question from the listener, refers to a QA database that defines a combination of an assumed question and an assumed answer regarding the content of the lecture, and selects the assumed answer that is associated with the assumed question having a high similarity to the listener question and has a high relevance to the listener question, and outputs the selected assumed answer in a perceptible manner to the listener; listener question and answer processing; creates a system question that is a question having an item regarding the content of the lecture as a correct answer, and outputs the system question in a perceptible manner to the listener; system question issuing processing; An information processing apparatus that executes at least one of the above.

2. The information processing apparatus according to claim 1, wherein the interaction processing unit executes at least the system question issuing processing, and the system question issuing processing includes a process of receiving a listener answer that is an answer from the listener to the system question. An information processing apparatus.

3. The information processing apparatus according to claim 2, wherein the system question issuing processing includes a process of outputting, in a perceptible manner to the listener, an evaluation based on the listener answer and the correct answer. An information processing apparatus.

4. The information processing apparatus according to claim 1, wherein the interaction processing unit executes at least the system question issuing processing, and the interaction processing unit selects a term with a high degree of importance from the terms included in the lecture materials, and creates a system question having the selected term as the correct answer. An information processing apparatus.

5. The information processing apparatus according to claim 4, wherein the interaction processing unit creates a summary including the correct answer based on the lecture transcript, and creates the system question using the correct answer and the summary. An information processing apparatus.

6. The information processing apparatus according to claim 5, wherein the interaction processing unit verifies the validity of the system question based on the degree of coincidence between a first predicted answer predicted based on the system question and the transcript and a second predicted answer predicted based on the system question and the summary, and outputs the system question with high validity. An information processing apparatus.

7. The information processing apparatus according to claim 1, wherein the interaction processing unit executes at least the lecture attendee question-and-answer processing, wherein the interaction processing unit selects the assumed question having a similarity to the lecture attendee question higher than a predetermined threshold value, and selects, from among the assumed answers associated with the selected assumed question, the assumed answer having a relevance to the lecture attendee question higher than a predetermined threshold value, the information processing apparatus.

8. The information processing apparatus according to claim 1, wherein the interaction processing unit executes at least the lecture attendee question-and-answer processing, wherein the interaction processing unit selects the assumed answer based on a predetermined index value calculated from the similarity to the lecture attendee question and the relevance to the lecture attendee question, the information processing apparatus.

9. An information processing method for realizing a lecture in a virtual space, comprising an interaction processing step for realizing a two-way interaction between a provider of the lecture and a lecture attendee, wherein the interaction processing step, receives a lecture attendee question which is a question from the lecture attendee, refers to a QA database defining a combination of an assumed question and an assumed answer regarding the content of the lecture, selects the assumed answer associated with the assumed question having a high similarity to the lecture attendee question and having a high relevance to the lecture attendee question, and outputs the selected assumed answer in a perceptible manner to the lecture attendee, the step of performing lecture attendee question-and-answer processing; creates a system question which is a question having an item regarding the content of the lecture as a correct answer, and outputs the system question in a perceptible manner to the lecture attendee, the step of performing system question issuance processing; and includes at least one of the above, the information processing method.

10. A computer program for performing information processing for realizing a lecture in a virtual space, causing a computer to execute interaction processing for realizing a two-way interaction between a provider of the lecture and a lecture attendee, wherein the interaction processing, receives a lecture attendee question which is a question from the lecture attendee, refers to a QA database defining a combination of an assumed question and an assumed answer regarding the content of the lecture, selects the assumed answer associated with the assumed question having a high similarity to the lecture attendee question and having a high relevance to the lecture attendee question, and outputs the selected assumed answer in a perceptible manner to the lecture attendee, the lecture attendee question-and-answer processing; Create a system question that is a question with an answer regarding the content of the lecture, and output the system question in a perceptible manner by the listener, and a system question issuance process; A computer program including at least one of the above.

Images