Video learning support device, information generation device, video learning support system, and program

Abstract

To easily check whether a learning activity is appropriately being performed according to an aim of an instruction for each scene in group learning using a video as a material.SOLUTION: A video learning support device includes a video reproduction unit, an information acquisition unit, and a learning status determination unit. The video reproduction unit reproduces a video content used for learning according to a reproduction control instruction by a group of learners. The information acquisition unit obtains, as needed, group activity information indicating information relating to a discussion that the group is performing while viewing the video content. The learning status determination unit generates, as needed, learning status information indicating a status of whether the discussion of the group is being performed according to an instruction point of the learning according to a reproduction point of the video content, based on both instruction point list including information of the instruction point and the group activity information obtained by the information acquisition unit.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a video learning support device, an information generating device, a video learning support system, and a program. [Background technology]

[0002] In recent school education, many learning methods are being implemented, such as comprehensive learning classes, that encourage independent learning through discussions in groups of several people. However, it is difficult for an instructor (teacher) to grasp the situation of multiple groups at once, and it is also difficult to see the individual opinions and levels of understanding of the learners (students). Therefore, in collaborative learning (group learning) in which multiple students discuss with each other, there is a technology that makes it easier for teachers to grasp the progress of group discussions by quantifying and showing the students' thought processes from the content of their speech (see, for example, Patent Document 1).

[0003] Furthermore, with the use of ICT (information and communication technology) in education, the number of lessons using video content is also increasing. To grasp the learning situation of a learner, there is a technology that detects the position touched by a fingertip or pen tip on the image of learning materials shown on a display, and estimates which item was studied and how long it was spent on (for example, see Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-147279 A [Patent Document 2] JP 2016-138995 A Summary of the Invention [Problem to be solved by the invention]

[0005] When learning through group discussions, the instructor needs to keep track of whether the discussions in each group are progressing in line with the instructional intent and proceed appropriately. However, for example, in a group of four students, if only three of them understand and reach a conclusion, and there is one student who does not participate in the discussion at all or does not understand, this may be overlooked. To prevent such oversights, it is common for students to write a report at the end of the group learning about their realizations, and the instructor will evaluate it after the learning (class) is over. However, since this is a post-learning evaluation, it is not a means of giving appropriate advice on the spot each time during group learning.

[0006] Furthermore, when using video content as teaching material for group learning, unlike discussions on one fixed topic (such as "environmental issues" or "why did the sankin-kotai system begin"), the content of the video changes over time. Therefore, different groups may discuss different scenes about a certain subject (a spatial position within the video) at a certain time in the video content, making it even more difficult to understand. Therefore, it is necessary to be able to grasp the individual discussions and levels of understanding for each scene in the video content.

[0007] The technology in Patent Document 1 cannot grasp the time position of the image of the subject of the learner's speech or whether the speech is in line with the teaching intent. In addition, the technology in Patent Document 2 uses an image of the learner while studying, which raises concerns that it may have undesirable effects, such as the learner becoming conscious of the camera.

[0008] The present invention has been made in consideration of the above circumstances, and provides a video learning support device, information generating device, video learning support system, and program that can easily confirm whether learning activities are being carried out appropriately in line with the teaching intent of each scene during group learning using video as teaching material. [Means for solving the problem]

[0009] [1] One aspect of the present invention is a video learning support device that includes a video playback unit that plays video content used for learning in accordance with playback control instructions from a group of learners; an information acquisition unit that acquires group activity information at any time that indicates information about discussions being held by the group while watching the video content; and a learning status determination unit that generates, at any time, learning status information indicating whether the group's discussions are in line with the teaching points, based on a teaching point list that includes information about teaching points corresponding to the playback position of the video content and the group activity information acquired by the information acquisition unit.

[0010] [2] One aspect of the present invention is the above-mentioned video learning support device, wherein the group activity information includes one or both of audio data of the learners' speech and playback control instructions made by the group, the teaching point list includes one or both of text data describing the teaching points and the start and end playback positions of a scene including the teaching points, and the learning status determination unit generates learning status information including one or both of information on words of a predetermined part of speech included in the text data that are included in the speech recognition result of the audio data and information on the number of times a predetermined type of playback control instruction was made between the start and end of the scene.

[0011] [3] One aspect of the present invention is the above-mentioned video learning support device, wherein the group activity information includes information on the length of the learners' speech, and the learning status assessment unit generates the learning status information further including a speech rate of each of the learners calculated based on the information on the length of the speech of each of the learners.

[0012] [4] One aspect of the present invention is an information generation device comprising: an input unit for inputting operations by an operator; a display unit for displaying a playback position in video content and text data of the audio of the video content at the playback position; and a teaching point creation unit for generating a teaching point list including information on the playback positions of the start and end of a scene containing a teaching point for learning, selected by the input unit from the playback position displayed on the display unit, and text data of the teaching point selected from the text data displayed on the display unit.

[0013] [5] One aspect of the present invention is a video learning support system comprising a plurality of video learning support devices each corresponding to a different group of learners, and an information management device, wherein the video learning support device comprises a video playback unit that plays video content used for learning in accordance with a playback control instruction from the group corresponding to the device itself and outputs the video content to a video output device corresponding to the group, an information acquisition unit that acquires group activity information at any time indicating information regarding discussions being held by the group corresponding to the device itself while the video content is being watched, and a learning status determination unit that generates learning status information at any time indicating whether the discussions of the group corresponding to the device itself are in line with the teaching points based on a teaching point list including information on teaching points corresponding to the playback position of the video content and the group activity information acquired by the information acquisition unit, and outputs the learning status information to the information management device, and the information management device displays the learning status information received from each of the plurality of video learning support devices on an instructor's information display device, or on the instructor's information display device and the video output device.

[0014] [6] One aspect of the present invention is a program for causing a computer to function as the above-mentioned video learning support device.

[0015] [7] One aspect of the present invention is a program for causing a computer to function as the information generation device described above. Effect of the Invention

[0016] According to the present invention, when a group study is conducted using video as a teaching material, it becomes possible to easily check whether the learning activity is being carried out appropriately in accordance with the teaching intent of each scene. [Brief description of the drawings]

[0017] [Figure 1] 1 is a diagram showing an overview of the configuration and operation of a video learning support system according to a first embodiment of the present invention. [Diagram 2] 1 is a functional block diagram of a teaching point creation device according to a first embodiment. [Diagram 3] 1 is a functional block diagram of a video learning support device and an instructor's device according to a first embodiment. [Figure 4] FIG. 11 is a diagram showing a display example of a teaching point list creation screen according to the first embodiment. [Diagram 5] FIG. 4 is a diagram showing an example of a teaching point list according to the first embodiment. [Figure 6] FIG. 4 is a diagram showing an example of utterance content data according to the first embodiment. [Figure 7] FIG. 4 is a diagram showing an example of playback control data according to the first embodiment. [Figure 8] 4 is a flow diagram showing a teaching point list creation process of the teaching point creation device according to the first embodiment. FIG. [Figure 9] 3 is a flow diagram showing a playback control process of the video learning support device according to the first embodiment. FIG. [Figure 10] 4 is a flow diagram showing a speech collection process of the video learning support device according to the first embodiment. FIG. [Figure 11] 4 is a flow diagram showing a learning status report generation process of the video learning support device according to the first embodiment. FIG. [Figure 12] FIG. 4 is a diagram showing a display example of a learning status report according to the first embodiment. [Figure 13] FIG. 11 is a diagram showing the configuration of a video learning support system according to a second embodiment. [Figure 14] FIG. 13 is a diagram showing the configuration of a video learning support system according to a third embodiment. [Figure 15] FIG. 1 is a diagram showing an example of a hardware configuration of a video learning support system according to first to fourth embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In group learning while watching video content, the video learning support system of this embodiment collects information on learning activities such as what each learner is saying in which scene of the video content and whether they are watching while paying attention to scenes that are key to instruction, and grasps in real time the state of the learners, such as whether they are interpreting and discussing in line with the instructor's intentions, and outputs the report. This allows the instructor to respond while patrolling the desks, by giving appropriate advice to learners, such as calling out to learners who do not speak much or requesting a group that has gone off topic to correct its course, or by changing the progress of the lesson.

[0019] (First embodiment) FIG. 1 is a diagram showing an outline of the configuration and operation of a video learning support system 1 according to the first embodiment. The video learning support system 1 includes a video storage device 10, a teaching point creation device 20, a video learning support device 30, a monitor 40, a playback control interface 50, a microphone 60, and an instructor device 70. The video storage device 10, the teaching point creation device 20, and the video learning support device 30 are connected via a network such as the Internet or a local network. For example, the video storage device 10 may be a storage on the Internet, or may be a server computer connected to the teaching point creation device 20 or the video learning support device 30 via a local network. The video learning support device 30, the monitor 40, the playback control interface 50, the microphone 60, and the instructor device 70 are used in a classroom where learners are engaged in group learning.

[0020] The video storage device 10 stores video content, which is a video teaching material used for video learning. An instructor at an educational institution creates a teaching point list for learning using the video content using the teaching point creation device 20. The teaching point list is data that chronologically summarizes points to be taught to students in important scenes in the video content according to the playback positions of the important scenes in the video content. In other words, the teaching point list shows a list of important learning instruction contents, i.e., what the learner is expected to understand between what points on the time axis in the video content. Before learning, the instructor imports the video content in the video storage device 10 and the teaching point list created by the teaching point creation device 20 into the video learning support device 30.

[0021] During learning, one monitor 40 is used for each group of learners. The group of learners discusses the learning content provided by the video content while watching the video content output by the video learning support device 30 on the monitor 40. The video learning support device 30 acquires the speech of each learner using the microphone 60. Upon receiving the learner's voice data picked up by each microphone 60, the video learning support device 30 acquires the speaking time for each learner and transcribes the speech content into text. The video learning support device 30 compares the transcribed speech content data of each learner with the text data of the teaching points described in the teaching point list.

[0022] Furthermore, while watching the video content, the learner uses the playback control interface 50 to control playback of the video content, such as pausing, slow playback (also written as "slow"), playback, and fast rewinding, as necessary. The video learning support device 30 controls the playback position and speed of the video content output to the monitor 40 according to the playback control received from the playback control interface 50, and records log data of the playback control. The video learning support device 30 presents the learning status of the group, such as the consistency between the teaching points described in the teaching point list and the utterances of each learner, and the number of times playback control was performed at important scenes, to the instructor's device 70 as a learning status report.

[0023] The video learning support device 30 acquires the learner's speech data and playback control data in real time, updates the learning situation report in real time, and outputs it to the instructor's device 70. This makes it possible to provide the instructor with the latest situation.

[0024] FIG. 2 is a functional block diagram showing the configuration of the teaching point creation device 20. The teaching point creation device 20 is an information generating device that generates a teaching point list. In FIG. 2, only functional blocks related to this embodiment are extracted and shown. The teaching point creation device 20 is realized by a computer device such as a personal computer, a server computer, a tablet terminal, or a smartphone. The teaching point creation device 20 has a voice recognition unit 21, a teaching point creation unit 22, an input unit 23, a display unit 24, and a voice output unit 25.

[0025] The voice recognition unit 21 recognizes voice data such as narration and speech contained in the video content read from the video storage device 10 and converts it into text data. Any existing technology can be used for the voice recognition. The voice recognition unit 21 generates transcription data by adding a voice start position, which is the start position in the video content of the narration or speech from which the voice recognition result was obtained, to the text data obtained by the voice recognition. The voice start position is indicated, for example, by the time code of the video content. The time code is represented by the elapsed time with the playback position at the beginning of the video content set as 0.

[0026] The instruction point creation unit 22 creates an instruction point list that associates an in point, which is the start position of an important scene, and an out point, which is the end position of the important scene input by the input unit 23, with an important comment, which is an instruction point in the important scene. The in point and out point of an important scene are specified by selecting the audio start timing included in the transcription data, and an important comment is input by selecting a portion of text data corresponding to the instruction point from the text data included in the transcription data. Note that the text data of the important comment may be input by the input unit 23.

[0027] The input unit 23 is a keyboard, a mouse, a button, a touch panel, etc., and receives information input by operation of an operator such as an instructor. The display unit 24 is an image display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, an organic EL (Electro Luminescence) display, etc. The audio output unit 25 is an audio output device such as a speaker, a headphone, an earphone, etc.

[0028] 3 is a functional block diagram showing the configuration of the video learning support device 30 and the instructor's device 70. A monitor 40 that displays the video content, a microphone 60 that picks up the voices of each learner, and a playback control interface 50 for inputting playback control of the video content are connected to the video learning support device 30 by wired or wireless connections. The video learning support device 30 and the instructor's device 70 are connected wirelessly, but may also be connected by wire.

[0029] The video learning support device 30 includes a storage unit 31, a video playback unit 32, a synthesis unit 33, an utterance information creation unit 34, and a learning situation determination unit 35. The storage unit 31 stores video content read from the video storage device 10 and a teaching point list generated by the teaching point creation device 20.

[0030] The video playback unit 32 receives playback control data from the playback control interface 50. The video playback unit 32 reads out from the video content video data and audio data at a playback position corresponding to the playback control type set in the received playback control data, and outputs the data to the synthesis unit 33. The playback control types include playback, pause, slow playback, fast forward, and fast rewind. The video playback unit 32 also adds the time code of the video data that was being played when the playback control data was received, and outputs the data to the learning status determination unit 35.

[0031] The synthesis unit 33 synthesizes the video data and audio data output by the video playback unit 32 with the video data, image data, audio data, etc. received from the instructor's device 70, and outputs the synthesized data to the monitor 40. When the synthesis unit 33 has not received video data and image data from the instructor's device 70, it outputs the video data output by the video playback unit 32 to the monitor 40 as is, and when the synthesis unit 33 has not received audio data from the instructor's device 70, it outputs the audio data output by the video playback unit 32 to the monitor 40 as is.

[0032] The speech information creation unit 34 has a speaker identification unit 341 and a voice recognition unit 342. The speaker identification unit 341 identifies the speaker based on the voice data received from the microphone 60. When each learner uses a different microphone 60, the speech information creation unit 34 can identify the speaker by the microphone 60 from which the voice data was obtained. The speaker identification unit 341 adds speaker identification information indicating the identified speaker and the time of speech to the voice data. The voice recognition unit 342 recognizes the voice data from which the speaker identification unit 341 has identified the speaker, and converts it into text data indicating the content of the speech. The voice recognition unit 342 generates speech content data by adding the speaker identification information and the time of speech acquired from the voice data to the text data of the content of the speech, and outputs the generated speech content data to the learning status determination unit 35.

[0033] The learning status determination unit 35 counts the number of times the playback control indicated by the playback control data has been performed in each important scene indicated by the teaching point list. Furthermore, the learning status determination unit 35 compares the important comments in each important scene indicated by the teaching point list with the speech content data to determine whether the contents of the important comments have been mentioned. The learning status determination unit 35 outputs a learning status report, which is data including the number of times the playback control has been performed in each important scene and the contents of the important comments mentioned in each important scene, to the instructor's device 70.

[0034] The monitor 40 is a video output device for learners to view video content. The monitor 40 has an image display device such as a CRT display, a liquid crystal display, an organic EL display, etc., and an audio output device such as a speaker. Since one group uses one monitor 40 to view video content, a large, stationary monitor is preferable.

[0035] The playback control interface 50 is a playback control instruction device that allows the learner to instruct playback control such as play, pause, slow play, fast forward, and fast rewind of the video content viewed on the monitor 40. The playback control interface 50 includes buttons for instructing various playback controls. When the playback control interface 50 is realized by, for example, a smartphone or a tablet terminal, these buttons may be images of buttons displayed on a touch panel. The playback control interface 50 outputs to the video learning support device 30 playback control data indicating a playback control type that indicates the type of playback control corresponding to the button pressed by the learner and a playback control instruction time that is the time when the playback control is instructed.

[0036] The microphone 60 is a voice collecting device that generates voice data of each speaker's speech so that the speaker can be identified. Here, each learner uses an individual microphone 60. The microphone 60 may be a headset or a tabletop type.

[0037] The instructor's device 70 is used by the instructor to grasp the progress and learning situation of each group. The instructor's device 70 is, for example, a tablet terminal, a smartphone, a personal computer, etc. that the instructor can carry while instructing the learners.

[0038] The instructor's device 70 includes a display unit 71, an input unit 72, an imaging unit 73, a sound collection unit 75, and a processing unit 74. The display unit 71 is, for example, a display device such as a liquid crystal display. The input unit 72 is a touch panel, a keyboard, a mouse, a button, or the like, and receives information input by an operator such as an instructor. The imaging unit 73 is a camera and captures an image of the instructor. The sound collection unit 75 is a microphone and collects the voice of the instructor. The processing unit 74 displays the learning status report received from the video learning support device 30 on the display unit 71. In addition, the processing unit 74 transmits output data to be output to the monitor 40 to the video learning support device 30 in accordance with an instruction input by the input unit 72. The output data is one or more of video data of the instructor captured by the imaging unit 73, voice data of the instructor collected by the sound collection unit 75, and video data, image data, voice data, and the like instructed by the input unit 72.

[0039] The instructor's device 70 may be a video display device such as a PC monitor. In this case, the instructor's device 70 does not need to include the input unit 72, the imaging unit 73, the processing unit 74, and the sound collection unit 75. The video learning support device 30 does not include the synthesis unit 33, and the video playback unit 32 transmits video data and audio data to the monitor 40.

[0040] The video storage device 10 may be a storage on the Internet, or may be a server computer connected to the teaching point creation device 20 or the video learning support device 30 via a local network. The video storage device 10 may be provided in the teaching point creation device 20 or the video learning support device 30. Any of the teaching point creation device 20, the video learning support device 30, and the instructor device 70 may be integrated into one device. The monitor 40 may display a screen used as the playback control interface 50. The monitor 40, the playback control interface 50, the microphone 60, and the instructor device 70 may be used in a classroom, and may communicate with the video learning support device 30 on the Internet via a communication device installed in the classroom. The instructor device 70 may be used in a place other than the classroom where learning is performed.

[0041] 4 is a diagram showing a display example of a teaching point list creation screen G10 displayed on the display unit 24 by the teaching point creation unit 22 of the teaching point creation device 20. The teaching point creation unit 22 displays video data of video content in a video display area G11 of the teaching point list creation screen G10. Below the video display area G11, a play button B11, a pause button B12, a fast-rewind button B13, and a fast-forward button B14 are displayed for instructing playback control of the video content. The teaching point creation unit 22 controls the playback position of the video content displayed in the video display area G11 according to the play button B11, the pause button B12, the fast-rewind button B13, and the fast-forward button B14 selected by the input unit 23.

[0042] Furthermore, the instruction point creation unit 22 displays the time code read from the transcription data and text data such as a narration or speech corresponding to the time code in the transcription display area G12 of the instruction point list creation screen G10. A scene button B15 and a marker button B16 are displayed at the bottom of the transcription display area G12. The operator uses the input unit 23 such as a mouse to drag and select the range of the time codes displayed in the transcription display area G12, and clicks the scene button B15. This allows the selected range to be easily set as the range of an important scene. Furthermore, the operator uses the input unit 23 such as a mouse to drag and select the portion of the instruction point in the text data displayed in the transcription display area G12, and clicks the marker button B16. This allows the operator to easily set an instruction point that will become an important comment.

[0043] Fig. 5 is a diagram showing an example of a guidance point list. The guidance point list shown in Fig. 5 is made up of a plurality of guidance point data in which the scene number (No.) of each important scene, time in and time out, and important comments are associated with each other. The scene numbers are assigned in the order of time in. Time in is a time code indicating the start position of an important scene in the video content. Time out is a time code indicating the end position of an important scene in the video content. An important comment is text data of a guidance point in an important scene.

[0044] 6 is a diagram showing an example of utterance content data. The utterance content data is information that associates the date and actual time when the utterance was made, the track code (TC) of the video data displayed on the monitor 40 when the utterance was made, and text data of the utterance content for each speaker.

[0045] 7 is a diagram showing an example of playback control data. The playback control data is data that associates the date and real time when playback control was instructed, the type of control button pressed in the playback control interface 50, and the track code (video TC) of the video data displayed on the monitor 40 when the playback control was instructed. The type of control button indicates the type of playback control instructed by the learner.

[0046] The following describes the processing of the video learning support system 1. First, the processing for creating a teaching point list will be described, and then the processing during learning using video content will be described.

[0047] 8 is a flow diagram showing the teaching point list creation process of the teaching point creation device 20. The instructor inputs, via the input unit 23 of the teaching point creation device 20, video identification information for identifying the video content for which the teaching point list is to be created, and an instruction to execute voice recognition. The voice recognition unit 21 reads out the video content identified by the video identification information from the video storage device 10, and performs voice recognition on the narration and speech contained in the voice data of the read out video content, converting it into text data. The voice recognition unit 21 generates transcription data by adding, to the converted text data, a time code indicating the playback position of the voice data from which the text data was obtained (step S11).

[0048] The instruction point creation unit 22 judges whether or not playback control has been input by the input unit 23 (step S12). When the instruction point creation unit 22 judges that playback control has been input (step S12: YES), the instruction point creation unit 22 outputs the video content from the display unit 24 and the audio output unit 25 according to the input playback control, and displays the transcription data of the playback portion on the display unit 24 (step S13).

[0049] For example, the teaching point creation unit 22 displays the teaching point list creation screen G10 of Fig. 4 on the display unit 24. The instructor inputs an instruction to play the video content by clicking the play button B11 on the input unit 23 (step S12: YES). The teaching point creation unit 22 displays the video data of the video content in the video display area G11 of the teaching point list creation screen G10, and outputs the audio data from the audio output unit 25. Furthermore, the teaching point creation unit 22 displays the transcription data near the time code of the video frame being played in the transcription display area G12 of the teaching point list creation screen G10 (step S13).

[0050] On the other hand, when it is determined that playback control has not been input (step S12: NO), the teaching point creation unit 22 continues the playback state of the current video content, and displays the transcription data of the playback portion on the display unit 24 (step S14).

[0051] After the processing of step S13 or step S14, the instruction point creation unit 22 judges whether or not the start and end of an important scene are specified (step S15). If the instruction point creation unit 22 judges that the start and end of an important scene are not specified (step S15: NO), the instruction point creation unit 22 repeats the processing from step S12. On the other hand, if the instruction point creation unit 22 judges that the start and end of an important scene are specified (step S15: YES), the instruction point creation unit 22 receives input of an important comment (step S16).

[0052] For example, the instructor checks the video content being played back in the video display area G11 and determines the important scene. The instructor selects the range of the time code displayed in the transcription display area G12 by the input unit 23, and clicks the scene button B15 to specify the in point, which is the start position, and the out point, which is the end position, of the important scene. As a result, the instruction point creation unit 22 determines that the start and end of the important scene have been specified (step S15: YES). Furthermore, the instructor selects a portion of the instruction point from the text data displayed in the transcription display area G12 by the input unit 23, and clicks the marker button B16 to specify an important comment (step S16). Note that the instructor may directly input the text data to be the important comment by the input unit 23, or may select it from any portion of the transcription data. The instruction point creation unit 22 generates instruction point data by associating the in point and out point of the important scene with the important comment.

[0053] The instruction point creation unit 22 determines whether or not an end instruction has been input by the input unit 23 (step S17). When the instruction point creation unit 22 determines that an end instruction has not been input (step S17: NO), it repeats the process from step S12.

[0054] When it is determined that an instruction to end has been given (step S17: YES), the teaching point creating unit 22 arranges the teaching point data in order of start timing, assigns scene numbers in order, and creates a teaching point list (step S18).

[0055] In addition, when text data such as a script or closed caption of the video content is added to the video content, the teaching point creation device 20 may use the text data as transcription data without performing the process of step S11.

[0056] In the above, an example has been described in which an instructor creates a teaching point list, but a creator or provider of video content may create a teaching point list using the teaching point creation device 20 and provide the teaching point list together with the video content. The teaching point list created by the teaching point creation device 20 may be stored in the video storage device 10 or a recording medium, or may be transmitted to the video learning support device 30.

[0057] Next, the processing of the video learning support device 30 during learning using video content will be described. The video learning support device 30 acquires video content from the video storage device 10 and stores it in the storage unit 31. Furthermore, the video learning support device 30 acquires a teaching point list generated by the teaching point creation device 20 and stores it in the storage unit 31. The video learning support device 30 performs the processing of Figs. 9 and 10 in parallel.

[0058] 9 is a flow diagram showing the playback control process of the video learning support device 30. First, the video playback unit 32 of the video learning support device 30 determines whether or not playback control data has been received (step S21). For example, the learner operates buttons such as play, stop, slow play, fast rewind, and fast forward on the playback control interface 50. The playback control interface 50 transmits playback control data to the video learning support device 30, which sets the date and real time when the operation was performed and the playback control type represented by the operated button. The video playback unit 32 of the video learning support device 30 receives the playback control data (step S21: YES).

[0059] The video playback unit 32 outputs the video content to the monitor 40 according to the received playback control data (step S22). Specifically, the video playback unit 32 reads out the video data and audio data of the playback position corresponding to the current playback position and the type of playback control indicated by the received playback control data from the video content stored in the storage unit 31, and outputs them to the synthesis unit 33. When the synthesis unit 33 has not received output data from the instructor's device 70, it transmits the video data and audio data received from the video playback unit 32 to the monitor 40 as is. On the other hand, when the synthesis unit 33 has received output data from the instructor's device 70, it synthesizes the video data and audio data received from the video playback unit 32 with the video data, image data, audio data, etc. of the output data, and then transmits them to the monitor 40. The monitor 40 outputs the video data and audio data received from the video learning support device 30.

[0060] The instructor can instruct the output and stop of output data at any timing through the input unit 72 of the instructor's device 70. The output data may be video data of the instructor captured by the imaging unit 73, audio data of the instructor captured by the audio capture unit 75, or video data, image data, or audio data read from a memory location specified by the input unit 23.

[0061] The video playback unit 32 adds the time code of the video data being played when the playback control data was received to the received playback control data, and outputs the data to the learning status determination unit 35. The learning status determination unit 35 adds the playback control data received from the video playback unit 32 to the playback control data log stored in the storage unit 31 (step S23). The learning status determination unit 35 generates a learning status report using the playback control data log stored in the storage unit 31 (step S24).

[0062] On the other hand, when it is determined in step S21 that the playback control has not been input (step S21: NO), the video playback unit 32 continues the current playback state of the video content (step S25).

[0063] After the process of step S24 or step S25, the video playback unit 32 determines whether or not an end instruction has been input (step S26). If the video playback unit 32 determines that an end instruction has not been input (step S26: NO), it repeats the process from step S21. If the video playback unit 32 determines that an end instruction has been input (step S26: NO), it ends the process.

[0064] In the above, one monitor 40 is used for the group, but the group may have two or more monitors 40, such as each learner using his / her own tablet terminal or the like as the monitor 40. In this case, in steps S23 and S25, the synthesis unit 33 outputs the same video data and audio data to each monitor 40 of the same group.

[0065] Fig. 10 is a flow diagram showing the speech collection process of the video learning support device 30. Learners talk in groups while watching the video content played by the process of Fig. 8 on the monitor 40. Each microphone 60 outputs audio data of the learners' speech to the video learning support device 30. The speech information creation unit 34 of the video learning support device 30 receives the audio data (step S31: YES).

[0066] The storage unit 31 prestores the identification information of the microphone 60 and the learner identification information in association with each other. The speaker identification unit 341 reads out the learner identification information corresponding to the identification information of the microphone 60 that transmitted the voice data from the storage unit 31 and adds it to the voice data (step S32). The speaker identification unit 341 further adds the speaking time, which is indicated by the date and actual time when the voice data was obtained, to the voice data and outputs it to the voice recognition unit 342. Note that the microphone 60 may add the speaking time to the voice data and transmit it to the video learning support device 30.

[0067] The voice recognition unit 342 recognizes the voice data received from the speaker identification unit 341 and converts the spoken content into text data (step S33). The voice recognition unit 342 generates spoken content data by adding the learner identification information and the time of speech added to the voice data to the text data indicating the spoken content, and outputs the generated spoken content data to the learning status determination unit 35 (step S34). The voice recognition unit 342 may add the time code of the video data being played when the voice data was received to the spoken content data. The learning status determination unit 35 writes the spoken content data received from the voice recognition unit 342 in step S34 into the storage unit 31. The learning status determination unit 35 generates a learning status report using the spoken content data stored in the storage unit 31 (step S35).

[0068] When it is determined in step S31 that no voice data has been input (step S31: NO), or after the processing of step S35, the utterance information creation unit 34 determines whether or not an end instruction has been input (step S36). When it is determined that an end instruction has not been input (step S36: NO), the utterance information creation unit 34 repeats the processing from step S31. On the other hand, when it is determined that an end instruction has been input (step S36: YES), the utterance information creation unit 34 ends the processing of FIG. 10.

[0069] FIG. 11 is a flow diagram showing the learning status report generation process of the video learning support device 30 in step S24 in FIG. 9 and step S35 in FIG. 10. The learning status determination unit 35 reads out the playback control data stored in the memory unit 31. Furthermore, the learning status determination unit 35 reads out the scene number and the time in and time out of each important scene from the teaching point list, and identifies the playback control data generated between the time in and the time out of each important scene. The learning status determination unit 35 counts the identified playback control data for each important scene and for each playback control type to obtain the number of controls. As a result, the learning status determination unit 35 obtains the number of controls for each of the playback control types "play", "stop", and "slow playback" and the total number of controls (step S41).

[0070] Meanwhile, the learning status determination unit 35 reads out the utterance content data from the storage unit 31. The learning status determination unit 35 reads out the important comments of each important scene from the teaching point list. The learning status determination unit 35 performs morphological analysis on the important comments and the utterance content data, and extracts independent words from each (step S42).

[0071] The learning status determination unit 35 judges whether the independent words extracted from the important comment include an independent word obtained from the speech content data. The learning status determination unit 35 may select, for each important scene, speech content data that includes a time code within the time in and time out, and judge whether the important comment includes an independent word obtained from the selected speech content data. For each important comment read from the teaching point list, the learning status determination unit 35 adds information indicating that the important comment was spoken to the independent word portion also included in the speech content data (step S43). The information indicating that the important comment was spoken may use information indicating that the display mode is changed from other text when the text is displayed.

[0072] Furthermore, the learning status determination unit 35 counts the number of independent words included in important comments among the independent words obtained from the utterance content data for each learner, and sets this as the number of important word utterances. The learning status determination unit 35 divides the number of important word utterances counted for each learner by the total number of important word utterances of all learners in the group to calculate the important word utterance rate (step S44).

[0073] Furthermore, the learning status determination unit 35 calculates the speech rate of each learner (step S45). The speech rate is obtained by dividing the total speech time of each learner by the sum of the total speech times of all learners in the group. The voice recognition unit 342 may obtain speech time information based on the voice data during voice recognition and add it to the speech content data, and the learning status determination unit 35 may calculate the total speech time by summing up the speech time information read from the speech content data for each learner. Alternatively, the speech information creation unit 34 may detect the voice, collect information on the time when the microphone 60 is active, and output it to the learning status determination unit 35, and the learning status determination unit 35 may sum up the time when the microphone 60 corresponding to each learner is active for each learner, and use it as the total speech time.

[0074] The learning status determination unit 35 creates a learning status report that integrates the results obtained in steps S41, S43, S44, and S45, and outputs the report to the instructor's device 70 (step S46).

[0075] FIG. 12 is a diagram showing a display example of a learning status report. The learning status report includes information that associates a segment number indicating the scene number of each important scene, a time in and a time out, an important comment, and the number of times playback control is performed. Among the independent words included in the important comments, those spoken by the learner are indicated by adding a highlight. The fact that the independent word was spoken by the learner may be indicated in other ways, such as by changing the color of the text or adding an underline. Furthermore, information about the learner who spoke the independent word may be added to the independent word spoken by the learner and displayed.

[0076] The learning status determination unit 35 displays the number of times each of the playback controls, play, stop, and slow play, was performed as the number of times of playback control, and the total count of these. By counting the number of times play, stop, and slow play were performed in each important scene, it is possible to determine whether the learner is carefully watching the video in the scene listed as the teaching point. For example, in scene number 3 (00:46:03 to 00:56:15) in the teaching point list shown in FIG. 5, the playback control data shown in FIG. 7 includes two "stops" and one "slow play," so the total count is "3."

[0077] In addition, the learning status determination unit 35 displays the important word utterance rate and utterance rate of each learner in a pie chart. This makes it possible to grasp whether each learner is talking about content that is in line with the teaching points, participating in discussions, etc.

[0078] By displaying a learning status report such as that shown in FIG. 12 on the instructor's device 70 in real time, the instructor can grasp the bias in speaking and the progress of the class, and can support the instructor in taking appropriate action, such as encouraging students who speak less while walking around the desks or pointing out scenes that have been left out of the discussion.

[0079] Second Embodiment In the first embodiment, there is one group of learners. However, there may be multiple groups of learners for one instructor. In the second embodiment, the learning status of multiple groups is managed. The second embodiment will be described focusing on the differences from the first embodiment.

[0080] FIG. 13 is a diagram showing a configuration example of the video learning support system 2 in the second embodiment. In the figure, the same parts as those in the video learning support system 1 in the first embodiment shown in FIG. 1 are given the same reference numerals, and their description is omitted. In FIG. 13, the video storage device 10 and the teaching point creation device 20 are omitted. The video learning support system 2 in the second embodiment shown in FIG. 13 is different from the video learning support system 1 in the first embodiment in that the instructor's device 70 and the video learning support device 30 of each of the multiple learning groups are connected to the information management device 80. Each video learning support device 30 may be installed in a geographically separated location. Each video learning support device 30 is connected to the monitor 40, the playback control interface 50, and the microphone 60 as in the first embodiment. However, the learning status determination unit 35 of the video learning support device 30 outputs a learning status report to the information management device 80. Note that the instructor's device 70 or any of the video learning support devices 30 and the information management device 80 may be integrated into a single device.

[0081] The processing unit 74 of the instructor's device 70 outputs group identification information of each of the one or more groups input by the instructor through the input unit 72 to the information management device 80. The information management device 80 returns to the instructor's device 70 the learning status report received from the video learning support device 30 of the group identified by the group identification information received from the instructor's device 70. The processing unit 74 of the instructor's device 70 displays the learning status report of each group received from the information management device 80 on the display unit 71. For example, the instructor's device 70 may display learning status reports of multiple groups on one screen, or may display the learning status report of each group in different windows or different tabs in the same window.

[0082] Furthermore, the processing unit 74 of the instructor's device 70 transmits the output data and information on the one or more output destination video learning support devices 30 to the information management device 80. The information management device 80 transmits the output data received from the instructor's device 70 to the output destination video learning support device 30. The synthesis unit 33 of the video learning support device 30 that receives the output data synthesizes the video data and audio data of the video content output by the video playback unit 32 with the output data received from the information management device 80, and outputs the synthesized data to the monitor 40 connected to the device itself.

[0083] According to the second embodiment, the learning status report of each group is managed by the information management device, so that the instructor can grasp the learning status of a plurality of groups.

[0084] (Third embodiment) In the second embodiment, a learning status report is displayed on the instructor's device 70. In the third embodiment, the learners are also allowed to view the learning status report. This allows the learners to review their learning status and also to grasp the progress of other groups. The third embodiment will be described focusing on the differences from the second embodiment.

[0085] Fig. 14 is a diagram showing an example of the configuration of a video learning support system 3 in the third embodiment. In Fig. 14, the same parts as those in the video learning support system 2 in the second embodiment shown in Fig. 13 are given the same reference numerals, and their description will be omitted. The video learning support system 3 in the third embodiment shown in Fig. 14 differs from the video learning support system 2 in the second embodiment shown in Fig. 13 in that an information management device 81 is provided instead of the information management device 80. Note that the instructor's device 70 or any of the video learning support devices 30 and the information management device 81 may be integrated into a single device.

[0086] The information management device 81 performs the same process as in the second embodiment. Furthermore, the information management device 81 outputs a learning status report received from a video learning support device 30 of a certain learning group to the video learning support device 30 of that learning group and to the video learning support devices 30 of other learning groups. Each video learning support device 30 outputs the received learning status report to the monitor 40 for display. Note that the information management device 81 may delete a predetermined portion of the learning status reports of other groups before transmitting them to the video learning support device 30. For example, information on speech rate and key word speech rate may be deleted.

[0087] (Fourth embodiment) In the above-mentioned first to third embodiments, group learning is performed using video learning materials. However, the present invention is also applicable to cases where group learning is performed without using video learning materials. The fourth embodiment will be described focusing on the differences from the first to third embodiments. The configuration of the video learning support system according to the fourth embodiment is similar to the video learning support system 1 of the first embodiment, the video learning support system 2 of the second embodiment, and the video learning support system 3 of the third embodiment.

[0088] However, if video teaching materials are not used, the teaching point list does not include information on the start and end timings. Therefore, the teaching point list is created by directly inputting sentences or words that are to be discussed in the important comments of the teaching point list. The teaching point list may be created using the teaching point creation device 20 or may be created using a text editor or the like.

[0089] Since the video learning support device 30 does not use playback control data, it does not perform the process in Fig. 9 and the process in step S41 in Fig. 11. In the process in step S46 in Fig. 11, the video learning support device 30 generates a learning status report that integrates the results of steps S43, S44, and S45. When an important comment contains an independent word that matches an independent word contained in the learner's utterance, the learning status determination unit 35 displays the independent word in a different manner from other characters, such as by highlighting the independent word. The learning status determination unit 35 outputs a graph of the speech rate and the results of the important word speech rate in the same manner as in the first embodiment.

[0090] According to the embodiment described above, the video learning support system can present information that serves as a guide to measure whether group learning is progressing in accordance with the instructional intent. Furthermore, the instructor can confirm whether the learners are reading the instructional intent along the timeline of the video in group learning using video content, by using the learning status report presented by the video learning support system. Furthermore, the learners do not need to perform special operations such as typing something on a keyboard, and can record their learning status only by operating the discussion and video playback in a manner similar to normal. Furthermore, the instructor can grasp the learning status in real time while the learning is in progress, not after the learning (class). In addition, since the processing is minor, it can be sufficiently realized by a general personal computer. Furthermore, the learning status report can be fed back not only to the instructor but also to the learners, and it is possible to look back on which group overlooked which item, as well as the main points of the video content. Furthermore, even when a small number of instructors are leading a large number of groups, the status of each group can be viewed at a glance, allowing for efficient instruction.

[0091] According to the embodiment described above, in group learning using video, it is possible to grasp the learning situation of the group in real time to see whether the discussion is in line with the teaching intention. This allows the instructor to respond while patrolling the desks by giving appropriate advice to the learners (such as calling out to those who don't speak much, or asking a group that is going off topic to get back on track) or by changing the progress of the lesson.

[0092] The teaching point creation device 20, the video learning support device 30, and the instructor's device 70 may each be realized by a plurality of computer devices connected to a network. In this case, it is possible to arbitrarily select which of the plurality of computer devices each of the functional units of the teaching point creation device 20, the video learning support device 30, and the instructor's device 70 is realized by. Also, the same functional unit of the teaching point creation device 20, the video learning support device 30, and the instructor's device 70 may be realized by a plurality of computer devices.

[0093] FIG. 15 is a diagram showing an example of a hardware configuration of the video learning support device 30. The video learning support device 30 includes a processor 91, a storage unit 92, a communication interface 93, and a user interface 94. The processor 91 is a central processing unit that performs calculations and control. The processor 91 is, for example, a central processing unit (CPU) or a graphics processing unit (GPU). The processor 91 reads out and executes a program from the storage unit 92. The storage unit 92 further includes a work area and the like for the processor 91 to execute various programs. The communication interface 93 is for connecting to other devices so as to be able to communicate with them. The user interface 94 includes a monitor 40, a microphone 60, and a playback control interface 50.

[0094] All or part of the functions of the video learning support device 30 are realized by the processor 91 reading and executing a program from the storage unit 92. All or part of these functions may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). Examples of the hardware configurations of the teaching point creation device 20 and the instructor's device 70 are also similar to those shown in FIG. 15. However, in the case of the teaching point creation device 20, the user interface 94 is the input unit 23, the display unit 24, and the audio output unit 25, and in the case of the instructor's device 70, the user interface is the display unit 71, the input unit 72, and the imaging unit 73.

[0095] According to the embodiment described above, the video learning support device has a video playback unit, an information acquisition unit, and a learning status determination unit. The video playback unit plays video content used for learning according to a playback control instruction by the group of learners, and outputs it to the video output device. The video output device corresponds, for example, to the monitor 40 in the embodiment. The information acquisition unit acquires group activity information indicating information on the discussion being held by the group while watching the video content at any time. The information acquisition unit corresponds, for example, to the video playback unit 32 and the speech information creation unit 34 in the embodiment, and the activity information corresponds to the voice data transmitted from the microphone 60 and the playback control data transmitted from the video learning support device 30 in the embodiment. The learning status determination unit generates learning status information indicating whether the group's discussion is in line with the teaching points at any time based on the teaching point list including information on the teaching points of learning according to the playback position of the video content and the group activity information acquired by the information acquisition unit.

[0096] The group activity information includes one or both of the voice data of the learners' utterances and the playback control instructions given by the group, and the teaching point list includes one or both of the text data describing the teaching points and the start and end playback positions of the scenes including the teaching points. The learning status determination unit generates learning status information including one or both of information on words included in the voice recognition result of the voice data among words of a predetermined part of speech included in the text data and information on the number of times a predetermined type of playback control instruction was given from the start to the end of a scene.

[0097] The group activity information may include information on the length of each learner's utterance. The learning status determination unit generates the learning status information further including the speaking rate of each learner calculated based on the information on the length of each learner's utterance.

[0098] The video learning support system also has a plurality of video learning support devices each corresponding to a different group of learners, and an information management device. The learning status determination unit of each video learning support device outputs learning status information to the information management device. The information management device may display the learning status information received from each of the plurality of video learning support devices on the instructor's information display device, or may display the information on the instructor's information display device and on the video output device of each group that outputs video content. The instructor's information display device corresponds to, for example, the instructor's device 70 of the embodiment.

[0099] The information generating device has an input unit, a display unit, and a teaching point creation unit. The information generating device corresponds to the teaching point creation device 20 of the embodiment. The input unit inputs operations by an operator. The display unit displays a playback position in video content used for learning and text data of the audio of the video content at the playback position. The teaching point creation unit generates a teaching point list including information on the playback positions of the start and end of a scene including a teaching point for learning, selected by the input unit from the playback position displayed on the display unit, and text data of the teaching point selected by the input unit from the text data displayed on the display unit.

[0100] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Explanation of symbols]

[0101] 1, 2, 3 Video learning support system 10 Video storage device 20 Guidance Point Creation Device 21 Voice Recognition Unit 22 Guidance Point Creation Department 23 Input section 24 Display 25 Audio output section 30 Visual learning support device 31 Storage section 32 Video playback section 33 Synthesis section 34 Speech information creation unit 35 Learning status assessment unit 40 Monitor 50 Playback control interface 60. Mike 70 Instructor equipment 71 Display section 72 Input section 73 Imaging unit 74 Processing section 75 Sound pickup section 80, 81 Information management device 91 Processor 92 Memory section 93 Communication Interface 94 User Interface 341 Speaker Identification Unit 342 Voice Recognition Unit

Claims

1. a video playback unit that plays video content used for learning in accordance with playback control instructions from a group of learners; an information acquisition unit that acquires group activity information indicating information regarding a discussion being held by the group while the group is watching the video content; a learning status determination unit that generates, on an ongoing basis, learning status information indicating whether the discussion of the group is in line with the teaching points, based on a teaching point list including information on teaching points of the learning corresponding to the playback position of the video content and the group activity information acquired by the information acquisition unit; A video learning support device comprising:

2. The group activity information includes one or both of voice data of the learners' utterances and playback control instructions made by the group, the instruction point list includes text data describing the instruction points and / or playback positions of the start and end of scenes including the instruction points; the learning status determination unit generates learning status information including one or both of information on words included in the speech recognition result of the speech data among words of a predetermined part of speech included in the text data and information on the number of times the playback control instruction of a predetermined type has been performed from the start to the end of the scene.

2. The video learning support device according to claim 1.

3. The group activity information includes information on the length of the learner's utterances, the learning status determination unit generates the learning status information further including a speech rate of each of the learners calculated based on information on the length of the speech of each of the learners.

3. The video learning support device according to claim 2.

4. an input unit for inputting an operation by an operator; a display unit that displays a playback position in a video content and text data of the audio of the video content at the playback position; a teaching point creation unit that generates a teaching point list including information on the start and end playback positions of a scene including a teaching point of learning, selected by the input unit from the playback position displayed on the display unit, and text data of the teaching point selected by the input unit from the text data displayed on the display unit; An information generating device comprising:

5. A plurality of video learning support devices each corresponding to a different group of learners; An image learning support system including an information management device, The video learning support device comprises: a video playback unit that plays video content used for learning in accordance with a playback control instruction from the group corresponding to the device itself and outputs the video content to a video output device corresponding to the group; an information acquisition unit that acquires group activity information indicating information regarding a discussion being held by the group corresponding to the device at any time while the video content is being viewed; a learning status determination unit that generates learning status information indicating whether the discussion of the group corresponding to the device is in line with the teaching points based on a teaching point list including information on teaching points of learning corresponding to the playback position of the video content and the group activity information acquired by the information acquisition unit, and outputs the learning status information to the information management device; Equipped with The information management device includes: The learning status information received from each of the plurality of video learning support devices is displayed on an information display device of the instructor, or is displayed on an information display device of the instructor and the video output device. Video learning support system.

6. Computer, A program for causing the video learning support device according to claim 1 to function.

7. Computer, A program for causing the information generating device according to claim 4 to function.

Images