Information processing device, information processing method, and program

Abstract

To accurately estimate a viewer's degree of interest.SOLUTION: An information processing device includes an estimation method determination unit that determines an estimation method for estimating a degree of interest of a viewer watching content on the basis of at least a content status, and an interest degree estimation unit that estimates the degree of interest on the basis of the estimation method and a plurality of parameters including a parameter related to the viewer's gaze.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] This technology relates to an information processing apparatus, an information processing method, and a program, and particularly relates to a technology for viewing content online.

Background Art

[0002] For example, Patent Document 1 describes an apparatus that evaluates the learning state of students (viewers) based on information regarding the actions of students taking an online class, and presents information to assist the progress of the online class to the teacher based on the learning state.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the above-described apparatus, the scenes of the online class and the state of the teacher are not reflected when evaluating the learning state of the students. Therefore, there is a possibility that the above-described apparatus cannot accurately evaluate the learning state of the students.

[0005] Therefore, an object of this technology is to accurately estimate the degree of interest of the viewer.

Means for Solving the Problems

[0006] An information processing apparatus according to this technology includes an estimation method determination unit that determines an estimation method for estimating the degree of interest of a viewer who views the content based on at least the situation of the content, and an interest degree estimation unit that estimates the degree of interest based on the estimation method and a plurality of parameters including parameters related to the viewer's line of sight. As a result, the information processing apparatus can determine an optimal estimation method according to the class situation.

Brief Description of Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Mode for Carrying Out the Invention

[0008] Hereinafter, embodiments will be described in the following order. <1. Configuration of Online Class System 1> <2. Configuration of Teacher's Terminal Device 2> <3. Configuration of Student's Terminal Device 3> <4. Configuration of Server 4> <5. An Example of Online Class> <6. Degree of Interest Estimation Notification Process> <7. Flow of Degree of Interest Estimation Notification Process> <8. Another Configuration Example of Online Class System 1> <9. Summary of Embodiments> <10. This Technology>

[0009] <1. Configuration of Online Class System 1> First, the configuration of the online class system 1 according to the embodiments of the present technology will be described. FIG. 1 is a diagram for explaining the configuration of the online class system 1.

[0010] As shown in FIG. 1, the online class system 1 includes a teacher's terminal device 2, a plurality of student's terminal devices 3, and a server 4.

[0011] The teacher's terminal device 2, the student's terminal device 3, and the server 4 are connected via a network 5 such as the Internet, and can communicate with each other via the network 5.

[0012] The teacher's terminal device 2 is assumed to be used by a teacher who conducts an online class. The student's terminal device 3 is assumed to be used by a student who takes an online class.

[0013] In the online class system 1, an online class can be conducted by allowing the student's terminal device 3 to view contents such as videos and materials of the class scene provided (distributed) from the teacher's terminal device 2.

[0014] Note that the teacher is an example of a provider who provides content, and the student is an example of a viewer who views content. However, the explainer and the viewer are not limited to this.

[0015] <2. Configuration of the Teacher's Terminal Device 2> FIG. 2 is a diagram for explaining the configuration of the teacher's terminal device 2. As shown in FIG. 2, the teacher's terminal device 2 is a computer including a CPU (Central Processing Unit) 20, a ROM (Read Only Memory) 21, and a RAM (Random Access Memory) 23. The teacher's terminal device 2 is, for example, a personal computer, a mobile terminal device such as a smartphone, or a tablet device.

[0016] The CPU 20 controls the entire teacher's terminal device 2 by expanding and executing a program stored in the ROM 21 or a storage unit (not shown) into the RAM 22.

[0017] In addition to the CPU 20, the ROM 21, and the RAM 22, the teacher's terminal device 2 includes a display unit 23, an operation unit 24, a gaze detection device 25, a communication unit 26, a microphone 27, a speaker 28, and an imaging unit 29.

[0018] The display unit 23 is a liquid crystal display, an OLPD (Organic Light Emitting Diode) display, etc., and displays various screens (images).

[0019] The operation unit 24 is an input device used by a user (here, the teacher), and is, for example, various operators and operation devices such as a keyboard, a mouse, a button, a dial, a touch pad, and a touch panel. When an operation is detected by the operation unit 24, a signal corresponding to the input operation is input to the CPU 20.

[0020] The line-of-sight detection device 25 is a device that detects the line-of-sight direction of a user (here, the teacher). The line-of-sight detection device 25 detects the line-of-sight direction of the user by, for example, the corneal reflection method or facial feature point detection. When detecting the line-of-sight direction of the user by the corneal reflection method, the line-of-sight detection device 25 includes an infrared light source that irradiates infrared light and an infrared camera that receives infrared light, and based on an image obtained by receiving the infrared light irradiated from the infrared light source and reflected by the user's pupil with the infrared camera, the line-of-sight direction of the user is detected. When detecting the line-of-sight direction of the user by facial feature point detection, the line-of-sight detection device 25 includes a visible light camera that receives visible light, detects feature points by analyzing the image of the user's face captured by the visible light camera, and estimates the line-of-sight direction of the user from the orientation of the face and the position of the iris, etc.

[0021] In addition, based on the detected line-of-sight direction of the user, the positional relationship between the line-of-sight detection device 25 and the display unit 23, etc., the line-of-sight detection device 25 detects the position on the screen displayed on the display unit 23 that the user is gazing at, that is, the coordinates (X, Y) of the fixation point. Here, X indicates the coordinate in the horizontal direction, and Y indicates the coordinate in the vertical direction. Note that the line-of-sight detection device 25 may be other than those described above as long as it can detect the line-of-sight direction of the user and the coordinates of the fixation point. Also, the line-of-sight detection device 25 may cooperate with the CPU 20 to detect the line-of-sight direction of the user.

[0022] The communication unit 26 performs communication via the network 5 with the student-side terminal device 3 and the server 4. The microphone 27 collects the voice uttered by the user (here, the teacher). The speaker 28 outputs voice to the user (here, the teacher). The imaging unit 29 is, for example, an image sensor of the CCD (Charge Coupled Device) type or the CMOS (Complementary Metal-Oxide-Semiconductor) type. The imaging unit 29 images the teaching scene performed by the teacher.

[0023] <3. Configuration of the Student Terminal Device 3> FIG. 3 is a diagram for explaining the configuration of the student terminal device 3. As shown in FIG. 3, the student terminal device 3 is a computer including a CPU 30, a ROM 31, and a RAM 32. The student terminal device 3 is, for example, a personal computer, a mobile terminal device such as a smartphone, a tablet device, or the like.

[0024] The CPU 30 controls the entire student terminal device 3 by expanding and executing a program stored in the ROM 31 or a storage unit (not shown) into the RAM 32.

[0025] In addition to the CPU 30, the ROM 31, and the RAM 32, the student terminal device 3 includes a display unit 33, an operation unit 34, a gaze detection device 35, a communication unit 36, a microphone 37, a speaker 38, and an imaging unit 39.

[0026] The display unit 33 is a liquid crystal display, an OLPD display, or the like, and displays various screens (images).

[0027] The operation unit 34 is an input device used by a user (here, a student), and is, for example, various operators and operation devices such as a keyboard, a mouse, buttons, a dial, a touch pad, and a touch panel. When an operation is detected by the operation unit 34, a signal corresponding to the input operation is input to the CPU 30.

[0028] The gaze detection device 35 is a device that detects the gaze direction of a user (here, a student). The gaze detection device 35 is configured in the same manner as the gaze detection device 25.

[0029] The communication unit 36 communicates with the teacher terminal device 2 and the server 4 via the network 5. The microphone 37 collects the voice uttered by a user (here, a student). The speaker 38 outputs voice to a user (here, a student). The imaging unit 39 is, for example, a CCD type or CMOS type image sensor. The imaging unit 39 images, for example, the upper body of a student.

[0030] <4. Configuration of Server 4> FIG. 4 is a diagram for explaining the configuration of server 4. FIG. 5 is a diagram for explaining the functional configuration of CPU 40. As shown in FIG. 4, server 4 is a computer including CPU 40, ROM 41, RAM 42, storage unit 43, and communication unit 44. CPU 40 controls the entire server 4 by expanding and executing a program stored in ROM 41 or storage unit 43 in RAM 42.

[0031] As shown in FIG. 5, CPU 40 functions as a class situation specifying unit 50, student characteristic specifying unit 51, estimation method determining unit 52, parameter value calculating unit 53, degree of interest estimating unit 54, and notification unit 55. The class situation specifying unit 50 specifies the class situation such as the class scene and the teacher's state based on the class data transmitted from the teacher-side terminal device 2. The student characteristic specifying unit 51 specifies student characteristics such as the characteristics and habits of the student based on the student data transmitted from the student-side terminal device 3. The estimation method determining unit 52 determines an estimation method for estimating the degree of interest for each student, which will be described in detail later. Specifically, the estimation method determining unit 52 determines a parameter for estimating the degree of interest from among a plurality of preset parameters based on at least the class situation specified by the class situation specifying unit 50, and determines the weighting of the determined parameter. Therefore, the estimation method includes the determination of the parameter for estimating the degree of interest and the weighting of the parameter, but only one of them may be sufficient. The parameter value calculating unit 53 calculates the parameter value of the parameter determined by the estimation method determining unit 52. The degree of interest estimating unit 54 estimates the degree of interest for each student based on the parameter value calculated by the parameter value calculating unit 53. The notification unit 55 causes the teacher-side terminal device 2 to issue a notification based on the degree of interest estimated by the degree-of-interest estimation unit 54. Details of the class situation identification unit 50, the student characteristic identification unit 51, the estimation method determination unit 52, the parameter value calculation unit 53, the degree-of-interest estimation unit 54, and the notification unit 55 will be described later in detail.

[0032] The storage unit 43 is composed of a storage medium such as a solid-state memory, for example. Various information described later can be stored in the storage unit 43. Further, the storage unit 43 can also be used for storing program data for the CPU 40 to execute various processes. The communication unit 44 communicates with the teacher-side terminal device 2 and the student-side terminal device 3 via the network 5.

[0033] <5. An Example of Online Class> FIG. 6 is a diagram showing a class screen 60 displayed on the display unit 33 of the student-side terminal device 3 in an online class. FIG. 7 is a diagram showing a student screen 61 displayed on the display unit 23 of the teacher-side terminal device 2 in an online class.

[0034] In the teacher-side terminal device 2, a class video is obtained by imaging the class scene by the imaging unit 29, and the class audio is collected by the microphone 27. Then, the teacher-side terminal device 2 transmits the obtained class video and class audio to the student-side terminal device 3 and the server 4 as class data.

[0035] In the student-side terminal device 3 that has received the class data, as shown in FIG. 6, the class video included in the class data is displayed as the class screen 60 on the display unit 33. Also, in the student-side terminal device 3, the class audio included in the class data is output from the speaker 38.

[0036] In this way, in the online class system 1, students using the student-side terminal device 3 can watch the class of the teacher using the teacher-side terminal device 2 online in real time.

[0037] Also, in the student terminal device 3, a student video is obtained by imaging the upper body of the student by the imaging unit 39, and student voices are collected by the microphone 37. Then, the student terminal device 3 transmits the obtained student video and student voices as student data to the teacher terminal device 2 and the server 4.

[0038] In the teacher terminal device 2 that has received student data from a plurality of student terminal devices 3, as shown in FIG. 7, a student screen 61 in which student videos (student images) included in the student data are arranged is displayed on the display unit 23. Also, in the teacher terminal device 2, the student voices included in the student data are output from the speaker 28. In this way, in the online class system 1, the teacher can view in real time the state of the students using the student terminal device 3.

[0039] By the way, it is difficult for the teacher using the teacher terminal device 2 to determine whether the student is listening to the class, that is, the degree of interest of the student in the class, just by checking the student screen 61.

[0040] Therefore, the server 4 performs a degree-of-interest estimation notification process of estimating the degree of interest (listening degree) of the student and notifying the teacher based on the estimated degree of interest. During the execution of the degree-of-interest estimation notification process, the teacher terminal device 2 transmits the gaze information detected by the gaze detection device 25 to the server 4 as part of the class data. Also, the student terminal device 3 transmits the gaze information detected by the gaze detection device 35 to the server 4 as part of the student data.

[0041] <6. Degree-of-Interest Estimation Notification Process> The degree-of-interest estimation notification process includes a listening degree calculation process of calculating (estimating) the listening degree as the degree of interest for each student, and a notification process of notifying the teacher based on the calculated listening degree. First, the listening degree calculation process will be explained, and then the notification process will be explained.

[0042] [6.1. Listening Degree Calculation Process] FIG. 8 is a diagram for explaining the outline of the listening degree calculation process. In the listening degree calculation process, the class situation is specified by the class situation specifying unit 50, and the student characteristics are specified by the student characteristic specifying unit 51. Then, in the listening degree calculation process, as shown in FIG. 8, based on the specified class situation and student characteristics, a parameter for calculating the listening degree is determined by the estimation method determination unit 52 from among a plurality of parameters. Thereafter, in the listening degree calculation process, after the parameter value is calculated by the parameter value calculation unit 53, the listening degree is calculated by the degree of interest estimation unit 54 using the parameter value. Hereinafter, these processes will be specifically described.

[0043] FIG. 9 is a diagram for explaining the specification of the class situation. The class situation specifying unit 50 specifies the class situation based on the class data transmitted from the teacher-side terminal device 2. The class situation to be specified is information that enables determination of what kind of class is being conducted, such as the presence or absence of a pointing operation, the presence or absence of speech, the presence or absence of text display, the presence or absence of motion content, the teacher's state (facial expression, movement), etc., as shown in FIG. 8.

[0044] The class situation specifying unit 50 specifies, for example, the presence or absence of text display, the presence or absence of motion content, and the teacher's state in the class scene by performing image analysis on the class video included in the class data.

[0045] More specifically, as shown in FIG. 9, the class situation specifying unit 50 estimates the teacher's posture using a known posture estimation technique (bone estimation) from the class video. Then, based on the estimated teacher's posture, the class situation specifying unit 50 specifies, for example, the state 71 where the hand is pointing, thereby specifying the presence or absence of a pointing operation. In addition, the class situation specifying unit 50 specifies the presence or absence of text display by performing image analysis on the class video to specify the area 72 where text is displayed. In addition, the class situation specifying unit 50 specifies the presence or absence of motion content (video content) by performing image analysis on the class video. In addition, the class situation identification unit 50 identifies the face 73 of the teacher using known facial expression recognition technology from the class video, and identifies the facial expression of the teacher as the teacher's state.

[0046] In addition, the class situation identification unit 50 performs voice analysis on the voice included in the class data to identify, for example, the presence or absence of speech in the class situation.

[0047] In this way, when the class situation identification unit 50 receives class data from the teacher-side terminal device 2, it identifies the class situation based on the received class data.

[0048] Note that each item and identification method of the class situation described above are examples, and other items may be identified as the class situation, or each item of the class situation may be identified by other methods.

[0049] The student characteristic identification unit 51 identifies the student characteristics of each student based on the student data transmitted from each student-side terminal device 3 and stores them in the storage unit 43. The student characteristics to be identified include, as shown in FIG. 8, the frequency of face gazing, reading speed, head movement characteristics, posture, presence or absence of smartphone holding, physiological indicators, etc.

[0050] Specifically, the student characteristic identification unit 51 identifies the gaze points of the students on the class screen 60 based on the class video included in the class data and the gaze information included in the student data. Then, the student characteristic identification unit 51 identifies the frequency with which the students gaze at the teacher's face during the teacher's speech as the face gazing frequency.

[0051] In addition, the student characteristic identification unit 51 calculates the moving speed of the gaze when the student is gazing at the text displayed in the class video as the reading speed.

[0052] In addition, the student characteristic identification unit 51 identifies the facial expressions of the students using known facial expression recognition technology from the student videos included in the student data. Then, the student characteristic identification unit 51 identifies head movement characteristics such as the frequency of blinking and head movement based on the changes in the identified facial expressions of the students.

[0053] In addition, the student characteristic identification unit 51 estimates the posture of the student using a known posture estimation technique (bone estimation) from the student video included in the student data. Then, based on the estimated posture of the student, the student characteristic identification unit 51 identifies physiological indexes related to physiological phenomena such as the presence or absence of smartphone gripping and yawning.

[0054] In this way, when the student characteristic identification unit 51 receives student data from the student-side terminal device 3, it identifies the student based on the received student data and stores it in the storage unit 43. Note that the identification of student characteristics may be performed in advance before the listening degree calculation process, or may be performed in parallel with the listening degree calculation process.

[0055] Also, each item and identification method of the above-described student characteristics are examples, and other items of student characteristics may be identified, or each item of student characteristics may be identified by other methods.

[0056] FIG. 10 and FIG. 11 are diagrams for explaining an example of determining parameters to be used. The estimation method determination unit 52 determines any one of a plurality of parameters for calculating the listening degree based on the class situation identified by the class situation identification unit 50 and the student characteristics identified by the student characteristic identification unit 51, and determines the weighting of the determined parameters.

[0057] Examples of the parameters include line-of-sight correlation, fixation on a position with eye-catching property, expression, line-of-sight position accumulation, movement of the body or head, frequency or duration of blinking, and a specific line-of-sight movement pattern. Note that line-of-sight correlation, fixation on a position with eye-catching property, and line-of-sight position accumulation can be said to be parameters related to the line of sight of the student.

[0058] For example, as shown in FIG. 10, in a classroom situation where the teacher is facing forward and speaking, and no text such as a passage is being displayed, it is presumed that students with a low face fixation frequency are highly likely not to be looking at the class screen 60. Therefore, the estimation method determination unit 52 determines, as parameters, fixation on a position with attractiveness (the teacher's face) and head movement (the presence or absence of nodding). Further, the parameter value calculation unit 53 weights the fixation on the position with attractiveness at 0.3 and the head movement at 0.7.

[0059] Also, when the teacher sets the time for silent reading of a passage and in a classroom situation where the teacher is not speaking and a text such as a passage is being displayed, it is presumed that students with a low blink frequency are highly likely to be reading the passage on the class screen 60. Therefore, the estimation method determination unit 52 determines, as parameters, the presence or absence of a specific eye movement pattern (a movement pattern for reading a passage), head movement (the presence or absence of a movement to bring the face closer to the screen), and blink frequency. Further, the estimation method determination unit 52 weights the presence or absence of a specific eye movement pattern at 0.4, the head movement at 0.2, and the blink frequency at 0.4.

[0060] Also, in a classroom situation where the teacher's face is not displayed on the class screen 60 and there is video content, it is presumed that students with a lot of eye movement are highly likely to be looking at the video content. Therefore, the estimation method determination unit 52 determines, as parameters, fixation on a position with attractiveness (a specific image of the video content) and the eye line correlation of multiple students. Further, the estimation method determination unit 52 weights the fixation on the position with attractiveness at 0.8 and the eye line correlation of multiple students at 0.2.

[0061] The estimation method determination unit 52 may determine any one of a plurality of parameters for calculating the attentiveness and determine the weighting of the determined parameter based only on the classroom situation specified by the classroom situation specification unit 50.

[0062] For example, as shown in FIG. 11, in a classroom situation where the teacher is facing forward and speaking, and no text or other sentences are being displayed, the estimation method determination unit 52 determines, as parameters, the fixation on a position with eye-catching property (the teacher's face) and the movement of the head (the presence or absence of nodding). Further, the parameter value calculation unit 53 assigns a weight of 0.7 to the fixation on the position with eye-catching property and a weight of 0.3 to the movement of the head.

[0063] Also, when the teacher sets a time for silent reading of the text, and in a classroom situation where the teacher is not speaking and text or other sentences are being displayed, the estimation method determination unit 52 determines, as parameters, the presence or absence of a specific eye movement pattern (a movement pattern for reading the text) and the movement of the head (the presence or absence of a movement to bring the face closer to the screen). Further, the estimation method determination unit 52 assigns a weight of 0.8 to the presence or absence of the specific eye movement pattern and a weight of 0.2 to the movement of the head.

[0064] Also, in a classroom situation where the teacher's face is not displayed on the classroom screen 60 and there is video content, the estimation method determination unit 52 determines, as parameters, the fixation on a position with eye-catching property (a specific image of the video content) and the line-of-sight correlation among multiple students. Further, the estimation method determination unit 52 assigns a weight of 0.5 to the fixation on the position with eye-catching property and a weight of 0.5 to the line-of-sight correlation among multiple students.

[0065] The parameter value calculation unit 53 calculates the parameter values of the determined parameters. Note that the parameter values are normalized in the range from the lowest 0 to the highest 1.

[0066] For example, when the line-of-sight correlation is determined as a parameter, the parameter value calculation unit 53 normalizes the correlation value of the line-of-sight information with other students, for example, using Pearson's product-moment correlation coefficient, and calculates it as the parameter value.

[0067] In addition, when the line of sight to a position with eye-catching property is determined as a parameter, the parameter value calculation unit 53 determines the position with eye-catching property from the class screen 60 by image analysis or the like, and normalizes the deviation amount between the determined position with eye-catching property and the fixation point of the student on the class screen 60 based on the line of sight information to calculate the parameter value.

[0068] In addition, when the expression is determined as a parameter, the parameter value calculation unit 53 identifies the facial expressions of the teacher and students using face recognition technology for the class video and the student video respectively. Then, the parameter value calculation unit 53 normalizes the degree of coincidence of the facial expressions of the teacher and students and calculates it as the parameter value.

[0069] In addition, when the line of sight position accumulation is determined as a parameter, the parameter value calculation unit 53 calculates a parameter value obtained by normalizing how much of the same location is being looked at based on the line of sight information.

[0070] In addition, when the movement of the body or head is determined as a parameter, the parameter value calculation unit 53 detects the presence or absence of a specific body or head movement from the student video using posture detection technology and normalizes it to calculate it as the parameter value.

[0071] In addition, when the blink frequency or duration is determined as a parameter, the parameter value calculation unit 53 calculates the blink frequency or duration from the student video using expression recognition technology and normalizes it to calculate it as the parameter value.

[0072] In addition, when the presence or absence of a specific line of sight movement pattern is determined as a parameter, the parameter value calculation unit 53 normalizes the presence or absence of the specific line of sight movement pattern based on the line of sight information and calculates it as the parameter value.

[0073] The interest degree estimation unit 54 multiplies the parameter value calculated by the parameter value calculation unit 53 and the weight of the parameter determined by the estimation method determination unit 52 for each parameter, and adds them together to calculate the attentiveness (any value between 0 and 1).

[0074] The listening degree calculated here indicates that the closer it is to 0, the less the class is being watched, and the closer it is to 1, the more the class is being watched.

[0075] [6.2. Notification Processing] When the listening degree is calculated by the interest degree estimation unit 54, the notification unit 55 gives a notification corresponding to the calculated listening degree to the teacher-side terminal device 2.

[0076] However, before giving a notification corresponding to the listening degree to the teacher-side terminal device 2, the notification unit 55 can give a notification to the student-side terminal device 3 used by the student with a low listening degree to reconfirm whether the student is watching the class.

[0077] Here, a threshold value for determining whether the student is watching the class is set. Also, a predetermined value α for giving a margin to the threshold value is set accordingly. When the listening degree is less than or equal to (threshold value - α), it is determined that the student is clearly not watching the class. Also, when the listening degree is within (threshold value ± α), since there is a possibility that the student is not watching the class, a notification for reconfirming the listening degree is given.

[0078] FIG. 12 is a diagram for explaining the notification at the time of reconfirming the listening degree. On the student-side terminal device 3 to which a notification for reconfirming whether the student is watching the class is given, as shown in FIG. 12, a class screen 60 with a caption image 81 for guiding the line of sight superimposed on the class scene is displayed.

[0079] Then, the CPU 40 of the server 4 recalculates the listening degree based on the student data transmitted after the caption image 81 is displayed.

[0080] As a result, if the student is watching the class, they will notice the presence of the caption image 81 and move their line of sight to the class screen 60, so the listening degree will increase. On the other hand, if the student is not watching the class, they will not notice the presence of the caption image 81 and the listening degree will remain low.

[0081] Note that the data of the caption image 81 may be stored in advance in the teacher terminal device 2 or the server 4 and transmitted to the student terminal device 3, or may be generated in the teacher terminal device 2 or the server 4 according to the class situation. Also, it is desirable that the caption image 81 has content that does not make the student notice that the listening degree is being reconfirmed.

[0082] Next, a specific example will be given and explained for the notification according to the listening degree calculated by the interest degree estimation unit 54.

[0083] FIG. 13 is a diagram for explaining a first example of the notification of the listening degree. In the first example, the notification unit 55 transmits the data of the listening degree notification image 82 in which the calculated listening degree is graphically represented as a bar graph to the teacher terminal device 2.

[0084] In the teacher terminal device 2 that has received the data of the listening degree notification image 82, as shown in FIG. 13, the listening degree notification image 82 is displayed on a part of the student screen 61. In the listening degree notification image 82, the graph in which the listening degree is less than or equal to (threshold value - α) is shown in a color different from other graphs. Thereby, the teacher can immediately grasp that there is a student with a low listening degree.

[0085] Also, when there is a student whose listening degree is less than or equal to (threshold value - α), the notification unit 55 transmits an instruction to output a predetermined voice from the speaker 28 to the teacher terminal device 2. Thereby, the teacher can immediately grasp that there is a student with a low listening degree.

[0086] FIG. 14 is a diagram for explaining a second example of the notification of the listening degree. In the second example, the notification unit 55 transmits information grouping students whose listening degree is, for example, equal to or higher than the threshold value and students whose listening degree is less than the threshold value to the teacher terminal device 2.

[0087] Then, in the teacher-side terminal device 2, as shown in FIG. 14, based on the grouped information, on the student screen 61, the student images with a listening degree equal to or higher than the threshold (high) and the student images with a listening degree lower than the threshold (low) are separately displayed.

[0088] Thereby, the teacher can easily grasp the students who are watching the class and the students who are not watching the class.

[0089] FIG. 15 is a diagram for explaining a third example of the notification of the listening degree. In the third example, the notification unit 55 transmits information indicating students with a listening degree equal to or higher than the threshold and students with a listening degree lower than the threshold to the teacher-side terminal device 2, so that the viewing icon 83 or the non-viewing icon 84 is displayed together with each student image.

[0090] In the teacher-side terminal device 2, as shown in FIG. 15, a viewing icon 83 indicating that the class is being watched is displayed together with the student image of a student with a listening degree equal to or higher than the threshold. Also, in the teacher-side terminal device 2, a non-viewing icon 84 indicating that the class is not being watched is displayed together with the student image of a student with a listening degree lower than the threshold.

[0091] Thereby, the teacher can easily grasp the students who are watching the class and the students who are not watching the class.

[0092] FIG. 16 is a diagram for explaining a fourth example of the notification of the listening degree. In the fourth example, the notification unit 55 performs a notification according to the parameter used when calculating the listening degree. For example, when there is a line-of-sight pattern of reading a text as a line-of-sight movement pattern of a specific student, the notification unit 55 transmits the information that the text is being read in association with that student to the teacher-side terminal device 2. Also, when it is specified that the frequency of blinking is high, the notification unit 55 transmits the information of having drowsiness in association with that student to the teacher-side terminal device 2. Also, when the movement of the nodding head is detected, the notification unit 55 transmits the information of having nodded in association with that student to the teacher-side terminal device 2.

[0093] Then, when the teacher-side terminal device 2 receives information according to the parameter, as shown in FIG. 16, it displays a parameter icon 85 corresponding to the information according to the parameter together with the student image. This enables the teacher to easily grasp the state of the students.

[0094] FIG. 17 is a diagram for explaining a fifth example of the notification of the listening degree. In the fifth example, an instruction is given to the teacher-side terminal device 2 to output noise having a volume corresponding to the listening degree from the speaker 28. For example, as shown in FIG. 17, the notification unit 55 calculates the average value of the listening degrees of all students, and gives an instruction to the teacher-side terminal device 2 so that the volume of the noise becomes larger as the calculated average value of the listening degrees is lower. In the teacher-side terminal device 2 that has received the instruction, noise is output from the speaker 28 at the volume indicated in the instruction.

[0095] This enables the teacher to take actions such as suggesting to name a student when the noise is loud. That is, the notification unit 55 enables the teacher to take a usage method such that the listening degree of the student increases and the noise becomes smaller.

[0096] <7. Flow of the degree of interest estimation notification process> FIG. 18 is a flowchart showing an example of the flow of the degree of interest estimation notification process. The CPU 40 of the server 4 executes the degree of interest estimation notification process, for example, at predetermined intervals, when the scene of the class changes (switches), for each frame of the class video transmitted, and so on.

[0097] As shown in FIG. 18, the class situation specifying unit 50 specifies the teacher state based on the class data in step S1, and specifies the class scene based on the class data in step S2. That is, in step S1 and step S2, the class situation specifying unit 50 specifies the class situation based on the class data.

[0098] In step S3, the estimation method determination unit 52 reads out the student characteristics stored in the storage unit 43. The student characteristics have been previously specified by the student characteristic specifying unit 51.

[0099] In step S4, based on the class situation identified in steps S1 and S2 and the student characteristics read in step S3, the estimation method determination unit 52 determines the parameters to be used when calculating the listening degree and the weighting of the parameters.

[0100] In step S5, the parameter value calculation unit 53 calculates the parameter values of the parameters determined in step S4. Then, in step S6, the interest degree estimation unit 54 calculates the listening degree based on the calculated parameter values and the weighting.

[0101] In step S7, the notification unit 55 determines whether the calculated listening degree is less than or equal to (threshold - α). As a result, if the listening degree is less than or equal to (threshold - α) (Yes in step S7), in step S8, the notification unit 55 sends the above notifications such as the first to fifth examples to the teacher-side terminal device 2 to cause the teacher to send a notification according to the listening degree.

[0102] On the other hand, if the listening degree is not less than or equal to (threshold - α) (No in step S7), in step S9, the notification unit 55 determines whether the listening degree is within (threshold ± α). And if the listening degree is within (threshold ± α) (Yes in step S9), in step S10, the notification unit 55 instructs the student-side terminal device 3 used by the student whose listening degree is within (threshold ± α) to display the caption image 81 for reconfirming the listening degree. After that, in step S11, the interest degree estimation unit 54 calculates the listening degree again.

[0103] In step S12, the notification unit 55 determines whether the reconfirmed listening degree is less than or equal to (threshold - α). As a result, if the listening degree is less than or equal to (threshold - α) (Yes in step S12), the process proceeds to step S8.

[0104] If the listening degree is not within (threshold ± α) (No in step S9), or if the reconfirmed listening degree is not less than or equal to (threshold - α) (No in step S12), the process ends.

[0105] <8. Other Configuration Examples of the Online Class System 1> Note that the embodiments are not limited to the specific examples described above, and configurations as various modifications can be adopted.

[0106] For example, the online class system 1 is configured to include the server 4. However, the online class system 1 may not include the server 4. In that case, the CPU 20 of the teacher-side terminal device 2 or the CPU 30 of the student-side terminal device 3 may function as the student characteristic identification unit 51, the estimation method determination unit 52, the parameter value calculation unit 53, the degree of interest estimation unit 54, and the notification unit 55.

[0107] Also, the student characteristic identification unit 51, the estimation method determination unit 52, the parameter value calculation unit 53, the degree of interest estimation unit 54, and the notification unit 55 may function in cooperation with the CPU 20 of the teacher-side terminal device 2, the CPU 30 of the student-side terminal device 3, and the CPU 40 of the server 4.

[0108] Also, in the above-described embodiment, the viewers (students) are made to view the class video and class audio as the content. However, the content is not limited to these and may be other things.

[0109] <9. Summary of the Embodiment> In the information processing apparatus (server 4) of the embodiment as described above, there are provided an estimation method determination unit 52 that determines an estimation method for estimating the degree of interest (listening degree) of a viewer (student) who views the content based at least on the situation of the content (class situation), and a degree of interest estimation unit 54 that estimates the degree of interest based on the estimation method and a plurality of parameters including parameters related to the viewer's line of sight. Thereby, the server 4 can determine an optimal estimation method according to the class situation. That is, the server 4 can accurately estimate the degree of interest reflecting the class situation.

[0110] Further, the estimation method determination unit 52 determines an estimation method based on the content status and the characteristics of the viewer (student characteristics). As a result, the server 4 can determine an optimal estimation method according to the class situation and the characteristics of each student. That is, the server 4 can estimate the degree of interest more accurately by reflecting the class situation and student identification.

[0111] Also, the estimation method determination unit 52 determines any one of a plurality of parameters based on at least the content status, and the degree of interest estimation unit 54 estimates the degree of interest based on the determined parameter. As a result, since the server 4 estimates the degree of interest based on the parameter optimized according to the class situation from among a plurality of parameters, the degree of interest can be estimated more accurately.

[0112] Also, the estimation method determination unit 52 determines any one of a plurality of parameters based on at least the content status, determines the weighting of the determined parameter, and the degree of interest estimation unit 54 estimates the degree of interest based on the determined parameter and the determined weighting. As a result, since the server 4 estimates the degree of interest based on the parameter and weighting optimized according to the class situation from among a plurality of parameters, the degree of interest can be estimated more accurately.

[0113] Also, the estimation method determination unit 52 determines an estimation method when the content status changes. As a result, the server 4 can estimate the degree of interest with a new estimation method when the class situation changes, and can estimate the degree of interest considering the class situation more.

[0114] Also, the estimation method determination unit 52 determines an estimation method at predetermined intervals. As a result, even if the class situation changes, the server 4 can calculate the degree of interest according to the changed class situation.

[0115] In addition, the server 4 includes a notification unit 55 that notifies a provider (teacher) who provides content based on the degree of interest. Thereby, the teacher can easily grasp the degree of interest of the students.

[0116] In addition, the notification unit 55 sends a notification to viewers with a low degree of interest to reconfirm their degree of interest. Thereby, the server 4 can induce viewers with a low degree of interest to watch the content.

[0117] In addition, the notification unit 55 notifies the provider of the degree of interest. Thereby, the server 4 can easily let the teacher grasp the degree of interest of the students.

[0118] In addition, the notification unit 55 notifies high-interest viewers and low-interest viewers separately. Thereby, the server 4 can easily let the teacher distinguish and grasp high-interest viewers and low-interest viewers.

[0119] In addition, the notification unit 55 displays an icon according to the viewer's degree of interest. Thereby, the server 4 can easily let the teacher grasp the degree of interest of the students.

[0120] In addition, the notification unit 55 displays an icon according to the viewer's parameters. Thereby, the server 4 can easily let the teacher grasp the state regarding the students' parameters.

[0121] In addition, the notification unit 55 makes a voice notification at a volume according to the degree of interest. Thereby, the server 4 can easily let the teacher grasp the degree of interest of the students by the volume.

[0122] As described above, in the information processing method of the embodiment, the information processing apparatus determines an estimation method based on at least the situation of the content, and estimates the degree of interest of the viewer based on the estimation method and a plurality of parameters including parameters related to the line of sight of the viewer who views the content. As described above, the program of the embodiment determines an estimation method based on at least the situation of the content, and estimates the degree of interest of the viewer based on the estimation method and a plurality of parameters including parameters related to the line of sight of the viewer who views the content.

[0123] Such a program can be pre-recorded in an HDD as a recording medium built in a device such as a computer device, or in a ROM in a microcomputer having a CPU. Alternatively, it can be temporarily or permanently stored (recorded) in a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a Blu-ray Disc (registered trademark), a magnetic disk, a semiconductor memory, or a memory card. Such a removable recording medium can be provided as so-called package software. In addition, such a program can be installed from a removable recording medium to a personal computer or the like, or can also be downloaded from a download site via a network such as a LAN (Local Area Network) or the Internet.

[0124] Note that the effects described in this specification are merely examples and are not limited, and there may be other effects.

[0125] <10. The present technology> The present technology can also adopt the following configuration. (1) An estimation method determination unit that determines an estimation method for estimating the degree of interest of a viewer who views the content based on at least the situation of the content; An interest degree estimation unit that estimates the degree of interest based on the estimation method and a plurality of parameters including parameters related to the viewer's line of sight; An information processing apparatus comprising the above. (2) The estimation method determination unit determines the estimation method based on the situation of the content and the characteristics of the viewer. The information processing apparatus according to (1). (3) The estimation method determination unit determines any one of the plurality of parameters based on at least the situation of the content. The interest degree estimation unit estimates the degree of interest based on the determined parameter. The information processing apparatus according to (1) or (2). (4) The estimation method determination unit determines any one of the plurality of parameters based on at least the situation of the content, and determines the weighting of the determined parameter. The interest degree estimation unit estimates the degree of interest based on the determined parameter and the determined weighting. The information processing apparatus according to (3). (5) The estimation method determination unit determines the estimation method when the situation of the content changes. The information processing apparatus according to any one of (1) to (4). (6) The estimation method determination unit determines the estimation method at predetermined intervals. The information processing apparatus according to any one of (1) to (4). (7) The apparatus further comprises a notification unit that notifies a provider who provides the content based on the degree of interest. The information processing apparatus according to any one of (1) to (6). (8) The notification unit notifies the viewer with a low degree of interest to reconfirm the degree of interest. The information processing apparatus according to (7). (9) The notification unit notifies the provider of the degree of interest. The information processing apparatus according to (7) or (8). (10) The notification unit notifies the viewers with a high degree of interest and the viewers with a low degree of interest separately. The information processing apparatus according to any one of (7) to (9). (11) The notification unit displays an icon according to the degree of interest of the viewer. The information processing apparatus according to any one of (7) to (10). (12) The notification unit displays an icon according to the parameter of the viewer. The information processing apparatus according to any one of (7) to (11). (13) The notification unit causes a voice notification to be performed at a volume according to the degree of interest. The information processing apparatus according to any one of (7) to (12). (14) An information processing apparatus determines an estimation method based on at least the situation of the content, and estimates the degree of interest of the viewer based on the estimation method and a plurality of parameters including parameters related to the line of sight of the viewer who views the content. An information processing method. (15) determines an estimation method based on at least the situation of the content, and estimates the degree of interest of the viewer based on the estimation method and a plurality of parameters including parameters related to the line of sight of the viewer who views the content. A program for causing an information processing apparatus to execute the process.

Explanation of Signs

[0126] 1 Online class system 2 Teacher-side terminal device 3 Student-side terminal device 4 Server 40 CPU 50 Class status determination unit 51 Student characteristic determination unit 52 Estimation method determination unit 53 Parameter value calculation unit 54 Degree of interest estimation unit 55 Notification unit

Claims

1. An estimation method determination unit that determines an estimation method for estimating the degree of interest of a viewer who views the content based at least on the situation of the content; An interest degree estimation unit that estimates the degree of interest based on the estimation method and a plurality of parameters including parameters related to the viewer's line of sight; An information processing apparatus comprising the above.

2. The estimation method determination unit determines the estimation method based on the situation of the content and the characteristics of the viewer. The information processing apparatus according to Claim 1.

3. The estimation method determination unit determines any one of the plurality of parameters based at least on the situation of the content, and the interest degree estimation unit estimates the degree of interest based on the determined parameter. The information processing apparatus according to Claim 1.

4. The estimation method determination unit determines any one of the plurality of parameters based at least on the situation of the content, and also determines the weighting of the determined parameter, and the interest degree estimation unit estimates the degree of interest based on the determined parameter and the determined weighting. The information processing apparatus according to Claim 3.

5. The estimation method determination unit determines the estimation method when the situation of the content changes. The information processing apparatus according to Claim 1.

6. The estimation method determination unit determines the estimation method at predetermined intervals. The information processing apparatus according to Claim 1.

7. Comprising a notification unit that notifies a provider who provides the content based on the degree of interest. The information processing apparatus according to Claim 1.

8. The notification unit sends a notification to reconfirm the degree of interest to the viewer with a low degree of interest. The information processing apparatus according to Claim 7.

9. The notification unit notifies the provider of the degree of interest. The information processing apparatus according to Claim 7.

10. The notification unit notifies separately the viewers with a high degree of interest and the viewers with a low degree of interest. The information processing apparatus according to Claim 7.

11. The notification unit displays an icon according to the degree of interest of the viewer. The information processing apparatus according to Claim 7.

12. The notification unit displays an icon according to the parameter of the viewer. The information processing apparatus according to Claim 7.

13. The notification unit causes an audio notification to be made at a volume according to the degree of interest. The information processing apparatus according to claim 7.

14. The information processing apparatus determines an estimation method based on at least the content situation, and estimates the degree of interest of the viewer based on the estimation method and a plurality of parameters including parameters related to the line of sight of the viewer who views the content. Information processing method.

15. determines an estimation method based on at least the content situation, and estimates the degree of interest of the viewer based on the estimation method and a plurality of parameters including parameters related to the line of sight of the viewer who views the content. A program for causing an information processing apparatus to execute the process.

Images