Information processing apparatus, information processing method, and program

Abstract

To more appropriately determine the degree of understanding of a student who learns by using teaching materials.SOLUTION: An information processing apparatus acquires a picked-up image obtained by imaging means picking up an image of a person and teaching materials used by the person, detects, from the picked-up image, a unit in which the person has an interest, of units constituting the teaching materials, specifies, from the picked-up image, the state of the person when the person has an interested in the detected unit, and determines the degree of understanding of the person on the unit detected by the detection means on the basis of the specified state of the person.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to processing based on captured images.

Background Art

[0002] As a method of utilizing video data in the educational field, a method of following students based on video data obtained by imaging students attending a lecture has been proposed.

[0003] Patent Document 1 proposes a method of estimating the degree of understanding of a lecture video by a student from video data obtained by imaging the student watching the lecture video.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The method described in Patent Document 1 is a method of estimating the degree of understanding of a lecture video by a student, and even when the degree of understanding of the student is low, information for estimating the reason cannot be obtained. Further, with the method of Patent Document 1, it is not possible to estimate which part of the textbook used by the student during the lecture is not understood. For this reason, the instructor may not be able to appropriately follow up on students who are studying using the textbook.

Means for Solving the Problems

[0006] The information processing apparatus of the present disclosure includes: an acquisition unit that acquires a captured image obtained by an imaging unit capturing a person and a teaching material used by the person; a detection unit that detects, from the captured image, a unit among the units constituting the teaching material that the person is paying attention to; a specification unit that specifies, from the captured image, the state of the person when the person is paying attention to the unit detected by the detection unit; and a determination unit that determines the degree of understanding of the person with respect to the unit detected by the detection unit based on the state of the person specified by the specification unit.

Advantages of the Invention

[0007] According to the technology of the present disclosure, the degree of understanding of a student learning using a teaching material can be determined more appropriately.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0009] The description of the embodiments will be made with reference to the accompanying drawings. The following embodiments do not limit the technology according to the claims. Although a plurality of features are described in the following embodiments, not all of these plurality of features are essential to the technology of the present disclosure. The plurality of features may be arbitrarily combined. The following embodiments exemplify the technology of the present disclosure and do not show it in a restrictive manner. The technology of the present disclosure can be implemented in various other modified and changed forms. In the accompanying drawings, the same or similar configurations are assigned the same reference numerals, and duplicate descriptions are omitted.

[0010] <Embodiment 1> [Hardware Configuration] FIG. 1 is a diagram showing the hardware configuration of the information processing apparatus in the present embodiment. The information processing apparatus 101 includes a CPU 102, a ROM 103, a RAM 104, and a storage unit 105, and each configuration is connected by a bus 106.

[0011] The CPU 102 controls the operations of each part in the information processing apparatus 101 in accordance with the contents of the ROM 103 or the RAM 104, or executes the program loaded into the RAM 104.

[0012] Note that it may have one or more dedicated hardware different from the CPU 102 or a GPU (Graphics Processing Unit), and at least a part of the processing by the CPU 102 may be performed by the GPU or the dedicated hardware. Examples of the dedicated hardware include an ASIC (Application Specific Integrated Circuit) and a DSP (Digital Signal Processor).

[0013] The ROM 103 is a read-only memory that stores a boot program, firmware, various processing programs for realizing the processes described later, and various data. The RAM 104 is a work memory that temporarily stores programs and data for the CPU 102 to perform processing. Various processing programs and data are loaded by the CPU 102. The storage unit 105 is a recording medium that stores changeable data such as a hard disk drive and a solid state drive, and stores programs and data related to various processes of the present embodiment.

[0014] The camera 108 is an imaging unit installed near the students attending the lecture, and captures the students attending the lecture and teaching materials such as textbooks in the hands of the students attending the lecture to generate a captured image. The camera 108 transmits the data of the generated captured image to the information processing apparatus 101 via the network 107. The captured image may be either a still image or a moving image (video). In the present embodiment, the captured image obtained by the camera 108 is described as a moving image (video).

[0015] The camera 108 captures an image so that a captured image including the students attending the lecture and textbooks is obtained. For this purpose, for example, as the camera 108, an imaging device equipped with a wide-angle lens that can cover the students attending the lecture and textbooks in the same angle of view is used. Alternatively, the camera 108 may be composed of two imaging devices, an imaging device for capturing the students attending the lecture and an imaging device for capturing textbooks. The network 107 is, for example, a communication means using a wired cable such as a LAN cable. The network 107 may be a wireless communication means such as Wi-Fi.

[0016] The information processing apparatus 101 may also have at least one of a display unit and an operation unit (not shown). Alternatively, there may be at least one of a display unit and an operation unit connected to the information processing apparatus 101. The display unit is composed of, for example, a liquid crystal display, an LED, etc., and displays a GUI (Graphical User Interface) or the like for the user to operate the information processing apparatus 101. The operation unit is composed of, for example, a keyboard, a mouse, a joystick, a touch panel, etc., and receives operations by the user and inputs various instructions to the CPU 102. The CPU 102 operates as a display control unit for controlling the display unit and an operation control unit for controlling the operation unit. The display unit may display information regarding the degree of understanding of the learning of the subject, which will be described later. In addition to or instead of the display unit, the information regarding the degree of understanding may be notified by another notification means such as a speaker.

[0017] [Functional Configuration of Information Processing Apparatus] Educational materials such as textbooks are composed of learning units called units. Also, at schools or cram schools, classes are advanced so that learning of a unit that is prerequisite knowledge is carried out first, and then learning of a unit that uses the prerequisite knowledge is carried out. For example, regarding arithmetic in the lower grades of elementary school, the order of the units is determined such that after learning the units of fractions and the units of the four arithmetic operations, learning of the unit of the four arithmetic operations using fractions is carried out.

[0018] The information processing apparatus according to the present embodiment is also an apparatus capable of estimating the degree of understanding, which is a value indicating how well a student taking a lecture understands the prerequisite knowledge required to understand the content of the lecture. For example, when a student is taking a lecture on the unit of the four arithmetic operations using fractions, not only the degree of understanding of the four arithmetic operations using fractions but also the degrees of understanding of the units of fractions and the four arithmetic operations, which are prerequisite knowledge, can be estimated. Therefore, according to the present embodiment, the lecturer can appropriately follow up with the students after the lecture.

[0019] Figure 2 is a block diagram showing the functional configuration of the information processing apparatus 101. The information processing apparatus 101 includes an image reception unit 201, a person motion specification unit 202, a unit of interest detection unit 203, a person state specification unit 204, and an understanding degree determination unit 205. The information processing apparatus 101 may include a notification unit that notifies information regarding the understanding degree. This notification unit may transmit information regarding the understanding degree to another device, or may display information regarding the understanding degree to the user, or may notify information regarding the understanding degree by voice.

[0020] The image reception unit 201 receives, via the network 107, the video obtained by the camera 108 capturing the student taking a lecture and the textbook used by the student. The person motion specification unit 202 specifies the motion of the student from the received video. The unit of interest detection unit 203 detects the unit that the student is interested in from the received video. The person state specification unit 204 specifies the state of the student from the received video. The understanding degree determination unit 205 determines the understanding degree of the student with respect to the unit detected by the unit of interest detection unit 203 and the understanding degree of the curriculum tree described later.

[0021] Each of these functional units is realized by the CPU 102 of the information processing apparatus 101 executing a predetermined program, but is not limited thereto. For example, a GPU (Graphics Processing Unit) for accelerating the operation, or hardware such as an FPGA (Field Programmable Gate Array) may be used. Each functional unit may be realized by the cooperation of software and hardware such as an application specific integrated circuit, or some or all of the functions may be realized by hardware only. [Processing for the video of the student taking a lecture] FIG. 3 is a flowchart showing the detailed processing performed by the information processing apparatus 101. A series of processes shown in the flowchart of FIG. 3 are performed by the CPU 102 of the information processing apparatus 101 expanding and executing program codes stored in the ROM 103 in the RAM 104. Also, some or all of the functions of the steps in FIG. 3 may be realized by hardware such as an ASIC or an electronic circuit. Note that the symbol "S" in the description of each process means that it is a step in the flowchart, and the same applies to the subsequent flowcharts.

[0022] In S301, the image receiving unit 201 receives the video obtained by the camera 108 capturing an image. The person motion specifying unit 202 acquires a frame to be processed from among the captured images (frames) constituting the received video. For example, the first captured image (frame) is acquired.

[0023] Note that the camera 108 will be described as imaging one student who is attending a lecture on a unit included in a textbook while using the textbook. For example, it is assumed that the camera 108 is associated with the student, and it is specified which student the student included in the received video is. When there are a plurality of students, cameras corresponding to each student are installed. Alternatively, a video obtained by imaging a plurality of students with one camera may be received. In that case, a step of identifying a specific student who is the target of determining the degree of understanding from the received video is added, and the following steps are to be performed for the identified specific student.

[0024] In S302, the person motion specifying unit 202 specifies the orientation of the student's face and the position of the textbook from the frame to be processed. The method of specifying the orientation of the student's face and the position of the textbook from the frame is not limited. For example, as a method of specifying the orientation of the student's face and the position of the textbook, a method using a learned model and a method using pattern matching can be mentioned.

[0025] As a method for identifying the orientation of a student's face using a learned model, for example, a learned model generated by subjecting a learning model that outputs the orientation of a face when an image including a person is input to machine learning is prepared. Then, the frame to be processed is input to the learned model, and the orientation of the student's face can be identified from the output result. Further, as a method for identifying the orientation of a student's face using pattern matching, template images of face images facing a plurality of different directions are held in advance. Then, the area of the person's face is cut out from the frame to be processed, and the orientation of the face indicated by the template image having the highest similarity to the cut-out image can be identified as the orientation of the student's face in the frame to be processed.

[0026] Then, the person motion identification unit 202 determines whether the student is paying attention to the textbook in the frame to be processed. For example, when the direction of the student's face identified from the frame to be processed matches the direction in which the textbook is present, the person motion identification unit 202 determines that the student is paying attention to the textbook. The match shall include the case where the difference is within the threshold.

[0027] If the person motion identification unit 202 does not determine that the student is paying attention to the textbook (S302 is NO), the process proceeds to S303. In S303, the person motion identification unit 202 acquires the next frame and changes the acquired frame to the frame to be processed. Then, the person motion identification unit 202 executes S302 for the new frame to be processed.

[0028] If the person motion identification unit 202 determines that the student is paying attention to the textbook (S302 is YES), the process proceeds to S304.

[0029] In S304, the person motion identification unit 202 identifies whether the motion of the student taking the course in the frame to be processed is a page-turning motion. If it is identified that the motion of the student taking the course is a page-turning motion, it is determined that the student has started the page-turning motion. The method by which the person motion identification unit 202 identifies the motion of the student taking the course from the frame is not limited. For example, the motion of the student taking the course may be identified by the method of using a learned model or the method of using pattern matching as described above.

[0030] If the person motion identification unit 202 does not determine that the student taking the course has started the page-turning motion of the textbook (when S304 is NO), the process proceeds to S305. In S305, the person motion identification unit 202 acquires the next frame and changes the acquired frame to the frame to be processed. Then, the person motion identification unit 202 executes S304 for the new frame to be processed.

[0031] If the person motion identification unit 202 determines that the student taking the course has started the page-turning motion of the textbook (when S304 is YES), the process proceeds to S306.

[0032] In S306, the person motion identification unit 202 determines whether the student taking the course has finished the page-turning motion of the textbook from the frame to be processed.

[0033] If the person motion identification unit 202 does not determine that the student taking the course has finished the page-turning motion of the textbook (when S306 is NO), the process proceeds to S307. In S307, the person motion identification unit 202 acquires the next frame and changes the acquired frame to the new frame to be processed. Then, the person motion identification unit 202 executes S306 for the new frame to be processed.

[0034] If the person motion identification unit 202 determines that the student taking the course has finished the page-turning motion of the textbook (when S306 is YES), the process proceeds to S308.

[0035] In S308, the attention unit detection unit 203 detects the unit that the student is paying attention to in the frame to be processed. The method for detecting the unit that the student is paying attention to is not limited.

[0036] FIG. 4 is a diagram showing an example of a curriculum tree used in an arithmetic lecture for lower grades in elementary school. A unit is a unit of learning, and in FIG. 4, each unit is represented by a quadrilateral.

[0037] The detection of the unit that the student is paying attention to is performed, for example, as follows. A database in which units and pages are associated, generated based on the table of contents of the textbook, is stored in advance in the storage unit 105 of the information processing apparatus 101 or the like. Then, the attention unit detection unit 203 performs image analysis on the page numbers of the right page and the left page of the open textbook after the student has finished turning the pages, for the frame to be processed, and detects them. The attention unit detection unit 203 acquires the unit corresponding to the two detected page numbers from the database. If there is only one acquired unit, the acquired unit is acquired as the unit that the student is paying attention to. If there are a plurality of acquired units, the line of sight of the student is specified from the frame to be processed, and the unit associated with the page number at the tip of the specified line of sight is acquired as the unit that the student is paying attention to.

[0038] In S309, the person state identification unit 204 identifies, as the state of the student, the reaction of the student such as the expression or head movement of the student with respect to the unit detected in S308. In this embodiment, the reactions of the student that the person state identification unit 204 can identify are assumed to be "nodded", "smiled", "tilted the head", and "narrowed the eyes". Among these, "nodded" and "smiled" are used as positive reactions. Also, "tilted the head" and "narrowed the eyes" are assumed to be used as negative reactions. In this way, the person state identification unit 204 identifies the reaction of the student from among positive reactions and negative reactions.

[0039] FIG. 5 is a diagram showing an example of a table indicating the processing result of the flowchart shown in FIG. 3. The table in FIG. 5 shows the result of executing the flowchart of FIG. 3 based on the video obtained by the camera 108 capturing student A.

[0040] In the table of FIG. 5, columns 501 to 504 corresponding to the units detected in S308 are provided. In the table of FIG. 5, there are columns corresponding to the units of "5.1", "4.2", "3.2", and "3.1" as the columns of the detected units. In each column, the number of times of the reaction of the student identified by S309 after the unit is detected in S308 is held.

[0041] In the row 511 of "number of nods", the number of times the reaction identified by the person state identification unit 204 in S309 was "nodded" is held for each target unit. For example, in FIG. 5, in the column 502 where the target unit is "4.2" in the row 511 of "number of nods", "3" is held. This indicates that the number of times the person state identification unit 204 identified "nodded" as the reaction of the student after the target unit was detected as "4.2" in S308 was 3 times.

[0042] Also in FIG. 5, in the row 512 of "number of times of smiling", the number of times the reaction identified by the person state identification unit 204 in S309 was "smiled" is held for each unit. In the row 513 of "number of times of narrowing eyes", the number of times the reaction identified by the person state identification unit 204 in S309 was "narrowed eyes" is held for each unit. In the row 514 of "number of times of tilting head", the number of times the reaction identified by the person state identification unit 204 in S309 was "tilted head" is held for each unit. Thus, in this embodiment, the number of times of the reaction identified by S309 is held for each unit that the student focused on. When the person state identification unit 204 identifies a certain reaction in S309, 1 is added to the number of times of that reaction. The number of times of each reaction with respect to the unit that the student is focusing on shown in FIG. 5 is used to determine the degree of understanding of the student with respect to the unit. The method for determining the degree of understanding will be described later.

[0043] The method for identifying the reactions of the students, such as the expressions or head movements of the students from the frames, is not limited. For example, the state of the students may be identified by the method using a learned model or the method using pattern matching as described above.

[0044] Also, a plurality of consecutive frames from the currently processed frame may be used as the frames to be processed, and reactions such as the expressions or head movements of the students may be identified. For example, as a method for determining the number of reactions of the students in S309, when the reaction of the student is first identified, the process of acquiring the next frame and identifying the reaction is repeated, and it is determined whether the first identified reaction has ended. Then, when it is determined that the first identified reaction has ended, the value is held in the table of FIG. 5 as if the first identified reaction has occurred once.

[0045] In S310, the human motion identification unit 202 determines whether the student has stopped paying attention to the textbook. For example, the human motion identification unit 202 identifies the direction of the student's face from the frame next to the frame to be processed in S309, and when the direction of the student's face does not match the direction where the textbook exists, it is determined that the student has stopped paying attention to the textbook.

[0046] When the human motion identification unit 202 determines that the student has stopped paying attention to the textbook (S310 is YES), this flowchart ends. Or, when it is determined that the student has stopped paying attention to the textbook, if there is a next frame in the video received in S301, the human motion identification unit 202 may transition to S302 with the next frame as the frame to be processed. And when there is no next frame in the video received in S301 or there is an instruction to end from the user, the flowchart of FIG. 3 may end. Also, the flowchart of FIG. 3 may end even when the next frame cannot be acquired in S303, S305, and S307.

[0047] On the other hand, if the learner action specifying unit 202 does not determine that the student has stopped paying attention to the textbook (S310 is NO), the process proceeds to S305. In S305, the learner action specifying unit 202 acquires the next frame and changes the acquired frame to the processing target. Then, the learner action specifying unit 202 executes S304 for the newly targeted frame for processing.

[0048] [Regarding the determination process of the unit understanding level] FIG. 6 is a flowchart for explaining the determination process of the understanding level. The determination of the understanding level is a process of determining the understanding level for each unit and the overall understanding level for the curriculum tree using the information on the reactions of the students specified for each unit shown in FIG. 5.

[0049] In S601, the understanding level determination unit 205 determines the understanding level of the student for each unit based on the number of reactions of the student specified for each unit on which the student is focusing.

[0050] FIG. 7 is a diagram showing an example of the understanding level score used for determining the understanding level of the student for each unit. FIG. 7 pre-holds the understanding level scores corresponding to the reactions of the student that can be specified by the person state specifying unit 204. Positive reaction scores with positive values are associated with positive reactions such as "nodded" and "smiled". Negative reaction scores with negative values are associated with negative reactions such as "squinted eyes" and "tilted head".

[0051] Then, as shown in the following formula, the understanding level of each unit is determined by multiplying the number of reactions of the student corresponding to the unit by the understanding level score.

[0052] Understanding level of unit = (number of nods) × (understanding level score for "nodded") + (number of smiles) × (understanding level score for "smiled") + (number of squinted eyes) × (understanding level score for "squinted eyes") + (number of tilted heads) × (understanding level score for "tilted head")

[0053] FIG. 8 is a diagram showing a table summarizing the understanding levels of each unit of student A. When student A shows a reaction as shown in FIG. 5, the understanding level of student A for the unit of "5.1" is determined as (0×2)+(0×1)+(3× -1)+(4× -2)= -11. And the determined understanding level is retained in the table of FIG. 8.

[0054] For the understanding level of this embodiment, it is assumed that the larger the absolute value in the positive direction, the better the understanding of the unit, and the larger the absolute value in the negative direction, the less the understanding. When no reaction of the student regarding the understanding level as shown in FIG. 5 is identified for the unit the student is focusing on, the understanding level of the unit is determined to be 0.

[0055] The curriculum tree in FIG. 4 shows the learning order of the units. The student learns the units in order from the units below the curriculum tree. For example, in order for the student to learn the unit of "5.1", which is "Four arithmetic operations combining fractions and decimals", it is necessary to learn the unit of "3.2", which is "Four arithmetic operations of fractions", and the unit of "4.2", which is "Four arithmetic operations of decimals".

[0056] According to the information on the understanding level of student A shown in FIG. 8, it can be seen that the units student A focused on are "5.1", "3.2", "4.2", and "3.1". Assuming this student A was attending the lecture on the unit of "5.1", it is considered that student A was reviewing the units of "3.2" and "4.2", which are the prerequisite knowledge for the unit of "5.1" during the lecture. Furthermore, it is considered that this student A was reviewing the unit of "3.1", which is the prerequisite knowledge for the unit of "3.2".

[0057] According to the comprehension information in FIG. 8, the instructor can know that student A has a low comprehension level in the units of "5.1" and "3.2" and a high comprehension level in the unit of "4.1". Therefore, the instructor can recognize that student A had a low comprehension level in the lecture of the "5.1" unit attended this time. Furthermore, the instructor can also recognize the comprehension level of the unit that is the prerequisite knowledge for the "5.1" unit attended by student A. Therefore, the instructor can obtain the reason why the comprehension level of the lecture attended by the student is low from the information shown in FIG. 8, which is the processing result of S601.

[0058] Note that the comprehension score in FIG. 7 may be generated for each student. For example, the value of the comprehension score in FIG. 7 may be corrected according to the habits of the students. For example, for students who rarely nod usually, the absolute value of each comprehension score may be made larger than the value in FIG. 7 to determine the comprehension level. By thus correcting the comprehension score according to the student and determining the comprehension level, the variation among students can be suppressed.

[0059] Note that the method for determining the comprehension level for each unit is not limited to the method determined based on the number of responses of the student and the student score. Additionally, the person state specifying unit 204 may specify the reading speed of the student for each unit as the state of the student, and the comprehension determining unit 205 may determine the comprehension level based on the reading speed.

[0060] The reading speed for each unit is determined as follows, for example. The person state specifying unit 204 determines the time the student focuses on the textbook for each page. Then, based on the database in which the page number and the unit are associated, the time of focusing on each unit is determined. And for the reading speed of each unit, the following formula is used for calculation. Note that the "number of characters in the unit" in the following formula is the number of characters for each unit included in the textbook, and it may be stored in the information processing apparatus 101 in advance.

[0061] Reading speed of unit = (Number of characters in unit) / (Time of focusing on unit)

[0062] The comprehensibility determination unit 205 may determine the comprehensibility of a unit such that the lower the calculated reading speed of the unit, the lower the comprehensibility.

[0063] [Regarding the comprehensibility determination process of the curriculum tree] In S602, the comprehensibility determination unit 205 determines the overall comprehensibility of the curriculum tree for the trainee based on the comprehensibility of each unit determined in S601.

[0064] FIG. 9 is a diagram showing the depth levels of the curriculum tree in FIG. 4. As described above, the units located lower in the curriculum tree are more basic units and are prerequisite knowledge for learning many units. The depth level shown in FIG. 9 is set such that the higher the value of the depth level for the units located lower in the curriculum tree. In the present embodiment, as an example of a method for calculating the overall comprehensibility based on the unit comprehensibility, a method for determining the overall comprehensibility of the curriculum tree for the trainee using the depth level for the curriculum tree will be described.

[0065] The overall comprehensibility is determined by using the depth level as a value for weighting the comprehensibility. For example, the overall comprehensibility is the sum of the values obtained by multiplying the comprehensibility of the unit for the target trainee by the depth level (referred to as the "unit score"). If the units focused on by the trainee are units 1 to unit N, the overall comprehensibility is calculated using the following formula.

[0066] Overall comprehensibility = (Comprehensibility of unit 1) × (Depth level of unit 1) + (Comprehensibility of unit 2) × (Depth level of unit 2) + … + (Comprehensibility of unit N) × (Depth level of unit N)

[0067] For example, the overall comprehensibility based on the comprehensibility of the units shown in FIG. 8 is calculated as follows. Overall comprehensibility = (-11 × 1) + (5 × 2) + (-9 × 2) + (-1 × 3) = -22

[0068] The overall comprehension level calculated by this formula indicates that the larger the value in the positive direction, the better the student understands the units that are prerequisite knowledge. On the other hand, the larger the value in the negative direction, the less the student understands the units that are prerequisite knowledge. Units with a comprehension level of 0 do not affect the overall comprehension level.

[0069] FIG. 10 is a diagram showing an example of a notification screen for notifying the instructor of the comprehension level of the curriculum tree determined in S602. The notification screen in FIG. 10 includes the overall comprehension level and, as a breakdown of the overall comprehension level, for each unit, the unit score which is the value obtained by multiplying the comprehension level of the unit by the depth level. Also, as shown in FIG. 10, a message may be generated based on the unit score and the depth level, and the message may be included in the notification screen. For example, the message may be obtained by inputting an instruction text including the determined comprehension level and depth level, etc. into an external generation model.

[0070] By using the notification screen shown in FIG. 10, the instructor can follow up on the students more appropriately. For example, the instructor who checks the notification screen in FIG. 10 can appropriately set the priority order of the units to be supplemented based on the displayed unit scores. Also, the instructor who checks the notification screen in FIG. 10 can confirm whether the units "3.1" and "3.2" which are prerequisite knowledge are understood. And when the instructor determines based on the notification screen that there is prerequisite knowledge that is not understood, the instructor can focus on guiding the student on that unit. For example, the instructor can create a comprehension test that focuses on asking questions about units with low comprehension levels. Therefore, with a notification screen like FIG. 10, the comprehension level of the students with respect to the curriculum tree can be improved.

[0071] As described above, according to this embodiment, it is possible to detect from the captured image which unit of the teaching material the student is reviewing, and determine the student's comprehension level with respect to the units that are prerequisite knowledge for the content of the lecture. Therefore, according to this embodiment, it is possible to follow up on the students more appropriately.

[0072] In the present embodiment, it has been described that the information processing apparatus 101 performs processing on the captured image obtained by the camera 108 capturing the student attending the lecture, and determines the degree of understanding of the prerequisite knowledge of the lecture content of the student. In addition, for the captured image obtained by the camera 108 capturing a student who is studying on their own, such as a student who is studying using a reference book without attending a lecture, the information processing apparatus 101 may perform processing. In this case, the degree of understanding of each unit included in the reference book for the student can be determined. Also, the units for determining the degree of understanding are not limited to the units that are prerequisite knowledge, and for example, may also include units that have not been studied. In this case, the instructor can grasp the progress of the student's preview.

[0073] <Other Embodiments> The present disclosure is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads and executes the program. Further, the present invention can also be realized by a plurality of processors cooperating to perform processing.

[0074] The disclosure of the above-described embodiments includes the following configuration.

[0075] (Configuration 1) An acquisition unit that acquires a captured image obtained by an imaging unit imaging a person and a teaching material used by the person, A detection unit that detects, from the captured image, a unit that the person is paying attention to among the units constituting the teaching material, An identification unit that identifies, from the captured image, the state of the person when the person is paying attention to the unit detected by the detection unit, A determination unit that determines the degree of understanding of the unit detected by the detection unit by the person based on the state of the person identified by the identification unit, An information processing apparatus characterized by having the above.

[0076] (Configuration 2) The specific means specifies, as the state of the person, the expression of the person or the movement of the person's head. The information processing apparatus according to Configuration 1, characterized in that.

[0077] (Configuration 3) The specific means specifies, as the state of the person, the length of time the person has focused on the unit detected by the detection means. The information processing apparatus according to Configuration 1, characterized in that.

[0078] (Configuration 4) The determination means determines the degree of understanding of the person with respect to the unit detected by the detection means based on the length of time specified by the specific means and the reading speed of the person determined based on the number of characters of the unit detected by the detection means in the teaching material. The information processing apparatus according to Configuration 3, characterized in that.

[0079] (Configuration 5) The unit constituting the teaching material is a unit constituting a curriculum tree representing the order of learning, The determination means further determines the degree of understanding of the person with respect to the curriculum tree based on the degree of understanding of the person with respect to the unit detected by the detection means. The information processing apparatus according to any one of Configurations 1 to 4, characterized in that.

[0080] (Configuration 6) The determination means weights the degree of understanding of the person with respect to the determined unit based on the curriculum tree to determine the degree of understanding of the person with respect to the curriculum tree. The information processing apparatus according to Configuration 5, characterized in that.

[0081] (Configuration 7) Further comprising a display control means for displaying a screen including information regarding the degree of understanding determined by the determination means The information processing apparatus according to any one of Configurations 1 to 6, characterized in that

[0082] (Configuration 8) The screen includes a message generated based at least on the determined degree of understanding The information processing apparatus according to Configuration 7, characterized in that

[0083] (Configuration 9) The captured image is an image of a person attending a lecture on any one of the units constituting the teaching material The information processing apparatus according to any one of Configurations 1 to 8, characterized in that

[0084] (Configuration 10) The unit detected by the detection means includes a unit that is prerequisite knowledge for the unit being attended by the person The information processing apparatus according to Configuration 9, characterized in that

[0085] (Configuration 11) The captured image includes a plurality of students, Further comprising an identification means for identifying a specific student from among the plurality of students included in the captured image, The person is the specific student identified by the identification means The information processing apparatus according to any one of Configurations 1 to 10, characterized in that

[0086] (Configuration 12) An acquisition step of acquiring a captured image obtained by the imaging means of a person and the teaching material used by the person, A detection step of detecting, from the captured image, the unit among the units constituting the teaching material that the person is focusing on, A specifying step of specifying, from the captured image, the state of the person when the person is focusing on the detected unit, A determination step of determining the degree of understanding of the person with respect to the detected unit based on the state of the specified person An information processing method characterized by having

[0087] (Configuration 13) A program for causing a computer to execute each means of the information processing apparatus according to any one of Configurations 1 to 11

Explanation of Signs

[0088] 101 Information processing apparatus 108 Camera 201 Image reception unit 203 Attention unit detection unit 204 Person state specification unit 205 Degree of understanding determination unit

Claims

1. An acquisition means for acquiring a captured image obtained by an imaging means capturing a person and a teaching material used by the person; A detection means for detecting, from the captured image, a unit among the units constituting the teaching material that the person is paying attention to; An identification means for identifying, from the captured image, the state of the person when the person is paying attention to the unit detected by the detection means; A determination means for determining the degree of understanding of the person with respect to the unit detected by the detection means based on the state of the person identified by the identification means; An information processing apparatus characterized by comprising the above.

2. The identification means identifies, as the state of the person, the expression of the person or the movement of the head of the person The information processing apparatus according to claim 1, characterized by the above.

3. The identification means identifies, as the state of the person, the length of time the person has been paying attention to the unit detected by the detection means The information processing apparatus according to claim 1, characterized by the above.

4. The determination means Determines the degree of understanding of the person with respect to the unit detected by the detection means based on the length of time identified by the identification means and the reading speed of the person determined based on the number of characters of the unit detected by the detection means in the teaching material The information processing apparatus according to claim 3, characterized by the above.

5. The unit constituting the teaching material is a unit constituting a curriculum tree representing the order of learning, The determination means Further determines the degree of understanding of the person with respect to the curriculum tree based on the degree of understanding of the person with respect to the unit detected by the detection means The information processing apparatus according to claim 1, characterized by the above.

6. The determination means Performs weighting based on the curriculum tree on the degree of understanding of the person with respect to the determined unit, and determines the degree of understanding of the person with respect to the curriculum tree The information processing apparatus according to claim 5, characterized by the above.

7. Further comprising a display control means for displaying a screen including information regarding the degree of understanding determined by the determination means The information processing apparatus according to claim 1, characterized by the above.

8. The screen includes a message generated based at least on the determined degree of understanding The information processing apparatus according to claim 7, characterized by the above.

9. The captured image is a captured image of the person attending a lecture on any one of the units constituting the teaching material. The information processing apparatus according to claim 1, characterized in that.

10. The unit detected by the detection means includes a unit that is prerequisite knowledge for the unit being attended by the person. The information processing apparatus according to claim 9, characterized in that.

11. The captured image includes a plurality of students attending the lecture, The apparatus further has an identification means for identifying a specific student among the plurality of students included in the captured image, The person is the specific student identified by the identification means. The information processing apparatus according to claim 1, characterized in that.

12. An acquisition step of acquiring a captured image obtained by the imaging means of a person and the teaching material used by the person, A detection step of detecting, from the captured image, the unit that the person is focusing on among the units constituting the teaching material, A specifying step of specifying, from the captured image, the state of the person when the person is focusing on the detected unit, A determination step of determining the degree of understanding of the person with respect to the detected unit based on the specified state of the person, An information processing method, characterized by comprising:

13. A program for causing a computer to execute each means of the information processing apparatus according to any one of claims 1 to 11.

Images