Learning support device and learning support system

Abstract

A learning assistance device (100) is designed for a user (1) to perform a learning task, and is provided with: a first concentration degree estimation unit (16) that estimates a first concentration degree of the user (1) by analyzing information from an imaging means (10) for capturing an image of the user (1); a second concentration degree estimation unit (26) that estimates a second concentration degree of the user (1) by analyzing information actively inputted by the user (1) when the user (1) performs a learning task; and a switching unit (30) that switches the contents of the learning task and a method for presenting the learning task on the basis of the first concentration degree and / or the second concentration degree.

Description

[Technical field]

[0001] The present invention relates to a learning support device and a learning support system. [Background technology]

[0002] A device has been devised that measures the degree of concentration of a user when the user is performing a task. Patent Document 1 discloses a video playback device that measures the degree of concentration of a user on tasks that the user actively performs, such as writing an email or browsing the web, and tasks that the user passively performs, such as watching a video, and plays back a video according to the measured degree of concentration. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2007 / 132566 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the video playback device and the like disclosed in Patent Document 1 cannot appropriately switch between an operation that the user actively performs and an operation that the user passively performs.

[0005] Therefore, the present invention provides a learning support device etc. that can appropriately switch between an activity that is actively performed by the user and an activity that is passively performed by the user in accordance with the user's level of concentration. [Means for solving the problem]

[0006] A learning support device according to one embodiment of the present invention is a learning support device for a user to perform a learning task, comprising: a first concentration level estimation unit that analyzes information from an imaging means that images the user and estimates a first concentration level of the user; a second concentration level estimation unit that analyzes information actively input by the user when the user performs the learning task and estimates a second concentration level of the user; and a presentation switching unit that switches the content of the learning task and the presentation method of the learning task based on at least one of the first concentration level and the second concentration level. a first learning task presentation unit that presents to the user a first learning task for the user to actively learn; Equipped with While the first learning task presenting unit is presenting the first learning task to the user, the presentation switching unit switches the content presented to the user to the first learning task of a different difficulty level depending on the magnitude relationship between the first concentration level and the second concentration level. .

[0007] A learning assistance system according to one embodiment of the present invention is a learning assistance system for a user to perform a learning task, comprising: a display; an imaging means for imaging the user; a first concentration level estimation unit for estimating a first concentration level of the user by analyzing information from the imaging means; a second concentration level estimation unit for estimating a second concentration level of the user by analyzing information actively input by the user when the user performs the learning task; and a presentation switching unit for switching the content of the learning task and the presentation method of the learning task based on at least one of the first concentration level and the second concentration level; a first learning task presentation unit that presents to the user a first learning task for the user to actively learn; Equipped with While the first learning task presenting unit is presenting the first learning task to the user, the presentation switching unit switches the content presented to the user to the first learning task of a different difficulty level depending on the magnitude relationship between the first concentration level and the second concentration level. . Effect of the Invention

[0008] A learning assistance device or the like according to one aspect of the present invention can appropriately switch between an activity that is actively performed by a user and an activity that is passively performed by the user in accordance with the user's level of concentration. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram of a learning support device according to an embodiment. [Diagram 2] FIG. 2 is a flowchart showing the process of the learning support device according to the embodiment. [Figure 3A] FIG. 3A is a diagram showing a user performing an active task. [Figure 3B]FIG. 3B is a diagram showing a user performing a passive task. [Figure 4A] FIG. 4A is a diagram showing how the concentration level of a user is being measured while the user is performing an active task. [Figure 4B] FIG. 4B is a diagram showing how the concentration level of the user is being measured while the user is performing a passive task. [Diagram 5] FIG. 5 is a flowchart showing a process of determining a first concentration level performed by the learning support device according to the embodiment. [Figure 6] FIG. 6 is a diagram showing an example of a subject's habits that the learning support device according to the embodiment uses to determine the first concentration level. [Figure 7] FIG. 7 is a diagram showing time slots for comparing the first and second concentration degrees performed by the concentration degree determination unit according to the embodiment. [Figure 8] FIG. 8 is a diagram showing an outline of measurement of the first concentration level in the learning support device according to the embodiment. [Figure 9] FIG. 9 is a diagram showing an overview of measurement of the second concentration level in the learning support device according to the embodiment. [Figure 10] FIG. 10 is a diagram illustrating switching between an active task and a passive task in the learning support device according to the embodiment. [Figure 11] FIG. 11 is a table showing details of switching between active tasks and passive tasks in the learning support device according to the embodiment. [Figure 12] FIG. 12 is a flowchart illustrating an example of processing performed by the learning support device according to the embodiment. [Figure 13] FIG. 13 is a flowchart illustrating another example of the process of the learning support device according to the embodiment. [Figure 14] FIG. 14 is a diagram illustrating an example of a state of a user determined by comparing a first concentration degree with a second concentration degree in the learning support device according to the embodiment. [Figure 15] FIG. 15 is a diagram illustrating guidance of a user's state based on a comparison between a first concentration degree and a second concentration degree in the learning support device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments will be described with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, arrangement positions and connection forms of the components, etc. shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components that are not described in the independent claims will be described as optional components.

[0011] Note that each drawing is a schematic diagram and is not necessarily a precise illustration. In addition, in each drawing, the same reference numerals are used for substantially the same configurations, and duplicated explanations may be omitted or simplified.

[0012] (Embodiment) [Configuration of the learning support device] First, a configuration of the learning support device 100 will be described. Fig. 1 is a block diagram of the learning support device 100 according to an embodiment. The learning support device 100 includes an imaging means 10, a body movement / pose determination unit 12, a gaze / facial expression determination unit 14, a first concentration level estimation unit 16, a concentration level determination unit 18, an answer input unit 20, a first learning task presentation unit 22, an information processing unit 24, a second concentration level estimation unit 26, a second learning task presentation unit 28, and a presentation switching unit 30.

[0013] The image capturing means 10 captures an image of the user's face or body. The image capturing means 10 is realized by a Web camera built into a personal computer, or a digital camera connectable to a personal computer. The image capturing means 10 also has an eye tracking function. The image capturing means 10 may also be realized by an infrared camera. The image capturing means 10 transmits the acquired image data to the body movement and pose determination unit 12 and the gaze and facial expression determination unit 14.

[0014] Body movement / pose determination unit 12 recognizes the position of each of two or more parts of the user's body in the image acquired by imaging means 10. Body movement / pose determination unit 12 is also a processing device that calculates a target positional relationship, which is the positional relationship between each of the recognized positions of two or more parts of the user's body. Body movement / pose determination unit 12 is realized, for example, by a processor, a storage device, and a program stored in the storage device.

[0015] The body movement / pose determination unit 12 identifies the user's body from other parts on the image received from the image capture unit 10 by image recognition. The body movement / pose determination unit 12 also identifies the identified user's body by parts and recognizes the position on the image for each part. This calculates a target positional relationship, which is a positional relationship between two or more parts of the user's body on the image. Here, the positional relationship between two or more parts is indicated by the distance between the two or more parts. For example, if the two or more parts are "a part of the user's face" and "a user's hand", the body movement / pose determination unit 12 calculates the target positional relationship such as "a part of the face and a hand are within a specific distance". The body movement / pose determination unit 12 transmits the calculated target positional relationship to the first concentration degree estimation unit 16.

[0016] A plurality of images are acquired, and the object positional relationship is calculated for each of them. More specifically, the images acquired by the imaging means 10 are a moving image in which the images are arranged continuously in a time series. Therefore, the body movement / pose determination unit 12 determines whether the user is in a concentration state or not for each frame of the image included in the moving image. That is, the body movement / pose determination unit 12 outputs a numerical sequence in which values ​​indicating either a concentration state or a non-concentration state that is not a concentration state, corresponding to the captured moving image of the user, are arranged continuously in a time series, based on the determination.

[0017] The gaze / facial expression determination unit 14 identifies the gaze or facial expression of the user on the image received from the photographing means 10 by image recognition. The gaze / facial expression determination unit 14 acquires images from a near-infrared LED (Light Emitting Diode) and the photographing means 10, and performs calculation processing including image detection, a 3D eye model, and a gaze calculation algorithm. The gaze / facial expression determination unit 14 detects the gaze of the user looking at a display or the like. Specifically, the near-infrared LED generates a light reflection pattern on the user's cornea, and the photographing means 10 acquires the reflection pattern. Then, the gaze / facial expression determination unit 14 estimates the position of the eyeball in space and the viewpoint using an image processing algorithm and a physiological 3D model of the eyeball based on the reflection pattern. Note that the gaze / facial expression determination unit 14 can also be configured using natural light illumination and a visible light color camera, and the above configuration is just one example.

[0018] Furthermore, the gaze / facial expression determination unit 14 learns the user's face and the like by deep learning or the like, extracts features of the captured facial image of the user, and determines the user's facial expression based on the learned data and the extracted features. The gaze / facial expression determination unit 14 is realized by, for example, a processor, a storage device, and a program stored in the storage device. The gaze / facial expression determination unit 14 transmits information related to the estimated user's viewpoint or information related to the determined user's facial expression to the first concentration level estimation unit 16.

[0019] The first concentration level estimation unit 16 is a processing device that determines whether the user is in a concentration state based on the target positional relationship and the user's facial expression. The first concentration level estimation unit 16 is realized by, for example, a processor, a storage device, and a program stored in the storage device.

[0020] The first concentration level estimation unit 16 estimates the first concentration level of the user based on the target positional relationship acquired from the body movement / pose determination unit 12. The first concentration level estimation unit 16 grasps the user's habits in advance and determines whether the target positional relationship calculated by the body movement / pose determination unit 12 is the user's habit or not. If the target positional relationship matches the user's habit, it can be determined that the user is performing an action that can be taken when in a state of concentration. In other words, the first concentration level estimation unit 16 can determine that the user's first concentration level is high because the user is performing the relevant action. Here, the first concentration level is the concentration level when the user performs a task that is passively performed (hereinafter referred to as a passive task). An example of a passive task is watching a video.

[0021] In this specification, a habit is a movement that a person may take when he or she is in a state of concentration, and is a movement that is estimated from the positional relationship (i.e., distance) of two or more parts of the person's body. Therefore, a habit can be defined as the positional relationship of two or more parts of the person's body, or a movement that is estimated from the positional relationship.

[0022] The first concentration level estimation unit 16 uses the target positional relationship output by the body movement / pose determination unit 12 to calculate the ratio of the time the user was in a state of concentration to the total measured time within a preset time range as the concentration level of the user 1. For example, if the total time the user 1 was in a state of concentration was 4 minutes out of a 5-minute video, then 4 / 5=0.8 is calculated as the concentration level. First concentration estimation unit 16 Alternatively, the concentration may be calculated as 0.8 x 100 = 80% using a percentage.

[0023] The first concentration level estimation unit 16 estimates the first concentration level of the user based on information about the user's viewpoint estimated by the gaze / facial expression determination unit 14 or information about the user's facial expression determined. For example, the first concentration level estimation unit 16 determines that the user's first concentration level is high when there is little movement over time in the space of the user's viewpoint estimated by the gaze / facial expression determination unit 14. Also, for example, the first concentration level estimation unit 16 may predetermine a facial expression that indicates a high concentration level, and determine that the user's first concentration level is high when the gaze / facial expression determination unit 14 determines that the facial expression is that facial expression. First concentration level Estimation The unit 18 outputs the calculated first concentration degree of the user to the concentration degree determination unit 18.

[0024] The answer input unit 20 is an interface such as a terminal where the user inputs an answer or a screen for answer input presented to the user. The user inputs an answer to the question presented by the first learning task presentation unit 22 to the answer input unit 20. The answer input unit 20 transmits the acquired answer to the information processing unit 24. The answer input unit 20 is realized by, for example, a processor, a storage device, and a program stored in the storage device. The answer input unit 20 may include a display such as a touch panel display or a liquid crystal display, and an input button or a keyboard.

[0025] The first learning task presenting unit 22 is an interface such as a terminal or a screen that presents to the user a first learning task that the user actively learns. The first learning task presenting unit 22 presents to the user a first learning task that the user actively learns, such as a problem for intellectual training, such as a calculation problem, a problem related to knowledge of kanji, a problem related to English vocabulary, etc., and a problem that requires the user to input an answer. The first learning task is also called an active task. The first learning task presenting unit 22 is realized by, for example, a processor, a storage device, and a program stored in the storage device. The answer input unit 20 may include a display such as a touch panel display or a liquid crystal display. The first learning task presenting unit 22 transmits information on what kind of problem the first learning task presenting unit 22 is presenting to the answer input unit 20. The first learning task presenting unit 22 also presents a problem based on a signal from the presentation switching unit 30.

[0026] The information processing unit 24 acquires the answer input by the user from the answer input unit 20, and calculates indicators related to the question presented to the user, such as whether the answer is correct or incorrect, the progress speed of the question, the amount of processing for the question, the answer score, and the correct answer rate. The information processing unit 24 is realized, for example, by a processor, a storage device, and a program stored in the storage device.

[0027] The second concentration level estimation unit 26 acquires an index related to the problem presented to the user from the information processing unit 24, and estimates the second concentration level of the user from the index. Here, the second concentration level is the concentration level when the user performs a task actively performed (hereinafter, referred to as an active task). An actively performed task is, for example, answering a given problem. For example, the second concentration level estimation unit 26 estimates that the second concentration level of the user is high when the user has a high rate of correct answers to the problems. Alternatively, for example, the second concentration level of the user may be estimated to be high when the user progresses through the problems at a high speed. The second concentration level estimation unit 26 outputs the calculated second concentration level of the user to the concentration level determination unit 18. The second concentration level estimation unit 26 is realized, for example, by a processor, a storage device, and a program stored in the storage device.

[0028] The concentration level determination unit 18 determines the concentration level of the user using the first concentration level or the second concentration level acquired from the first concentration level estimation unit 16 or the second concentration level estimation unit 26. Specifically, when the learning support device 100 presents a question to the user, the concentration level determination unit 18 acquires the first concentration level and the second concentration level from the first concentration level estimation unit 16 or the second concentration level estimation unit 26, and determines the concentration level of the user by normalizing and comparing the first concentration level and the second concentration level.

[0029] Furthermore, when the learning support device 100 presents a video to the user, the concentration level determination unit 18 compares the first concentration level acquired from the first concentration level estimation unit 16 with the first value to determine the concentration level of the user. The concentration level determination unit 18 outputs information related to the determined concentration level of the user to the presentation switching unit 30. The concentration level determination unit 18 is realized by, for example, a processor, a storage device, and a program stored in the storage device.

[0030] The presentation switching unit 30 switches between the video and the questions to be presented on the display based on at least one of the first and second concentration levels. Based on information on the user's concentration level acquired from the concentration level determination unit 18, it determines how to switch the content presented to the user. For example, when the learning support device 100 is presenting a video to the user and the first concentration level is higher than the second concentration level, the presentation switching unit 30 determines to switch the content presented to the user to a video with a lower level of difficulty than the currently presented video. The presentation switching unit 30 is realized by, for example, a processor, a storage device, and a program stored in the storage device. The presentation switching unit 30 acquires a signal from the concentration level determination unit 18 and transmits a signal regarding the switching of the content presented to the user to the first learning task presentation unit 22 and the second learning task presentation unit 28.

[0031] The second learning task presenting unit 28 is an interface such as a terminal or a screen that presents to the user a second learning task that the user passively learns. The second learning task is, for example, a video. The second learning task is also called a passive task. The second learning task presenting unit 28 is realized, for example, by a processor, a storage device, and a program stored in the storage device. The second learning task presenting unit 28 may include a display such as a touch panel display or a liquid crystal display. The second learning task presenting unit 28 presents the video based on a signal from the presentation switching unit 30.

[0032] [Learning support device processing] Next, a description will be given of the processing performed by the learning support device 100. Fig. 2 is a flowchart showing the processing performed by the learning support device 100 according to the embodiment.

[0033] First, learning support device 100 presents a video or a question to the user (Step S100). Learning support device 100 presents a video through second learning task presenting unit 28, or presents a question through first learning task presenting unit 22.

[0034] Next, the learning support device 100 estimates the first concentration level or the second concentration level (step S101). The learning support device 100 estimates the first concentration level by the first concentration level estimation unit 16, or estimates the second concentration level by the second concentration level estimation unit 26.

[0035] Next, the learning support device 100 compares the first concentration degree with the second concentration degree, or determines the value of the first concentration degree (step S102). The learning support device 100, in the concentration degree determination unit 18, compares the first concentration degree with the second concentration degree, or determines the first concentration degree with the first value.

[0036] Then, the learning support device 100 switches the content to be presented to the user depending on the result of comparing the first concentration level with the second concentration level or the result of judging the value of the first concentration level (step S103). The learning support device 100 determines how to switch the content to be presented to the user using the presentation switching unit 30, and transmits the determined switching method to the first learning task presentation unit 22 or the second learning task presentation unit 28. The contents described in Figs. 1 and 2 will be described in detail below.

[0037] [Active and Passive Tasks] Next, the active task and the passive task presented by the learning support device 100 will be described in detail. The active task and the passive task are videos or questions presented in step S100 shown in FIG. 2. Here, the video (passive task) or the question (active task) presented in step S100 shown in FIG. 2 will be described. FIG. 3A is a diagram showing a state where the user 1 is performing an active task. Also, FIG. 3B is a diagram showing a state where the user 1 is performing a passive task.

[0038] The active task shown in FIG. 3A refers to a task in which the user 1 actively inputs answers, etc. The active tasks are specifically calculation problems, problems related to kanji, problems related to English vocabulary, other problems that require knowledge answers, geometric problems, problems that require reading and comprehension of sentences, etc. The passive task shown in FIG. 3B refers to a task in which the user 1 passively watches videos, etc. The passive task specifically refers to watching videos of lessons on arithmetic, Japanese, English, science, social studies, etc., watching musical performances, watching paintings or visual art works, watching plays, watching videos with educational content, etc.

[0039] FIG. 4A is a diagram showing how the concentration level of user 1 is measured while performing an active task. When the learning support device 100 presents an active task to user 1, user 1 looks at a question displayed on the display 2 or the like, and inputs an answer to the learning support device 100 through a keyboard, a touch panel, or the like. The learning support device 100 estimates a second concentration level of user 1 from task information. Here, the task information is the touch rate of the touch panel display when user 1 answers a question, the correct answer rate of the question, the response time until the answer is input, the progress speed of the question, the amount of processing of the question, the answer score, and the like. Note that user 1 may input an answer to the active task by voice through a microphone or the like.

[0040] Furthermore, while presenting the active task to the user 1, the learning support device 100 acquires an image of the face or body of the user 1 from the image capturing means 10. The learning support device 100 analyzes the acquired image and estimates a first concentration level of the user 1. Specifically, the learning support device 100 determines the facial expression of the user 1, information related to the line of sight or viewpoint of the user 1, a target positional relationship indicating the pose of the user 1, and the like, and estimates the first concentration level.

[0041] FIG. 4B is a diagram showing a state where the concentration level of the user 1 during the performance of a passive task is measured. When the learning support device 100 presents the passive task to the user 1, the user 1 watches a moving image displayed on the display 2 or the like. The learning support device 100 analyzes an image acquired by the image capturing means 10, and estimates a first concentration level of the user 1 from a face image of the user 1, information regarding the line of sight or viewpoint, an object positional relationship indicating the pose of the user 1, or a physiological index such as body temperature. The learning support device 100 may perform a more accurate analysis during the performance of a passive task than during the performance of an active task. The learning support device 100 may acquire a physiological index of the user 1, such as a pulse or body temperature, from a wearable device or a smartphone.

[0042] [First concentration estimate] Next, a description will be given of a process of estimating a first concentration level performed by the learning support device 100. Fig. 5 is a flowchart showing a process of determining a first concentration level performed by the learning support device 100 according to an embodiment. Here, a description will be given of the estimation of the first concentration level performed in step S101 shown in Fig. 2.

[0043] In this embodiment, the image capturing means 10 receives an image captured by the user 1, thereby acquiring the image, and performs an acquisition step (S101). The image capturing means 10 also transmits the acquired image to the body movement / pose determination unit 12.

[0044] Next, body movement / pose determination unit 12 distinguishes between the body of user 1 and other parts of the image received from imaging means 10 by image recognition, and further distinguishes between each part of user 1's body. Body movement / pose determination unit 12 recognizes the position on the image for each part of user 1's body. Body movement / pose determination unit 12 further performs a recognition step (S102) of calculating, from the recognized positions, a target positional relationship, which is the positional relationship between each combination of two or more parts of user 1's body on the image.

[0045] Next, the first concentration level estimation unit 16 performs a determination step of determining whether or not the user 1 is in a state of concentration based on the target positional relationship in the acquired image and the positional relationship of two or more parts of the body that define the habit of the user 1. The first concentration level estimation unit 16 acquires the positional relationship of two or more parts of the body that define the habit of the user 1, for example, by using habit information of the user 1 stored in a storage unit. The first concentration level estimation unit 16 further performs a determination of whether or not the user 1 is in a state of concentration by determining whether or not the positional relationship that corresponds to the habit of the user 1 is included in the target positional relationship calculated from the image (S103).

[0046] For example, if the positional relationship corresponding to the habit of user 1 corresponds (matches or can be considered equivalent to) the target positional relationship calculated from the image (Yes in S103), the first concentration level estimation unit 16 determines that user 1 is in the concentration state (S104). Also, for example, if the positional relationship corresponding to the habit of user 1 does not correspond to the target positional relationship calculated from the image (No in S103), the first concentration level estimation unit 16 determines that user 1 is not in the concentration state (S105).

[0047] Next, the first concentration level estimation unit 16 judges whether or not the user 1 is in a state of concentration for each image frame acquired by the image capture means 10. Here, the learning support device 100 in this embodiment calculates the concentration level of the user 1 in a preset time range. That is, the first concentration level estimation unit 16 judges whether or not the user 1 is in a state of concentration for a predetermined number (number of frames) of images corresponding to a preset time range.

[0048] The first concentration level estimation unit 16 judges whether the number of images for which the judgment has been performed has reached a predetermined number (S106), and if the predetermined number has not been reached (No in S106), repeats the acquisition step (S101) to step (S106). As a result, the first concentration level estimation unit 16 acquires images and judges whether the user 1 in the images is in a concentration state or not until the number of images for which the judgment has been performed reaches the predetermined number.

[0049] When the number of images for which the determination has been performed reaches a predetermined number (Yes in S106), the first concentration level estimation unit 16 performs a calculation step (S107) of calculating the concentration level of the user 1 using the determination results of whether the user 1 is in a concentration state, which are determined for the predetermined number of images. This enables the learning support device 100 to quantify the degree to which the user 1 was concentrated within a preset time range. The first concentration level estimation unit 16 outputs information regarding the calculated concentration level of the user 1 to the output unit 106. (not shown) This allows the user 1 or an administrator or the like who manages the user 1 to check the concentration level measured by the learning support device 100.

[0050] The determination of whether the user 1 is in a state of concentration in the first concentration level estimation unit 16 will be described in more detail with reference to Fig. 6. Fig. 6 is a diagram showing an example of a subject's habit used by the learning support device 100 according to the embodiment to determine the first concentration level. Fig. 6(a) is a diagram showing an image of the user 1 captured when in a state of concentration. Fig. 6(b) is a diagram showing an image of the user 1 captured when not in a state of concentration. Note that the habit of the user 1 when concentrating here is the action of touching the chin (i.e., part of the face) with the hand.

[0051] As shown in Figure 6(a), body part recognition To the department Therefore, the user 1 in the image is a part of the body. Jaw and The other part is Hands and The position of the chin is recognized as a coordinate on the image. Furthermore, based on the user 1's habit of touching his / her chin when concentrating, a concentration habit is determined. The part Then, it is determined whether the shortest distance between the chin and the hand is 0 or within a distance that can be considered equivalent to 0. In FIG. 6(a), the shortest distance between the chin and the hand on the image coordinates of user 1 is 0. Therefore, user 1 in FIG. 6(a) is determined to be in a state of concentration that indicates a habit of concentration.

[0052] On the other hand, in Figure 6 ( d 6, the body movement / pose determination unit 12 recognizes the position of only the chin, which is one part of the body of the user 1, as the coordinates on the image. Because the hand is not recognized in the image, the distance between the chin and the hand is not calculated, and the action of touching the chin, which is a habit of the user 1 when concentrating, is not shown. Therefore, as shown in ( d ) is determined to be not in a state of concentration that indicates a habit of concentration.

[0053] [Second concentration estimate] Next, the estimation of the second concentration level performed by the learning support device 100 will be described. Here, the estimation of the second concentration level performed in step S101 shown in FIG. 2 will be described. The second concentration level is represented as a ratio of the time during which the user 1 was concentrating to the time during which the task was performed. The time during which the user 1 was concentrating is calculated by multiplying the expected value of the response time by the total number of responses. The response time is expressed by a mixed normal distribution. Specifically, it is represented by the following formulas (1) to (5).

[0054] TIFF0007607272000001.tif94162

[0055] f(t) represents the distribution of response times. l and f h is the log-normal distribution used for mixed normal distributions. l is μ l and σ l Also, f h is μ h and σ h The parameter p is the mixing coefficient. CT is the concentration time (the time that user 1 was concentrated), N is the total number of responses, CTR is the concentration time ratio (the ratio of the time that user 1 was concentrated to the time that the task was performed), and T total is the total task execution time (total amount of time the target task was executed). Note that the second concentration level is defined here for the entire time the task was executed, but it may be defined for a shorter time period (time slot). In that case, the second concentration level will also be a value that indicates temporal fluctuation, like the first concentration level.

[0056] [Determining User 1's status] Next, the determination of the state of the user 1 by the concentration level determination unit 18 will be described. Here, the process performed in step S102 shown in Fig. 2 will be described. Fig. 7 is a diagram showing time slots for comparing the first concentration level and the second concentration level performed by the concentration level determination unit 18 according to the embodiment.

[0057] The concentration level determination unit 18 determines the state of the user 1 by comparing the first concentration level and the second concentration level of the user 1. The concentration level determination unit 18 does not determine the state of the user 1 continuously, but intermittently. Specifically, the concentration level determination unit 18 calculates an average value of the first concentration level or the second concentration level of the user 1 estimated from data acquired during each period during which the learning support device 100 performs one active task or passive task, and uses the average value to determine the state of the user 1 during that period.

[0058] Alternatively, the concentration level determination unit 18 may estimate a first or second concentration level of the user 1 from data acquired during a time period when the learning support device 100 is performing one active task or passive task, determine the concentration level of the user 1 from the estimated first or second concentration level, and set an average value of the multiple concentration levels of the user 1 determined during the time period as a representative value of the concentration level of the user 1 during the time period. For example, the concentration level determination unit 18 may determine the state of the user 1 by comparing the magnitude of the first concentration level with that of the second concentration level.

[0059] In addition, the concentration level determination unit 18 may use a median instead of an average value when performing the above process. The time during which the learning support device 100 performs one active task or passive task is, specifically, for example, 30 minutes.

[0060] Furthermore, in order to make it possible to compare the first concentration degree with the second concentration degree, the data indicating the first concentration degree and the data indicating the second concentration degree are normalized. Any method of normalization may be used.

[0061] For example, as shown in FIG. 7, the learning support device 100 first performs an active task. The performing time is, for example, 30 minutes. Then, the concentration level determination unit 18 determines the state of the user 1 during the performing time from the representative value of the first concentration level and the representative value of the second concentration level estimated during the performing time. Next, the learning support device 100 performs a passive task. The performing time is, for example, 30 minutes. Then, the concentration level determination unit 18 determines the concentration level of the user 1 during the performing time from the representative value of the first concentration level estimated during the performing time. Next, the learning support device 100 performs an active task. The performing time is, for example, 30 minutes. Then, the concentration level determination unit 18 determines the state of the user 1 during the performing time from the representative value of the first concentration level and the representative value of the second concentration level estimated during the performing time. In this way, the concentration level determination unit 18 does not continuously and sequentially determine the concentration level of the user 1, but determines the state of the user 1 in a predetermined period from the average value of the first concentration level or the second concentration level in the predetermined period. Note that the predetermined period may be the entire implementation time of 30 minutes, or may be a short time period (time slot) of 1 minute, 3 minutes, etc.

[0062] Next, a specific description will be given of the measurement of the concentration level performed by the learning support device 100. Fig. 8 is a diagram showing an overview of the measurement of the first concentration level in the learning support device 100 according to the embodiment.

[0063] First, consider a case where user 1 is watching a video (passive task). The time for which user 1 watches the video is, for example, 30 minutes. While user 1 is watching the video, learning support device 100 acquires image data of user 1 from imaging means 10. The image acquired by imaging means 10 is analyzed by body movement / pose determination unit 12 and gaze / facial expression determination unit 14.

[0064] For example, in the example shown in Fig. 10, analysis by the body movement / pose determination unit 12 and the gaze / facial expression determination unit 14 confirms that immediately after the user 1 starts watching a video, the user 1 is in a pose of taking notes with a serious expression. Next, the user 1 is confirmed to be in a pose of yawning. Then, the user 1 is confirmed to have a cheerful expression, and finally, the user 1 is confirmed to have a tired expression with his chin resting on his hand.

[0065] 8 shows a graph of the concentration level of user 1 estimated by first concentration level estimation unit 16 based on these poses and facial expressions. For example, in a time period in which user 1 is confirmed to be taking notes with a serious expression, the first concentration level is estimated to be relatively high, and in a time period in which user 1 is confirmed to be yawning, the first concentration level is estimated to be relatively low. In addition, in a time period in which user 1 is confirmed to have a cheerful expression, the first concentration level is estimated to be higher than the immediately preceding time period, and in the last time period in which user 1 is confirmed to have a tired expression with his chin resting on his hand, the first concentration level is estimated to be relatively low.

[0066] The concentration level determination unit 18 determines the state of the user 1 based on a representative value, such as the average value of the first concentration level for 30 minutes, as described above.

[0067] FIG. 9 is a diagram showing an outline of the measurement of the second concentration level in the learning support device 100 according to the embodiment. Next, consider a case where the user 1 is answering a question (active task). The time for the user 1 to answer the question is, for example, 30 minutes. While the user 1 is answering the question, the learning support device 100 acquires image data of the user 1 from the imaging means 10. The image acquired by the imaging means 10 is analyzed by the body movement / pose determination unit 12 and the gaze / facial expression determination unit 14. In addition, the information processing unit 24 acquires, as work information, the touch rate on the touch panel display when the user 1 answers the question, the correct answer rate of the question, the response time until the answer input, the progress speed of the question, the amount of processing of the question, the answer score, and the like. Then, the second concentration level estimation unit 26 estimates the second concentration level of the user 1 based on the work information.

[0068] For example, the second concentration level estimation unit 26 estimates that the second concentration level is high when the correct answer rate for the question is high. The second concentration level estimation unit 26 may also estimate that the second concentration level is high when the response time until the answer input is short. As shown in Fig. 9, the second concentration level is sequentially estimated during the 30 minutes during which the learning support device 100 presents the active task.

[0069] The concentration level determination unit 18 determines the state of the user 1 based on a representative value, such as the average value of the first concentration level for 30 minutes, as described above.

[0070] [Switch between active and passive tasks] Next, the switching between active tasks and passive tasks performed by the learning support device 100 according to the state of the user 1 will be described. Here, the process performed in step S103 shown in FIG. 2 will be specifically described. FIG. 10 is a diagram showing the switching between active tasks and passive tasks in the learning support device 100 according to the embodiment. As shown in FIG. 10, the learning support device 100 switches between a passive task of watching a lesson video and an active task of performing a quiz or exercise related to the lesson video according to the state of the user 1. Note that the learning support device 100 may switch from one passive task to a passive task with a different level of difficulty from the passive task, or may switch from one active task to an active task with a different level of difficulty from the active task, according to the state of the user 1.

[0071] FIG. 11 is a table showing details of switching between active tasks and passive tasks in the learning support device 100 according to the embodiment. When the learning support device 100 presents an active task, if the second concentration level is higher than the first concentration level, the difficulty level of the active task is determined to be too low for the user 1. This is because, although the user 1's task performance, such as the answer rate to the questions, is high, this is not reflected in the user's facial expression. It is considered that the user 1 has fully achieved learning and understanding of this task. Therefore, the presentation switching unit 30 switches to a passive task with a higher level of difficulty. This means that the learning support device 100 introduces the user 1 to a lesson video of the next learning stage that is one step more advanced.

[0072] Also, when the learning support device 100 presents an active task, if the first concentration level is higher than the second concentration level, the difficulty level of the active task is determined to be too high for the user 1. This is because the actual work performance, such as the answer rate to questions, is low, even though the user 1 appears to be sufficiently concentrated from the appearance of his / her facial expression. Alternatively, the learning support device 100 determines that the user 1 is in a so-called mind wandering state. Then, the presentation switching unit 30 switches to a passive task with a low level of difficulty. This means, for example, returning the assignment to the previous lesson video and having the user 1 review it again. In this case, the presentation switching unit 30 may also switch to a break. When the learning support device 100 presents an active task, if the first concentration level is higher than the second concentration level, the presentation switching unit 30 determines whether to switch to a passive task with a low level of difficulty or to a break based on whether the first concentration level or the second concentration level is higher than a third value.

[0073] Furthermore, when the learning support device 100 presents an active task, the presentation switching unit 30 may switch to a passive task of different difficulty level depending on the level of the second concentration level. For example, when the learning support device 100 presents an active task, if the second concentration level is higher than a first predetermined value, the presentation switching unit 30 may switch to a passive task of higher difficulty level. Conversely, for example, when the learning support device 100 presents an active task, if the second concentration level is lower than a second predetermined value, the presentation switching unit 30 may switch to a passive task of lower difficulty level.

[0074] Furthermore, when the learning support device 100 presents a passive task and the first concentration level is higher than the first value, the presentation switching unit 30 determines that the user 1 is sufficiently concentrating on watching a lesson video or the like, and switches to an active task of higher difficulty as the next stage. 2 When the value is lower than this, the presentation switching unit 30 determines that the user 1 is not concentrating on the lesson video, and may switch to a break or to an active task that prompts the user 1 to answer questions with a relatively low level of difficulty. Here, the second value is smaller than the first value. When the learning support device 100 presents a passive task, if the first concentration level is lower than a first value, the presentation switching unit 30 determines whether to switch to a rest or to an active task with a lower level of difficulty based on whether the first concentration level is higher than a third value.

[0075] Next, an overview of the process of switching between an active task and a passive task performed by the learning support device 100 will be described. FIG. 12 is a flowchart showing an example of the process of the learning support device 100 according to the embodiment. Using FIG. 12, the process of switching between an active task and a passive task based on the concentration level of the user 1 when the learning support device 100 presents a passive task to the user 1 will be described. The process shown in FIG. 12 is a specific example of the process shown in FIG. 2.

[0076] First, the second learning task presenter 28 presents a video to the user 1 (Step S300).

[0077] Next, the first concentration level estimating unit 16 estimates a first concentration level of the user 1 (Step S301).

[0078] Next, the concentration degree determination unit 18 determines whether the first concentration degree is higher than a first value (step S302).

[0079] When the concentration level determination unit 18 determines that the first concentration level is higher than the first value (Yes in step S302), the presentation switching unit 30 switches to question presentation (step S303). Specifically, the presentation switching unit 30 causes the second learning task presentation unit 28 to stop outputting the video, and causes the first learning task presentation unit 22 to output a question. Here, the presentation switching unit 30 causes the first learning task presentation unit 22 to output a more difficult question.

[0080] When the concentration level determination unit 18 determines that the first concentration level is lower than the first value (No in step S302), the presentation switching unit 30 switches to break (step S304). Specifically, the presentation switching unit 30 causes the second learning task presentation unit 28 to stop outputting the video, and causes the first learning task presentation unit 22 to output content that prompts the user 1 to take a break. In addition, when the concentration level determination unit 18 determines that the first concentration level is lower than the first value and higher than the second value, the presentation switching unit 30 causes the first learning task presentation unit 22 to output a low-difficulty question instead of the content that prompts the user 1 to take a break.

[0081] Fig. 13 is a flowchart showing another example of the process of the learning support device 100 according to the embodiment. A process of switching between an active task and a passive task based on the concentration level of the user 1 when the learning support device 100 presents an active task to the user 1 will be described with reference to Fig. 13. The process shown in Fig. 13 is a specific example of the process shown in Fig. 2.

[0082] First, the first learning task presenter 22 presents a problem to the user 1 (step S 5 00).

[0083] Next, the first concentration level estimation unit 16 estimates a first concentration level of the user 1 (step S 5 01).

[0084] Next, the second concentration level estimation unit 26 estimates a second concentration level of the user 1 (step S 5 02). Note that step S 5 01 and Step S 5 02 may be in the reverse order.

[0085] Then, the concentration level determination unit 18 determines whether the second concentration level is higher than the first concentration level (step S 5 03).

[0086] When the concentration level determination unit 18 determines that the second concentration level is higher than the first concentration level (step S 5 If the answer is YES in step S03, the presentation switching unit 30 switches to presenting a video with a high level of difficulty (step S 5 04). Specifically, the presentation switching unit 30 causes the first learning task presentation unit 22 to stop outputting questions, and causes the second learning task presentation unit 28 to output a video with a higher level of difficulty.

[0087] When the concentration level determination unit 18 determines that the second concentration level is lower than the first concentration level (step S 5 If the answer is No in step S03, the presentation switching unit 30 switches to presenting a video with a low level of difficulty (step S 5 Specifically, the presentation switching unit 30 causes the first learning task presentation unit 22 to stop outputting the questions, and causes the second learning task presentation unit 28 to output a video with a low level of difficulty. In addition, when the concentration level determination unit 18 determines that the second concentration level is lower than the first concentration level (step S 5If the answer is No in 03, the presentation switching unit 30 may switch to a break. Depending on whether the first concentration level or the second concentration level is higher than the third value, the second learning task presenting unit 28 may switch between outputting a video with a low level of difficulty or outputting content that encourages the user 1 to take a break.

[0088] [Specific examples of determining concentration and switching tasks] Next, a specific description will be given of the determination of the concentration level of the user 1 by the learning support device 100 and the switching of tasks presented to the user 1 by the learning support device 100. Fig. 14 is a diagram showing an example of the state determination of the user 1 by comparing the first concentration level with the second concentration level in the learning support device 100 according to the embodiment. Fig. 15 is a diagram showing the guidance of the state of the user 1 by comparing the first concentration level with the second concentration level in the learning support device 100 according to the embodiment.

[0089] FIG. 14 shows a graph plotting the concentration level of the user 1 when an active task is presented to the user 1 by the learning support device 100, with the first concentration level on the vertical axis and the second concentration level on the horizontal axis. Specifically, task A is plotted at a point where the first concentration level is 0.567 and the second concentration level is 0.477. Task B is plotted at a point where the first concentration level is 0.748 and the second concentration level is 0.384. The concentration level determination unit 18 interprets that the first concentration level and the second concentration level are almost equal in task A, and that the working attitude and performance of the user 1 are balanced. In the state of task A, the presentation switching unit 30 switches to a passive task with a low level of difficulty.

[0090] Furthermore, the concentration level determination unit 18 interprets that in task B, the first concentration level is greater than the second concentration level, and that the user 1's working attitude is good but performance is declining. That is, the concentration level determination unit 18 determines that the user 1 is in a absent-minded state in task B. In the state of task B, the presentation switching unit 30 switches to a break. At this time, the learning support device 100 presents to the user 1 an image or music that has a relaxing effect on the user 1.

[0091] Alternatively, in the state of task B, the presentation switching unit 30 may switch to an active task having a lower difficulty level than the currently presented active task. Alternatively, in the state of task B, the presentation switching unit 30 may switch to a passive task. The passive task presented at this time is, for example, a video for reviewing the active task that was performed immediately before.

[0092] By switching tasks as described above, the learning support device 100 can guide the user 1 to a state in which the first and second concentration levels of the user 1 are balanced by lowering the first concentration level of the user 1 when task B is performed or by improving the second concentration level of the user 1 when task B is performed, as shown in Fig. 15. Thus, by switching tasks as described above, the learning support device 100 can guide the user 1 to a more concentrated state.

[0093] [Effects, etc.] The learning support device 100 of this embodiment is a learning support device for user 1 to perform a learning task, and includes a first concentration level estimation unit 16 that analyzes information from an imaging means 10 that images user 1 and estimates a first concentration level of user 1, a second concentration level estimation unit 26 that analyzes information actively input by user 1 when user 1 performs the learning task and estimates a second concentration level of user 1, and a switching unit 30 that switches the content of the learning task and the presentation method of the learning task based on at least one of the first concentration level and the second concentration level.

[0094] As a result, the learning support device 100 can present an appropriate learning task between a first learning task for which the user 1 actively studies and a second learning task for which the user 1 passively studies, depending on the state of the user 1 estimated from the user 1's concentration level.

[0095] In addition, for example, the learning support device 100 further includes a first learning task presentation unit 22 that presents to the user 1 a first learning task that the user 1 is to actively learn, and while the first task presentation unit 22 is presenting the first learning task to the user 1, the switching unit 30 switches the content presented to the user 1 to a first learning task of a different difficulty level depending on the magnitude relationship between the first concentration level and the second concentration level.

[0096] As a result, when presenting a first learning task to user 1, learning support device 100 can switch the presentation to a second learning task of appropriate difficulty depending on the state of user 1 estimated from the concentration level of user 1.

[0097] Also, for example, in the learning support device 100, while the first learning task presentation unit 22 presents the first learning task to the user 1, the switching unit 30 switches the content presented to the user 1 to a second learning task that the user 1 passively learns and that has a different level of difficulty depending on the level of the second concentration level.

[0098] As a result, when presenting a first learning task to user 1, learning support device 100 can switch the presentation to a second learning task of appropriate difficulty depending on the state of user 1 estimated from the concentration level of user 1.

[0099] Also, for example, the learning support device 100 further includes a second learning task presentation unit 28 that presents a second learning task to the user 1, and while the second learning task presentation unit 28 is presenting the second learning task to the user 1, when the first concentration level is higher than the first value, the switching unit 30 switches the content presented to the user 1 to the first learning task.

[0100] As a result, when presenting the second learning task to the user 1, the learning support device 100 can switch the presentation to a first learning task of an appropriate level of difficulty depending on the state of the user 1 estimated from the concentration level of the user 1.

[0101] In addition, for example, the learning support device 100 further includes a concentration level determination unit 18 that determines that the user 1 is in a absent-minded state and prompts the user 1 to take a break when the first concentration level is higher than the second concentration level while the first learning task presentation unit 22 is presenting the first learning task to the user 1.

[0102] As a result, when the learning support device 100 presents the first learning task to the user 1, it is possible to prompt the user 1 to take a break depending on the state of the user 1 estimated from the degree of concentration of the user 1, thereby improving the work efficiency of the user 1.

[0103] Also, for example, in the learning support device 100, while the second learning task presentation unit 28 is presenting the second learning task to the user 1, when the first concentration level is lower than the second value, the concentration level determination unit 18 prompts the user 1 to take a break.

[0104] As a result, when the learning support device 100 presents the second learning task to the user 1, it is possible to prompt the user 1 to take a break depending on the state of the user 1 estimated from the degree of concentration of the user 1, thereby improving the work efficiency of the user 1.

[0105] Furthermore, the learning support system of the present disclosure is a learning support system for user 1 to perform a learning task, and includes a display 2, an imaging means 10 for imaging user 1, a first concentration level estimation unit 16 for estimating a first concentration level of user 1 by analyzing information from the imaging means 10 for imaging user 1, a second concentration level estimation unit 26 for estimating a second concentration level of user 1 by analyzing information actively input by user 1 when user 1 performs the learning task, and a learning task presentation method switching unit that switches the content of the learning task and the presentation method of the learning task based on at least one of the first concentration level and the second concentration level. presentation A switching unit 30 is provided.

[0106] As a result, the learning assistance system of the present disclosure can achieve the same effects as the learning assistance device 100 described above.

[0107] [others] Although the embodiments have been described above, the present disclosure is not limited to the above-described embodiments.

[0108] For example, in the above embodiment, the process executed by a specific processing unit may be executed by another processing unit. Also, the order of multiple processes may be changed, or multiple processes may be executed in parallel.

[0109] Also, for example, in the above embodiment, a learning support method for user 1 performing a learning task may be executed, which includes a first concentration level estimation step of estimating a first concentration level of user 1 by analyzing information from an imaging means 10 that images user 1, a second concentration level estimation step of estimating a second concentration level of user 1 by analyzing information actively input by user 1 when user 1 performs the learning task, and a switching step of switching the content of the learning task and the presentation method of the learning task based on at least one of the first concentration level and the second concentration level.

[0110] In the above embodiment, each component may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

[0111] Furthermore, each component may be realized by hardware. For example, each component may be a circuit (or an integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit, or a dedicated circuit.

[0112] In addition, the general or specific aspects of the present disclosure may be realized in a system, an apparatus, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM. In addition, the general or specific aspects of the present disclosure may be realized in any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0113] For example, the present disclosure may be realized as a program for causing a computer to execute the learning support method of the above-described embodiment. The present disclosure may be realized as a computer-readable non-transitory recording medium having such a program recorded thereon.

[0114] In addition, the present disclosure also includes forms obtained by applying various modifications to each embodiment that a person skilled in the art may conceive, or forms realized by arbitrarily combining the components and functions of each embodiment within the scope that does not deviate from the spirit of the present disclosure. [Industrial Applicability]

[0115] The learning assistance device and learning assistance system of the present disclosure can provide a user with an effective learning experience. [Explanation of symbols]

[0116] 1 User 2. Display 10. Shooting Method 12 Body movement and pose determination section 14 Gaze / expression determination unit 16 1st concentration estimation part 18 Concentration level determination section 20 Answer input section 22 First learning task presentation section 24 Information Processing Section 26 Second concentration estimation part 28 Second learning task presentation section 30 Switching section 100 Learning support device

Claims

1. A learning assistance device for a user to perform a learning task, comprising: a first concentration level estimation unit that estimates a first concentration level of the user by analyzing information from an imaging means that images the user; a second concentration level estimation unit that estimates a second concentration level of the user by analyzing information actively input by the user when the user performs the learning task; a presentation switching unit that switches a content of a learning task and a presentation method of the learning task based on at least one of the first concentration level and the second concentration level; a first learning task presentation unit that presents to the user a first learning task that the user actively learns, While the first learning task presentation unit is presenting the first learning task to the user, the presentation switching unit switches the content to be presented to the user to the first learning task having a different level of difficulty depending on a magnitude relationship between the first concentration level and the second concentration level; Learning aids.

2. While the first learning task presentation unit is presenting a first learning task to the user, the presentation switching unit switches the content presented to the user to a second learning task of a different difficulty level depending on the second concentration level, the second learning task being passively learned by the user; The learning support device according to claim 1 .

3. Further, a second learning task presenting unit presents a second learning task to the user, When the first concentration level is higher than a first value while the second learning task presenter is presenting the second learning task to the user, the presentation switching unit switches the content presented to the user to the first learning task. The learning support device according to claim 1 or 2.

4. a concentration level determination unit that determines that the user is in a absent-minded state and prompts the user to take a break when the first concentration level is higher than the second concentration level while the first learning task presentation unit is presenting the first learning task to the user, The learning support device according to any one of claims 1 to 3.

5. a second learning task presenting unit presenting a second learning task to the user, and when the first concentration level is lower than a second value, The concentration level determination unit prompts the user to take a break. The learning support device according to claim 4.

6. A learning support system for a user to perform a learning task, comprising: A display and An imaging means for imaging a user; a first concentration level estimation unit that analyzes information from the image capturing means to estimate a first concentration level of the user; a second concentration level estimation unit that estimates a second concentration level of the user by analyzing information actively input by the user when the user performs the learning task; a presentation switching unit that switches a content of a learning task and a presentation method of the learning task based on at least one of the first concentration level and the second concentration level; a first learning task presentation unit that presents to the user a first learning task that the user actively learns, While the first learning task presentation unit is presenting the first learning task to the user, the presentation switching unit switches the content to be presented to the user to the first learning task having a different level of difficulty depending on a magnitude relationship between the first concentration level and the second concentration level; Learning support system.

Images