Training systems, training-testing integration systems, and programs

Abstract

We provide a training system and a training / testing integration system that enable appropriate and efficient training for each user. [Solution] In the training system 10, the timer adjustment means 24 determines whether predetermined time limit reduction conditions are met using the results of the training response result determination means 23, such as the percentage of tasks that were handled correctly or that were completed, or the number of consecutive tasks. If these time limit reduction conditions are met, the time limit reduction process is executed. At the same time, the timer adjustment means 24 determines whether predetermined time limit extension conditions are met using the results of the training response result determination means 23, such as the percentage of tasks that were handled incorrectly or that were not completed, or the number of consecutive tasks. If these time limit extension conditions are met, the time limit extension process is executed.

Description

Technical Field

[0001] The present invention relates to a training system, a training-test linkage system, and a program configured by a computer for performing training to improve a user's coping ability with respect to a given problem. For example, it can be used in a language learning system, a game system, a life skill acquisition system such as cooking, sewing, ikebana, calligraphy, a vocational skill acquisition system, a pet breeding method learning system, a musical instrument performance acquisition system, and the like.

Background Art

[0002] Generally, there are various trainings for improving the coping ability with respect to a given problem. For example, language learning training for improving language proficiency in languages other than the mother tongue (English, Chinese, French, etc.), gaming training for improving the ability to overcome difficult scenes or situations that appear in sequence and clear steps to progress in a game, a shogi game, a go game, a mahjong game, a battle game where characters with weapons fight, etc., training for improving life skills for acquiring knowledge and skills necessary in real life such as cooking, sewing, ikebana, calligraphy, vocational training for acquiring vocational skills such as surgical methods, methods of operating vehicles such as airplanes and ships, methods of operating heavy machinery such as cranes and forklifts, pet breeding training for learning the breeding methods of pets such as pet cats and pet dogs, musical instrument performance training for musical instruments such as pianos and guitars, and the like. And among these trainings, there are also those in which a user performs training using a training system configured by a computer, such as a language learning system.

[0003] Furthermore, the terms "challenge," "solution," and "solution ability" in this application are comprehensive terms that can be used for the various types of training described above. For example, in a language learning system, they mean "problem," "answer," and "language ability." In a game system, they mean "goals or objectives set for the game to be achieved, or what needs to be done to achieve them (usually not easy to achieve)," "input operations performed to achieve goals or objectives," and "skills to overcome difficulties or skills to defeat enemies."

[0004] By the way, when individuals deal with a given task, the focus can be either solely on whether they succeeded or failed in dealing with it—that is, whether their approach to the task was correct or not, or whether they achieved the task—or it can also be on the time it took to deal with it.

[0005] In the former case, the time spent addressing the issue is not important; the task is one that can be achieved even if it takes a long time, such as six months, a year, or five years, or one where the emphasis is on the success of the addressing the issue (taking the correct action, achieving the objective) rather than the time taken. On the other hand, in the latter case, the task is one that is preferable to be addressed in a shorter amount of time, given that the same action needs to be taken.

[0006] Among the various training methods mentioned above, many focus on the time required to address a problem; that is, they aim to improve the ability to handle tasks where it is preferable to complete them in a shorter amount of time, given the same content. For example, in language learning training to improve language proficiency in a language other than one's native tongue, speed is an essential element of language ability, as it is desirable to be able to respond quickly to what the other person says in order to achieve smooth conversation. Therefore, some language learning systems set a time limit between the presentation of a question and the submission of the answer.

[0007] Furthermore, a rapid response training system is known that can efficiently conduct rapid response training for multiple-choice questions using information technology (IT) (see Patent Document 1). In this rapid response training system, as described in paragraph

[0028] of Patent Document 1, in the "drill practice," the time limit for answering each multiple-choice question is divided into three stages and one of them can be selected. For example, by practicing the drill with "Time Trial (1) - 25 seconds," then proceeding to "Time Trial (2) - 20 seconds (= mock test)," and finally to "Time Trial (3) - 15 seconds," the time limit for answering is gradually shortened, allowing for efficient rapid response training. [Prior art documents] [Patent Documents]

[0008] [Patent Document 1] Japanese Patent Publication No. 2007-334108 (paragraph

[0028] , abstract) [Overview of the project] [Problems that the invention aims to solve]

[0009] As mentioned earlier, some conventional language learning systems impose time limits on users when they answer language learning questions (or more generally, when they tackle assignments). To encourage users to answer within the time limit, a timer is sometimes displayed on the screen (the screen presenting the question) showing the remaining time. However, the set time limit is uniformly determined without considering individual circumstances such as each user's language proficiency, and is not set with each user's individual needs in mind. Consequently, this can lead to the problem that the training may not be appropriate, optimal, or efficient for each user.

[0010] In other words, if the set time limit is too long for the user undergoing training, that user will always have ample time to answer (or more generally, deal with) the questions. While this in itself is not necessarily a bad thing, from the perspective of improving language skills by increasing answer speed (or more generally, dealing speed), it cannot be said that the training is appropriate and efficient for that user.

[0011] On the other hand, if the set time limit is too short for the user undergoing training, they may frequently run out of time and be unable to answer, or they may rush and repeatedly give careless answers. In such cases, it cannot be said that the training is appropriate and efficient for that user.

[0012] Furthermore, the rapid response training system described in Patent Document 1, mentioned above, allows users to select from three different time limits, thus increasing the likelihood of efficient training for each user. However, if a user is unaware that they can answer faster, or if they are not actively motivated to improve their answering speed, the time limit will not be changed. This can lead to the disadvantage of not receiving appropriate and efficient training for each user.

[0013] In other words, if a user frequently finds that their response speed cannot keep up with the set time limit, resulting in them being unable to answer due to the time expiring, they can clearly recognize that the current time limit is not suited to their current language ability (more generally, their problem-solving ability). This allows them to realize that they are not receiving appropriate and efficient training. However, if a user is able to answer with ample time to spare, they may not notice that they are not receiving appropriate and efficient training because no problems arise.

[0014] Furthermore, the aforementioned inconveniences occur not only when training involves answering given problems within a time limit (a typical example being training using a language learning system), but also more broadly when training involves dealing with given tasks within a time limit. For example, in a game system, although the user does not perform actions to answer problems, if the game involves controlling a character to move, overcoming (clearing) a series of difficult scenes and situations to advance to the next step, and competing to reach a target stage as quickly as possible within a time limit, then controlling the character's movements to overcome those difficulties is equivalent to user actions to deal with a given task within a time limit, thus resulting in similar inconveniences. Moreover, these inconveniences are not limited to language learning systems and game systems; if a time limit is set for dealing with a task, similar inconveniences occur in systems for acquiring life skills such as cooking, sewing, flower arranging, and calligraphy, as well as systems for acquiring occupational skills, pet care methods, and musical instrument playing.

[0015] The objective of the present invention is to provide a training system, a training-testing system, and a program that enable appropriate and efficient training for each user. [Means for solving the problem]

[0016] <Invention of a training system>

[0017] The present invention relates to a training system comprising a computer that provides training to improve a user's ability to deal with a given task, It includes a training management server that is connected to the user terminal operated by the user via a network and performs the process of managing the training. The training management server is, A training task presentation means that presents a time-limited task to the user on the user terminal screen, and displays a timer on the user terminal screen showing the remaining time or elapsed time until the time limit given to the user to complete the task, while the user is working on the task. A training response result determination means that accepts user input operations on a user terminal as training to address a task, and makes a judgment including evaluation or aggregation to obtain the response result, including whether the response to the task was correct or whether the task was achieved or not, This system is characterized by including a timer adjustment means that uses the results obtained from the judgment process by the training response result judgment means to determine whether the response was correct or whether the percentage or number of consecutive tasks achieved, or other predetermined conditions for shortening the time limit that the response results must satisfy, meets a predetermined condition for shortening the time limit, and if this condition for shortening the time limit is met, it executes a time limit shortening process to shorten the time limit used for the timer display. It also determines whether the percentage or number of consecutive tasks that were not correctly handled or were not achieved, or other predetermined conditions for extending the time limit that the response results must satisfy, meets a predetermined condition for extending the time limit, and if this condition for extending the time limit is met, it executes a time limit extension process to extend the time limit used for the timer display.

[0018] Here, "other results of actions" in the "conditions predetermined for shortening the time limit, such as the percentage or number of consecutive tasks for which the action was taken correctly or achieved, or other results of actions that must be met" refers to results of actions other than the "percentage" of the number of tasks for which the action was taken correctly or achieved (hereinafter sometimes collectively referred to as "successfully taken tasks") relative to the total number of tasks, or the "number of consecutive tasks" for which the action was taken successfully. For example, this could be the "remaining time" until the time limit at the time of action, or a combination of multiple results such as a combination of "percentage" or "number of consecutive tasks" and "remaining time." Details of the "conditions for shortening the time limit" will be described later in [Modes for Carrying Out the Invention] (see Figure 13).

[0019] Furthermore, in the "conditions for extending the time limit that are predetermined as conditions that the proportion or number of consecutive tasks that were not handled correctly or that could not be achieved, or other handling results must satisfy," "other handling results" refer to handling results other than the "proportion" of the number of tasks that were not handled correctly or that could not be achieved (hereinafter sometimes collectively referred to as "tasks that were not handled") to the total number of tasks, or the "number of consecutive times" of tasks that were not handled. For example, this includes the proportion of the number of tasks that could not be handled due to time running out to the total number of tasks, the number of consecutive times tasks that could not be handled due to time running out, and handling results that combine multiple handling results. Details of the "conditions for extending the time limit" will be described later in [Modes for Carrying Out the Invention] (see Figure 14).

[0020] Furthermore, the "time limit reduction process" may involve, for example, reducing the time limit by a fixed amount of time, such as 1 second, each time the time limit reduction condition is met, or reducing the time limit by a coefficient obtained by multiplying the average remaining time until the time limit was reached for a task that was successfully addressed. The "time limit extension process" may involve extending the time limit by a fixed amount of time, such as 1 second, each time the time limit extension condition is met. Details on how to determine the time range for reduction and extension in these "time limit reduction processes" and "time limit extension processes" will be described later in [Modes for Carrying Out the Invention] (see Figures 13 and 14).

[0021] In the training system of the present invention, since a timer adjustment means is provided, it is possible to determine whether the response result obtained by the training response result determination means satisfies a predetermined time limit reduction condition, and if this time limit reduction condition is met, a time limit reduction process is executed. In addition, it is possible to determine whether the response result satisfies a predetermined time limit extension condition, and if this time limit extension condition is met, a time limit extension process is executed.

[0022] Therefore, training can be conducted in a state suitable for improving the coping ability of each user, enabling appropriate and efficient training for each user. That is, when it is determined that the situation is one where the user is dealing with tasks with a margin, the time limit can be automatically shortened, so that the user can always approach a state of dealing with tasks while being tense, or in other words, while fully exerting the coping ability they currently possess. On the other hand, when it is determined that the user's coping ability for the task is not keeping up with the presentation of the task, the time limit can be automatically extended, so that the user can approach a state where they can fully exert their coping ability.

[0023] Then, by the timer adjustment means, using the coping result obtained in the determination process by the training coping result determination means, it is determined whether or not the time limit shortening condition predetermined as a condition that the coping result should satisfy is met, and when this time limit shortening condition is met, the time limit shortening process is executed. Thus, like the quick response training system described in Patent Document 1 mentioned above, when the user does not notice on their own that they can cope by increasing the coping speed more (when they do not notice the height of their current coping ability), or when they do not actively think that they will be able to cope by increasing the coping speed more, it is possible to avoid the inconveniences that occur, and the above object is achieved thereby.

[0024] <Using the coping result obtained in the determination process by the training coping result determination means and the identification result by the facial expression identification means, the timer adjustment means executes a composite time limit adjustment process>

[0025] Also, in the training system described above, The user terminal is provided with photographing means for photographing an image of the user's face, The training management server has face image acquisition means for acquiring, via a network, the face image data of the user dealing with the task photographed by the photographing means of the user terminal, This system includes a face image acquisition means that inputs user face image data acquired by this face image acquisition means, and uses a face expression recognition model that has been pre-trained and stored in a face expression recognition model storage means to identify whether the user's facial expression while dealing with a task is a tense expression indicating concentration and tension while dealing with the task, or a relaxed expression indicating a relaxed attitude while dealing with the task, and performs a pattern recognition process that outputs this identification result. The timer adjustment means is, Using the response result obtained by the training response result determination means and the identification result by the facial expression recognition means, a process is executed to adjust the time limit used for the timer display. In this case, if the judgment process by the training response result determination means determines that the response result obtained satisfies the predetermined time limit reduction condition, and the facial expression recognition means outputs an identification result of a relaxed expression, then either a relaxed expression-incorporating process is executed to shorten the time limit used for the timer display, or Alternatively, it is desirable that the system be configured to perform a process that incorporates relaxed facial expressions, and if the response result obtained by the training response result judgment means determines that the predetermined time limit extension conditions must be met, and the facial expression recognition means outputs an identification result that indicates a tense expression, then perform a process that incorporates tense facial expressions to extend the time limit used for the timer display.

[0026] Here, "face image data of the user during the task" in the "face image acquisition means" refers to face image data from the time the task is presented to the user (time P1 shown in Figure 11) to the time the user performs the final input operation to address the task (time P2 shown in Figure 11, which in this application is referred to as the "task time"). Therefore, "during the task" includes not only the task time P2, when the user performs the final input operation to send data indicating the content of the task to the training management server, but also the time the user is thinking up to the final operation. Note that the final operation refers to an operation that leads to the transmission of data to the training management server (for example, a mouse click operation to send answer data), and excludes operations that do not lead to the transmission of data to the training management server (for example, an operation that is not necessary for the answer, such as the user unconsciously moving the mouse pointer while thinking).

[0027] Furthermore, the term "timer adjustment means" here is described in such a way that it is sufficient to perform at least the "relaxed expression processing" out of the two "relaxed expression processing" options.

[0028] In this configuration, where the response results obtained by the training response result judgment means and the identification results by the facial expression recognition means are used to perform a complex time limit adjustment process by the timer adjustment means, it becomes possible to determine whether or not to shorten the time limit using the user's facial expression while they are working on the task. This makes it possible to capture user information that is closer to their true state and utilize it in the time limit adjustment process by the timer adjustment means. As a result, users can perform training more reliably and efficiently.

[0029] <A configuration that allows the user to select a self-configuration mode and set the time limit themselves.>

[0030] Furthermore, in a configuration in which the timer adjustment means performs a complex time limit adjustment process using the response result obtained by the training response result determination means described above, and the identification result by the facial expression recognition means, The means of presenting training tasks are: If the user selects the self-setting mode to set the time limit themselves, the system accepts input from the user terminal to set the time limit, and then uses the accepted time limit to display a timer on the user terminal screen. The timer adjustment means is, If the user has selected self-configuration mode, the system may be configured to display an advice message on the user terminal screen recommending that the user shorten the time limit used for the timer display if the response result obtained by the training response result determination means is determined to satisfy the predetermined conditions for shortening the time limit, and the facial expression recognition means outputs an identification result indicating a relaxed expression.

[0031] In this configuration, where users can select a self-configuration mode and set their own time limits, an advice message recommending a reduction in the time limit will be displayed when certain conditions are met. By referring to this advice and changing the time limit setting, users can perform training more reliably and efficiently.

[0032] <Configuration for awarding points>

[0033] Furthermore, in the aforementioned training system, The training management server is, The system includes a point storage means that stores the number of points held by a user, associated with user identification information, which can be used to pay service fees for training or other services provided to the user by the training management server operator or another company affiliated with the operator. The means for determining the outcome of training is, If a user sets a time limit shorter than the predetermined standard time limit, the system may be configured to determine whether the numerical value of the data indicating the user's performance, obtained by aggregating the results of the actions taken, is equal to or greater than a predetermined threshold. If it is equal to or greater than the threshold, the system may award the user a predetermined number of points, or an amount of points corresponding to the difference between the standard time limit and the time limit set by the user. The system may then add the number of points awarded to the number of points held by the user, which is stored in the point storage means in association with the user's identification information.

[0034] By implementing a point-based system in this way, users are informed that they can earn points for completing tasks within a short time limit and performing well. This makes users aware that they can earn points when their problem-solving abilities improve, increasing their motivation to improve their problem-solving skills and boosting their training motivation, thus accelerating the speed at which their problem-solving abilities improve.

[0035] <Invention of a training-testing integration system: A configuration that transmits integration data from the training management server to the test management server>

[0036] Furthermore, the present invention relates to a training-test linkage system composed of a computer that performs training to improve a user's ability to deal with a given task, and tests to measure a user's ability to deal with a task, through data linkage, The training system described above comprises a training management server and a test management server that is connected to a user terminal operated by the user via a network and executes a process to manage tests in the same format as the training management server, which presents and handles time-limited tasks. The training management server is, A timer setting information storage means that stores the time limit set for each user in association with user identification information, This system includes a data transmission means for sending data to a test management server, which transmits the time limit for a user stored in this timer setting information storage means, along with the user's user identification information, to the test management server via the network. The test management server is This process manages tests where the user can set their own time limit for testing to address the given issue. The system is characterized by including a timer setting means that either automatically sets the time limit at the start of a test to measure the user's ability to handle a task, using the user's time limit received from the data transmission means for the test management server along with user identification information, or displays the time limit received from the data transmission means for the test management server along with user identification information on the user terminal screen as reference information for the user to set the time limit at the start of the test, and accepts user terminal operations for the user to set the time limit themselves.

[0037] In the training-test collaboration system of the present invention, the timer setting means of the test management server receives the user's time limit from the collaboration data transmission means of the training management server to the test management server. Using the received time limit, it is possible to automatically set the time limit for the user at the start of the test, or to display the received time limit to the user taking the test as reference information for self-setting the time limit at the start of the test. Therefore, the test management server can set an appropriate time limit using the collaboration data (user's time limit) from the training management server.

[0038] It should be noted that the test is conducted to measure the user's ability to handle the tasks. If users were allowed to set their own time limits during the test, shortening the time limit would reduce the time they have to handle the tasks. This might make it seem as though the user is deliberately creating a disadvantageous situation for themselves, and that they are creating an unfair test environment rather than being tested under the same conditions as other users. However, the test conducted here is not of that nature. If the test is one in which users receive a higher score for handling many tasks within a certain time (for example, solving many problems within a certain time), then it is better to spend less time on each task. Therefore, users should always be in a state of tension while handling the tasks, and will set a shorter time limit to match their ability. The data linked from the training management server (the user's time limit) will be used for this setting. In other words, if the user's time limit, which has been adjusted to match their ability during training, can be received and used for self-setting at the start of the test, users can set an appropriate time limit for themselves, always be in a state of tension while handling the tasks, and receive a high score on the test. Furthermore, even if a test is not designed to reward higher scores for completing more tasks within a set time (for example, solving more problems within a set time), if users can set an appropriate time limit for themselves and take the test, they can finish the test earlier even if they complete the same number of tasks, allowing them to make better use of the time afterward.

[0039] <Invention of a training-test integration system: A configuration that transmits integration data from the test management server to the training management server>

[0040] Furthermore, the present invention relates to a training-testing linkage system comprising a computer that performs training to improve a user's ability to deal with a given task, and tests to measure a user's ability to deal with a task, through data linkage, The training system described above comprises a training management server and a test management server that is connected to a user terminal operated by the user via a network and executes a process to manage tests in the same format as the training management server, which presents and handles time-limited tasks. The test management server is A test response result storage means stores the results of how users who have taken the test have handled test tasks, including test summary result data showing aggregated results of how multiple tasks have been handled, in association with user identification information. This system includes a training management server data transmission means that transmits the test summary result data stored in this test result storage means, along with user identification information, to the training management server via the network. The timer adjustment method for the training management server is: This system is characterized by receiving test summary result data transmitted via the network using a data transmission means for the training management server, and using the received test summary result data to set the training start time limit shorter the better the test performance shown in the test summary result data.

[0041] In the training-test collaboration system of the present invention, the timer adjustment means of the training management server receives test aggregation result data from the collaboration data transmission means for the training management server of the test management server. Using the received test aggregation result data, it becomes possible to set a shorter training start time limit for better test performance, according to the test performance indicated in the test aggregation result data. As a result, users can perform training that is appropriate and efficient for them.

[0042]

[0043] Furthermore, in the training system described above, The problem is a language learning problem, the user's ability to deal with it is their language proficiency, and the means for determining the outcome of the training can be configured to perform a process that determines whether or not the answer to the problem was correct.

[0044] When the training system of the present invention is applied to a language learning system, the user (the person learning the language) can perform language learning training that is appropriate and efficient for them.

[0045] <Invention of a program>

[0046] Furthermore, the program of the present invention is intended to make a computer function as the training system described above.

[0047] Furthermore, the above program or any part thereof can be recorded and stored or distributed on recording media such as magneto-optical disks (MO), compact discs (CD), digital versatile disks (DVD), flexible disks (FD), magnetic tape, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM), flash memory, random access memory (RAM), hard disk drives (HDD), solid state drives (SSD), and flash disks. It can also be transmitted using transmission media such as wired networks like local area networks (LAN), metropolitan area networks (MAN), wide area networks (WAN), the Internet, intranets, extranets, wireless communication networks, or combinations thereof, and can also be transmitted on carrier waves. Moreover, the above program may be part of another program, or may be recorded on a recording media together with a separate program. [Effects of the Invention]

[0048] As described above, the present invention includes a timer adjustment means, which allows the system to use the response result obtained from the training response result determination means to determine whether the response result satisfies a predetermined time limit reduction condition, and execute a time limit reduction process if this condition is met. It also allows the system to determine whether the response result satisfies a predetermined time limit extension condition, and execute a time limit extension process if this condition is met. This has the effect of enabling appropriate and efficient training for each user. [Brief explanation of the drawing]

[0049] [Figure 1] An overall system configuration diagram of one embodiment of the present invention. [Figure 2] A diagram showing the configuration of the training task storage means and the test task storage means of the above embodiment. [Figure 3] A diagram showing the configuration of the training response result storage means and the test response result storage means of the above embodiment. [Figure 4] A diagram illustrating the configuration of the timer setting information storage means for training and testing according to the above embodiment. [Figure 5] A diagram illustrating the processing of the learning and operation phases of the facial expression recognition device according to the above embodiment. [Figure 6] A diagram showing an example of the training mode selection screen of the above embodiment. [Figure 7] This figure shows an example of the test mode selection screen of the above embodiment. [Figure 8] A flowchart illustrating the processing flow during training according to the above embodiment. [Figure 9] This figure shows an example of a screen for presenting problems during training according to the above embodiment. [Figure 10] A flowchart illustrating the processing flow during testing of the above embodiment. [Figure 11] An explanatory diagram of the processing content by the timer adjustment means of the above embodiment (Part 1: Section for extracting face image data). [Figure 12]An explanatory diagram of the processing content (part 2: flowchart) by the timer adjustment means of the above embodiment. [Figure 13] A diagram illustrating the processing details (part 3: time limit reduction conditions and time limit reduction processing) by the timer adjustment means of the above embodiment. [Figure 14] A diagram illustrating the processing details (part 4: time limit extension conditions and time limit extension processing) by the timer adjustment means of the above embodiment. [Figure 15] A diagram illustrating the processing content by the timer adjustment means of the above embodiment (Part 5: Processing content by problem type). [Modes for carrying out the invention]

[0050] One embodiment of the present invention will be described below with reference to the drawings. Figure 1 shows the overall configuration of the training-test linkage system 1 and the training system 10 of this embodiment. Figure 2 shows the configuration of the training task storage means 31 and the test task storage means 61, Figure 3 shows the configuration of the training response result storage means 32 and the test response result storage means 62, and Figure 4 shows the configuration of the timer setting information storage means 33 and 63 for training and testing. Figure 5 is an explanatory diagram of the processing during the learning and operation stages of the facial expression recognition device 27. Furthermore, Figure 6 shows an example of the training mode selection screen 80, Figure 7 shows an example of the test mode selection screen 90, Figure 8 shows a flowchart of the processing flow during training, Figure 9 shows an example of the problem presentation screen 100 during training, and Figure 10 shows a flowchart of the processing flow during testing. In addition, Figures 11 to 15 are explanatory diagrams of the processing content by the timer adjustment means.

[0051] <Regarding the "problems" and "time limits" in this invention>

[0052] In this invention, the "problem" is a time-limited task that the user must complete. Therefore, a task without a time limit is not a "problem" in this invention. For this reason, the training / test linkage system 1 and the training system 10 of this embodiment are equipped with functions to provide training and testing for the user to complete time-limited tasks. However, they may also include functions to provide training and testing for tasks without a time limit.

[0053] Furthermore, the "problem" in this invention corresponds to the "time limit." For example, in a game system, if a time limit is set for the time required to overcome (clear) the difficulties given in several stages (scenes or situations) and reach the final objective or goal, then the entire process of reaching the goal (a collection of multiple stages) is one "problem" in this invention. Therefore, if a time limit is not set for each of the difficulties in the various scenes or situations given to the user on the way to the goal, then the difficulties given in each individual stage are not the "problem" itself in this invention, but rather elements that constitute the "problem" (problem components). Thus, for example, if a time limit of 1 minute is set to clear stages 1 to 6 and reach the goal, then a timer adjustment process will be performed to shorten or lengthen that 1 minute.

[0054] On the other hand, if a time limit is set for each stage, then each stage corresponds to a "problem" in this invention. Therefore, for example, if a time limit of 10 seconds is set for each of the 1st to 6th stages, a timer adjustment process will be performed to shorten or lengthen that 10 seconds. In this case, for example, if the time limit is shortened by 1 second and changed to 9 seconds, then all stages from the 1st to the 6th must be cleared in 9 seconds each.

[0055] In this embodiment, we will primarily use an example of a language learning system to explain the process of presenting language learning questions and having the user answer them, using this as a concrete example. However, in this invention, "addressing" a "problem" includes not only language learning but also answering various other types of problems, and "problems" are not limited to problems but also include tasks that the user must accomplish, such as in a game. Therefore, when answering a problem, the outcome of the action is whether the action (answer) was correct or incorrect (correct or incorrect), but as in the case of the game described above, the outcome of the action may be whether or not the task was completed. Thus, in addition to the example of language learning, we will also provide generalized explanations of "problems," "actions," and "results of action" as appropriate.

[0056] <Definitions of other terms>

[0057] (The relationship between "results of action" or "results data" and "data showing the content of the action") Here, we will explain terms that may be misinterpreted. First, "action result" and "action result data" are information that is determined through judgment processing by the training action result determination means 23 and the test action result determination means 53. Therefore, the status information transmitted from the user terminal 70 via the network 2 has not undergone judgment processing by the training action result determination means 23 and the test action result determination means 53, and is not "action result" or "action result data." In this application, such undetermined information is referred to as "data indicating the content of the action."

[0058] Specifically, "results of action" and "results data of action" include information indicating that the action was correct (if the task is a problem, information indicating that the answer was correct), information indicating that the action was incorrect (information indicating that the answer was incorrect), information indicating that the task was completed, information indicating that the task was not completed, or information indicating that the action could not be taken due to time running out (information indicating that it was treated as incorrect due to time running out). The remaining time until the time limit at the time of action (time of answering) is also included in "results of action" and "results data of action".

[0059] On the other hand, "data indicating the content of the action taken" is information generated by the user's operation of the input means 73 to address the issue, and does not indicate whether the action taken was correct or incorrect, whether the issue was achieved or not, etc. Specifically, if the issue is a problem, "data indicating the content of the action taken" is "answer data," and does not indicate whether the answer is correct or incorrect.

[0060] Furthermore, the judgment processing by the training action result judgment means 23 and the test action result judgment means 53 for obtaining "action results" and "action result data" is not limited to judgment processing of match or mismatch using "data indicating the content of the action" transmitted from the user terminal 70 via the network 2, but also includes judgment processing by comparing with a threshold. Specifically, the former, the match or mismatch judgment processing, is a process in which, for example, correct answer data for a problem is prepared, and the system judges whether there is a match or mismatch with that correct answer data. If there is a match, the action result is that the action is correct, and if there is a mismatch, the action result is that the action is incorrect. On the other hand, the latter, the judgment processing by comparing with a threshold, is a process in which, for example, when the number of seconds that the user continues to operate the input means 73 (such as pressing a button) is obtained via the network 2 as "data indicating the content of the action", that number of seconds is compared with a threshold, and if it is judged to be greater than or equal to the threshold, the action result is that the task has been completed, and if it is judged to be less than the threshold, the action result is that the task has not been completed. In the latter method of judgment processing based on comparison with a threshold, if two or more thresholds are set, the degree of achievement (degree of success) can be used as the outcome of the action.

[0061] (The relationship between "results" or "results data" and "aggregated results" or "aggregated results data": the same applies when "training" or "test" is prefixed to the term.) "Response results" and "response result data" are higher-level terms that also include "aggregate results" and "aggregate result data." Since the judgment process by the training response result judgment means 23 and the test response result judgment means 53 includes evaluation and aggregation, "rates" and "percentages" such as the correct answer rate obtained by aggregating the response results of each task (each problem) on a section-by-section (each task group) basis, "total values" such as the number of correct answers in a given number of tasks (each problem), and various calculated values ​​such as "average value," "standard deviation," and "variance" are also included in "response results" and "response result data." Accordingly, although the training response result storage means 32 and the test response result storage means 62 in Figure 3 are provided with a task-by-task response result table and an aggregate result table, both tables store "response results" and "response result data."

[0062] <Overall configuration of training and test integration system 1, including training system 10>

[0063] In Figure 1, the training-test integration system 1 is configured by connecting a training management server 20, which performs processing to manage training aimed at improving the user's ability to handle given tasks; an authentication server 40, which performs user authentication processing; a test management server 50, which performs processing to manage tests that measure the user's ability to handle given tasks; and user terminals 70, which are operated by each user, via a network 2.

[0064] As shown by the dashed line in Figure 1, the training system 10 is composed of the training management server 20, the authentication server 40, and the user terminal 70 within the training-test integration system 1. Furthermore, the test system is composed of the test management server 50, the authentication server 40, and the user terminal 70 within the training-test integration system 1.

[0065] Here, Network 2 is primarily the internet, but it can also be a combination of the internet and an internal network (such as a company intranet or company LAN), and it does not matter whether it is wired, wireless, or a hybrid of wired and wireless.

[0066] The training management server 20 is composed of one or more computers and performs various training processes to improve the user's ability to handle given tasks, and stores data necessary for these training processes. It includes a login means 21, a training task presentation means 22, a training response result determination means 23, a timer adjustment means 24, a data transmission means 25 for the test management server, a face image acquisition means 26, a face expression recognition device 27, a training task storage means 31, and a training response The facial expression recognition device is composed of a result storage means 32, a training timer setting information storage means 33, an identification result storage means 34, and a point storage means 35.

[0067] Here, each of the means 21-26 and the facial expression recognition means 27A are implemented by a central processing unit (CPU) located inside the training management server 20, one or more programs that define the operating procedure of this CPU, and working memory such as main memory and cache memory. Details of each of these means 21-26, 27A will be described later.

[0068] Furthermore, non-volatile memory such as a hard disk drive (HDD) or solid-state drive (SSD) can be used as each of the storage means 31-35 and the facial expression recognition model storage means 27B. Details of each of these storage means 31-35 and 27B will be described later.

[0069] The authentication server 40 is composed of one or more computers and performs user authentication when a user logs in from the user terminal 70 to the training management server 20 or test management server 50 in order to perform training or tests on the user terminal 70. The authentication server 40 is equipped with authentication means 41 and authentication information storage means 42. User ID (user identification information), password, and user face image data that has been previously captured and registered are associated and stored in the authentication information storage means 42. The authentication means 41 performs user authentication using the user ID, password, and face image data stored in the authentication information storage means 42. Authentication processing using face image data is performed not only during login but also during the execution of tests and is used for monitoring purposes to prevent fraud such as impersonation or substitution of test takers.

[0070] Here, the authentication means 41 is implemented by a central processing unit (CPU) located inside the authentication server 40, one or more programs that define the operating procedures of this CPU, and working memory such as main memory and cache memory. Furthermore, as the authentication information storage means 42, non-volatile memory such as a hard disk drive (HDD) or solid-state drive (SSD) can be used.

[0071] The test management server 50 is composed of one or more computers and performs various tests to measure the user's ability to handle given tasks, as well as storing data necessary for these tests. It includes a login means 51, a test task presentation means 52, a test response result determination means 53, a timer setting means 54, a data transmission means 55 for the training management server, a face image acquisition means 56, a monitoring means 57, a test task storage means 61, a test response result storage means 62, and a timer setting information storage means 63 for the test.

[0072] Here, each of the means 51 to 57 is implemented by a central processing unit (CPU) located inside the test management server 50, one or more programs that define the operating procedure of this CPU, and working memory such as main memory and cache memory. Details of each of these means 51 to 57 will be described later.

[0073] Furthermore, each of the storage means 61 to 63 can be, for example, a non-volatile memory such as a hard disk drive (HDD) or a solid-state drive (SSD). Details of each of these storage means 61 to 63 will be described later.

[0074] The user terminal 70 is a computer, and its main body 71 is equipped with processing means 72 that perform various processes necessary for training and testing conducted with the training management server 20 and the test management server 50 (including screen display processing, user input reception processing, data transmission and reception processing, and image capture processing from a camera). The user terminal 70 also includes input means 73 such as a mouse, keyboard, touch panel, or microphone for voice input, display means 74 such as an LCD display, and shooting means 75 such as a camera. The user terminal 70 may also be a mobile device such as a smartphone or tablet.

[0075] Here, the processing means 72 is realized by a central processing unit (CPU) located inside the main unit 71, one or more programs that define the operating procedures of this CPU (including a web browser and programs associated with web pages received from the training management server 20 and the test management server 50), and working memory such as main memory and cache memory. Although not shown in the figures, the main unit 71 of the user terminal 70 is equipped with non-volatile memory such as a hard disk drive (HDD) or solid-state drive (SSD).

[0076] <Configuration of Training Management Server 20 / Login Method 21>

[0077] The login means 21 receives the user ID and password transmitted from the user terminal 70 via the network 2 during training, or in addition to these, receives facial image data via the facial image acquisition means 26, transmits this authentication information to the authentication server 40 via the network 2, receives the authentication result from the authentication server 40, and if authentication fails, executes the process of sending notification data to the user terminal 70 to that effect. Note that authentication using facial image data may be omitted during training.

[0078] <Configuration of training management server 20 / training task presentation means 22: Figures 6 and 9>

[0079] The training task presentation means 22, during training, presents the user with a time-limited task stored in the training task storage means 31 (see Figure 2) on the screen of the user terminal 70, and also performs a process to display a timer on the screen of the user terminal 70 for the user while they are working on the task, showing the current remaining time until the time limit given to the user to complete the task, or the current elapsed time since the task was presented.

[0080] More specifically, the training task presentation means 22 uses the task data (problem data) stored in the training task storage means 31 (see Figure 2) to create display data for the task presentation screen (problem presentation screen, see Figure 9) (for example, data for a web page, including audio data for reading aloud the task text (problem text)), and transmits the created display data to the user terminal 70 via the network 2. As a result, the user terminal 70 uses the processing means 72 to display the task presentation screen (problem presentation screen) on the display means 74 and to read aloud the task text (problem text).

[0081] Furthermore, the display of the task presentation screen (problem presentation screen) includes a timer display. This timer display by the training task presentation means 22 shows the current remaining time or current elapsed time, which changes while the user is considering what action to take, from the time the problem is presented (task presentation time) P1 (see Figure 11). The example in Figure 9 is a timer display showing the remaining time. Note that the time the problem is presented (task presentation time) P1 is the start time of the display of the task presentation screen (problem presentation screen) or the start time of the voice reading.

[0082] Therefore, the elapsed time T2 and remaining time T3 shown in Figure 11 are the elapsed time and remaining time at the time of answering (time of action) P2 when the answer was given (i.e., information generated when the user finishes dealing with the task). The timer display by the training task presentation means 22, when displaying the remaining time, shows how much time is left to answer (deal with) the task at the moment the user is looking at the timer, and when displaying the elapsed time, it shows how much time has passed since the time the problem was presented (time the task was presented) P1 at the moment the user is looking at the timer.

[0083] Note that the time limit T1 in Figure 11 is the current setting and does not change while the user is working on a particular task (problem). This time limit T1 is changed when the timer adjustment means 24 changes the time limit setting and presents a task (problem) after that setting change. In this embodiment, for example, the time limit can be changed on a section-by-section basis, so it may have been changed when presenting a task (problem) in a different section. Also, since the time limit is determined for each task type (problem type), it may have been changed when presenting a task of a different task type.

[0084] Furthermore, the timer display by the training task presentation means 22 may be a numerical value indicating the remaining time or elapsed time (for example, 6 seconds, 1 minute, etc.) (as in the example in Figure 9), or it may be a display using the length of a bar graph (a display in which a bar graph showing elapsed time extends, or a display in which a bar graph showing remaining time shrinks, or a display combining both), or it may be a display in which the current position (position on the time axis) is indicated by a mark such as a triangle or an arrow on a line or bar of a fixed length indicating the length of the time limit T1 (therefore, the elapsed time and remaining time are displayed simultaneously, for example, if the position of the triangle mark moves to the right in ――――△――, the left side of the triangle mark indicates the elapsed time and the right side indicates the remaining time).

[0085] Furthermore, the process of changing the timer display by the training task presentation means 22 as time progresses (as described above, the process of changing numerical values ​​such as seconds or minutes, the length of lines or bars, the position of marks such as triangles, etc.) may be implemented by sequentially transmitting data indicating the remaining time or elapsed time from the training task presentation means 22 via the network 1, or by a program provided by the training task presentation means 22 along with the data for displaying the web screen (for example, JavaScript (ECMAScript), i.e., a program obtained by a web browser), or by an auxiliary means of the training task presentation means 22 (part of the processing means 72) that is pre-installed on the user terminal 70.

[0086] Furthermore, the training task presentation means 22 also processes the user's selection of a training mode. Specifically, the training task presentation means 22 transmits the display data (Web page data) for the training mode selection screen 80 shown in Figure 6 to the user terminal 70 via the network 2, and receives the user's training mode selection information transmitted from the user terminal 70 via the network 2. The received selection information is then associated with user identification information and stored in the training timer setting information storage means 33 (see Figure 4).

[0087] In Figure 6, the training mode selection screen 80 is provided with a selection section 81 for (Trg1) simple automatic adjustment mode, a selection section 82 for (Trg2) complex automatic adjustment mode, and a selection section 83 for (Trg3) self-setting mode. Trg is an abbreviation for training.

[0088] (Trg1) The simple automatic adjustment mode is a mode in which the timer limit is automatically adjusted using only the results of how the task was handled (whether the answer to the problem was correct or incorrect). Note that when this (Trg1) simple automatic adjustment mode is selected, the automatic adjustment process of the time limit is performed by the timer adjustment means 24, so the training task presentation means 22 only accepts the mode selection.

[0089] □(Trg2) The combined automatic adjustment mode is a mode that automatically adjusts the timer limit using the results of how the task was handled (whether the answer to the problem was correct or incorrect) and the facial expression recognition results. Note that when this (Trg2) combined automatic adjustment mode is selected, the automatic adjustment process of the time limit is performed by the timer adjustment means 24, so the training task presentation means 22 only accepts the mode selection.

[0090] □(Trg3) Self-Setting Mode is a mode in which the user sets the timer limit themselves. The time limit can be changed freely, and the user can freely input and set the amount of time they want to use for training. When this (Trg3) Self-Setting Mode is selected, the acceptance of the user's input for the self-set time limit is performed by the timer adjustment means 24, so the training task presentation means 22 only accepts the mode selection. When this (Trg3) Self-Setting Mode is selected, if certain conditions are met, the timer adjustment means 24 will display an advice message to the user recommending that they shorten the time limit. This is because users may not realize that they are capable of shortening the time limit (their own ability to cope), so the advice message is displayed to encourage the user to shorten the time limit through self-setting.

[0091] Furthermore, the training mode selection screen 80 in Figure 6 is provided with two selection units for how to set the initial value of the timer limit when either the (Trg1) simple automatic adjustment mode or the (Trg2) combined automatic adjustment mode is selected: (TrgA) a selection unit 84 to set the initial value to a standard value predetermined by the system (stored in the training task storage means 31 as shown in Figure 2), and (TrgB) a selection unit 85 to set the initial value according to the test performance using the test summary result data received from the test management server 50. The selection in these selection units 84 and 85 determines the method for determining the initial value when the timer limit is automatically adjusted by the timer adjustment means 24 when either the (Trg1) simple automatic adjustment mode or the (Trg2) combined automatic adjustment mode is selected. If no test has been performed yet, there is no test summary result data in the test management server 50's test handling result storage means 62 (see Figure 3), so (TrgA) is selected. However, if a test has been performed, there is test summary result data, so (TrgB) can be selected. However, even if test summary result data exists, (TrgA) may be selected. When the (TrgB) selection unit 85 is selected, the timer adjustment means 24 automatically sets the initial timer limit value according to the test results using the test summary result data received from the test management server 50.

[0092] Furthermore, the training mode selection screen 80 in Figure 6 is provided with a "Send" button 86. When the user presses the "Send" button 86 after making a selection in the selection sections 81-85, the training mode selection information is transmitted from the user terminal 70 to the training management server 20 via the network 2, received by the training task presentation means 22, and stored in the timer setting information storage means 33 (see Figure 4).

[0093] <Configuration of Training Management Server 20 / Training Response Result Determination Means 23: Figure 9>

[0094] The training response result determination means 23 receives the user's input means 73 on the user terminal 70 as training to deal with a task (answer a problem), and makes a judgment including evaluation or aggregation to obtain a response result, including whether the response to the task (answer to the problem) was correct or whether the task was achieved or not, and then executes a process to store the obtained user response result data (various information derived from objective judgment from data showing the content of the user's response received from the user terminal 70, including not only information on whether the response was correct or whether the task was achieved or not, but also training aggregation result data obtained by aggregation and analysis) in the training response result storage means 32 (see Figure 3).

[0095] More specifically, the training response result determination means 23 first receives data via the network 2 indicating the content of the response (data indicating the user's intentions, thought processes, or physical reactions) resulting from the user's operation of the input means 73 of the user terminal 70, which is performed as training to respond to a task (answer a problem). The received data indicating the content of the response is compared with the correct answer data (more generally expressed as data for successful responses) stored in the training task storage means 31 (see Figure 2), and it is determined whether they match. The result of this determination is then stored in the training response result storage means 32 (see Figure 3) as user response result data, associated with user identification information. The data stored here is the data to be stored in the task-unit response result table of the training response result storage means 32 in Figure 3.

[0096] In this case, the training response result determination means 23 treats the response as incorrect if the answer operation (operation of the input means 73 for dealing with the task) is not performed within the time limit, as shown in Figure 9. Note that in the example in Figure 9, since it is a multiple-choice question, the expression is "treated as incorrect" (treated as the response was not correct), but to include "treated as unable to complete the task" for other task types, a more general expression would be "treated as having failed to deal with the task".

[0097] Furthermore, when the training response result determination means 23 receives data from the user terminal 70 indicating the content of the response to a certain task (problem), it determines whether that task is the last task in a section (group of tasks) based on the number of tasks (number of tasks) determined by the problem type number (task type identification information) and section number (task group identification information) for that task (problem), and the problem number (task identification information). If it is not the last task, it transfers processing to the training task presentation means 22 in order to present the next task (problem).

[0098] In this case, as shown in Figure 9, the training response result determination means 23 proceeds to the screen 100 that presents the next problem (task) if the answer is correct (more generally, if the task has been successfully addressed). On the other hand, if the answer is incorrect or treated as incorrect (more generally, if the task has been failed to be addressed or treated as a failure to address), the correct answer is displayed (or a message indicating that the task could not be completed, more generally, a message indicating that the task could not be addressed or the reason for the failure) along with a "Proceed to next problem" button (more generally, a "Proceed to next task" button) (not shown). When the "Proceed to next problem" button ("Proceed to next task" button) is pressed (clicked), processing is transferred to the training task presentation means 22 in order to present the next task (problem).

[0099] On the other hand, when the training response result determination means 23 receives data indicating the content of the response to a certain task (problem), if that task is the last task in a section (group of tasks) (at the end of the section), it performs aggregation and analysis processing for that section, and then stores the obtained training aggregation result data as user response result data in the training response result storage means 32 (see Figure 3), associating it with user identification information, problem type number, and section number. The data stored here is the data to be stored in the aggregation result table of the training response result storage means 32 in Figure 3. Therefore, the section aggregation and analysis processing is a process to determine the number of correct answers (number of correct responses, or number of tasks completed) and the accuracy rate (percentage of correct responses, or task completion rate) out of the number of problems (number of tasks) in the section, using the problem-unit (unit of task) response result data stored in the task-unit response result table in Figure 3.

[0100] Furthermore, at the end of a section, the training response result determination means 23 performs a process to convert the facial expression recognition results obtained by the facial expression recognizer 27 on a problem-by-problem (task-by-task) basis into section-by-section (task-group) basis, in addition to the aggregation and analysis process of the section described above. That is, it uses the facial expression recognition results (tense expression, relaxed expression, other) for each problem-by-problem (task-by-task) stored in the task-by-task identification result table of the identification result storage means 34 in Figure 5 to obtain a section-by-section facial expression recognition result (overall), and stores the obtained facial expression recognition result (overall) in the overall identification result table of the identification result storage means 34 in Figure 5, in association with the user identification information, problem type number, and section number.

[0101] This section-based conversion process involves, for example, determining the overall facial expression recognition result for a section if the proportion of problems (tasks) that make up a section (group of tasks) that result in a "tense expression" is above a predetermined threshold, and determining the overall facial expression recognition result for a section if the proportion of problems (tasks) that result in a "relaxed expression" is above a predetermined threshold, and so on.

[0102] Specifically, for example, if a section consists of 33 questions, and 28 of those questions involve "tense facial expressions," then the proportion of questions involving "tense facial expressions" is 28 / 33 = 0.84 (84%), which is above the threshold of 80%. Therefore, the facial expression recognition result (overall) for that section is considered to be "tense facial expression."

[0103] For example, if a section consists of 33 questions, and 18 of them are questions describing a "relaxed expression," then the proportion of questions describing a "relaxed expression" is 18 / 33 = 0.54 (54%), which is below the threshold of 80%. Therefore, the overall facial expression recognition result for the section will not be "relaxed expression." However, in this case, not having a "relaxed expression" does not mean that it is a "tense expression," so we also determine whether or not it is a "tense expression." Therefore, if a section consists of 33 questions, and 13 of them are questions describing a "tense expression," then the proportion of questions describing a "tense expression" is 13 / 33 = 0.39 (39%), which is below the threshold of 80%. Therefore, the overall facial expression recognition result for the section will not be "tense expression." Thus, the overall facial expression recognition result for the section is neither a "relaxed expression" nor a "tense expression," and will be "other." In other words, if an expression is judged to be either "relaxed" or "tense," there is no need to determine whether it is the other. However, if an expression is judged to be neither "relaxed" nor "tense," there is a possibility that it is the other, so it is necessary to determine whether it is the other.

[0104] Alternatively, instead of simply comparing the true proportion with the threshold, one could calculate a weighted average hypothetical proportion that gives more weight to facial expression recognition results for problems in the latter half of the section (and therefore, situations where the user is becoming more familiar with solving problems), and then compare this hypothetical proportion with the threshold. In this case, it would suffice to assume that there are K times (where K is an integer greater than or equal to 2) facial expression recognition results for problems in the latter half of the section. Specifically, if the recognition results for problems in the latter half of the section are arranged in the order of "relaxed expression," "tense expression," "relaxed expression," ... then one could assume that they are arranged in the order of "relaxed expression" × K, "tense expression" × K, "relaxed expression" × K, ... and calculate the hypothetical proportion.

[0105] Furthermore, at the end of a section, the training response result determination means 23 compares the predetermined standard time limit (see Figure 2) with the time limit set by the user (when the current selected mode stored in the training timer setting information storage means 33 in Figure 4 is (Trg3) self-set mode) (the "current time limit" stored in the training timer setting information storage means 33 in Figure 4). If the user has set a time limit shorter than the standard value (see Figure 2), the means determines whether the numerical value of the data indicating the user's performance obtained by aggregating the response results (for example, the accuracy rate stored in the training response result storage means 32 in Figure 3) is above a predetermined threshold (for example, 60%). If it is above the threshold (i.e., the performance is good), the means awards the user a predetermined number of points, or a number of points corresponding to the difference between the standard time limit and the time limit set by the user, and adds the number of points awarded to the number of points held by the user, which is stored in the point storage means 35 in association with the user identification information for that user.

[0106] Here, "awarding a number of points corresponding to the magnitude of the difference between the standard time limit and the user's self-set time limit" means, for example, if the difference in time is less than 10% of the standard time limit, 10 points will be awarded; if the difference is 10% or more but less than 20% of the standard time limit, 20 points will be awarded; if the difference is 20% or more but less than 30% of the standard time limit, 30 points will be awarded; if the difference is 30% or more but less than 40% of the standard time limit, 40 points will be awarded; and if the difference is 50% or more of the standard time limit, 50 points will be awarded, and so on.

[0107] Furthermore, the points can be used to pay service fees that users pay for training or other services provided by the operating company of the training management server 20 or another company affiliated with this operating company.

[0108] Subsequently, the training response result determination means 23, upon completion of a section, proceeds to the timer adjustment means 24. After the timer adjustment means 24 has finished processing, it determines whether to terminate the training based on user instruction information from the user terminal 70. If it receives instruction information to continue training (including the section number of the next training target), it passes the next section number to the training task presentation means 22 and proceeds to the training task presentation means 22.

[0109] Furthermore, if the user's performance in a section for which training has been completed is unsatisfactory (for example, if the accuracy rate of the section (see Figure 3) is below a predetermined threshold (for example, 60%)), the training result determination means 23 may display a message on the user terminal 70 screen indicating that the next training should be in the same section as the one just completed and that it would be better to redo the training for that section (redo recommendation message) or a message forcing the user to redo the training (redo forced message).

[0110] <Configuration of training management server 20 / timer adjustment means 24: Figures 11-15>

[0111] ((Trg1) Simple automatic adjustment mode, time limit reduction conditions, time limit extension conditions, time limit reduction process, time limit extension process) When the user performing the training (Trg1) has selected the simplified automatic adjustment mode (see Figure 6), the timer adjustment means 24 uses the response results obtained from the judgment process by the training response result judgment means 23, that is, using the training response result data (including training summary result data) stored in the training response result storage means 32 (see Figure 3), to determine whether the response was correct or whether the percentage or number of consecutive tasks achieved, or other response results that are predetermined conditions for shortening the time limit are met. If these conditions for shortening the time limit are met, the timer adjustment means 24 executes a time limit shortening process to shorten the time limit used for the timer display. At the same time, it determines whether the percentage or number of consecutive tasks that were not achieved or other response results that were incorrect or unsuccessful are met, or other conditions for extending the time limit are predetermined conditions that are predetermined conditions for extending the time limit are met.

[0112] In this case, when the timer adjustment means 24 determines which training mode the user performing the training has selected, it refers to the training mode stored in the "Current Selected Mode" column of the training timer setting information storage means 33 (see Figure 4), which is associated with the user identification information for that user and the problem type number (task type identification information) of the problem being trained for that user.

[0113] Furthermore, regardless of which training mode is selected, if the time limit is changed, the timer adjustment means 24 stores the new time limit in the "Current Time Limit" column of the training timer setting information storage means 33 (see Figure 4), associating it with the user identification information and the problem type number (task type identification information). Accordingly, the previous time limit that was already stored in the "Current Time Limit" column is moved to the "Previous Time Limit" column, and the time limit that was already stored in the "Previous Time Limit" column is moved to the "Time Limit Before That" column.

[0114] Here, the time limit reduction conditions and time limit extension conditions are predetermined conditions that the action results (including aggregated results) must satisfy. Specifically, the time limit reduction conditions and time limit reduction process are the conditions and processes shown in the example in Figure 13, and the time limit extension conditions and time limit extension process are the conditions and processes shown in the example in Figure 14.

[0115] As shown in Figure 13, in this embodiment, examples of time limit reduction conditions are provided as condition SC1 (the percentage of correct answers out of all questions is X1% or more), condition SC2 (the number of consecutive correct answers is X2 or more), and condition SC3 (the remaining time T3 until the time limit T1 at the time of answering a question P2 is X3 seconds or more). However, it is not limited to these, and in short, any condition predetermined by the results of the response is acceptable. In this embodiment, condition SC3 is provided as a combination condition with condition SC1 or condition SC2.

[0116] Furthermore, as shown in Figure 14, in this embodiment, examples of time limit extension conditions are provided, such as condition LC1 (the percentage of incorrect answers out of all questions is Y1% or more), condition LC2 (the number of consecutive incorrect answers is Y2 or more), condition LC3 (the ratio of the number of questions that could not be addressed due to time running out to the total number of questions is Y3% or more), and condition LC4 (the number of consecutive questions that could not be addressed due to time running out is Y4 or more). However, the conditions are not limited to these, and in short, any condition predetermined by the outcome of the response is acceptable.

[0117] Furthermore, as shown in Figure 13, in this embodiment, as examples of time limit reduction processes, process SP1 (reduces the currently set time limit T1 by X4 seconds) and process SP2 (reduces the time limit by a value obtained by multiplying the average of the remaining time T3 until the time limit T1 at the time of answering P2 for the correctly answered questions by a coefficient X5). However, the process is not limited to these; in short, it is sufficient to reduce the time limit according to predetermined rules.

[0118] Furthermore, as shown in Figure 14, in this embodiment, process LP1 (extending the currently set time limit T1 by Y4 seconds) is provided as an example of time limit extension processing, but it is not limited to this; in short, it is sufficient to extend the time limit according to predetermined rules.

[0119] Furthermore, the time limit reduction conditions and processing shown in Figure 13, and the time limit extension conditions and processing shown in Figure 14, are predetermined for each problem type. Also, the standard value of the time limit T1 (see Figure 11) is predetermined for each problem type.Therefore, as shown in Figure 15, the time limit reduction conditions and processing, the time limit extension conditions and processing, and the standard value of the time limit T1 can be organized in a correspondence table in association with the problem type number (task type identification information).The contents organized in such a correspondence table may be written in each program that implements the processing for each problem type (each task type), or they may be stored in a condition / processing content storage means (not shown) and read by each program.

[0120] ((Trg2) Combined automatic adjustment mode, processing to add relaxed expression, processing to add tense expression) Furthermore, if the user performing the training has selected the (Trg2) composite automatic adjustment mode (see Figure 6), the timer adjustment means 24 performs a process to adjust the time limit used for the timer display using the response result obtained by the training response result determination means 23, i.e., the training response result data (including training summary result data) stored in the training response result storage means 32 (see Figure 3), and the identification result by the facial expression identification means 27A (however, the comprehensive identification result converted into section units by the training response result determination means 23), i.e., the facial expression identification result (tense expression or relaxed expression) stored in the comprehensive identification result table of the identification result storage means 34 (see Figure 5).

[0121] More specifically, in this embodiment, when the (Trg2) combined automatic adjustment mode is selected, the timer adjustment means 24 performs both the relaxed expression addition processing and the tense expression addition processing. Alternatively, the configuration may perform only the relaxed expression addition processing.

[0122] Here, the processing that takes into account relaxed facial expressions is a process that shortens the time limit used for displaying the timer when it is determined that the time limit reduction condition is met using the response result obtained by the training response result determination means 23 (training response result data stored in the training response result storage means 32 (see Figure 3)) and when the facial expression identification means 27A outputs an identification result that indicates relaxed facial expression (however, this is the overall identification result converted into section units by the training response result determination means 23).

[0123] Furthermore, the tension expression processing is a process that extends the time limit used for the timer display when it is determined that the time limit extension condition is met using the response result obtained by the training response result determination means 23 (training response result data stored in the training response result storage means 32 (see Figure 3)) and when the facial expression identification means 27A outputs an identification result that indicates a tension expression (however, this is the overall identification result converted into section units by the training response result determination means 23).

[0124] ((Trg3) Self-configuration mode) Furthermore, if the user performing the training (Trg3) has selected the self-setting mode (see Figure 6), the timer adjustment means 24 transmits display data (Web page data) for a self-setting time limit screen (not shown) or a confirmation screen (not shown) regarding whether or not to change the time limit settings (not shown) to the user terminal 70 via the network 2 at the start and / or end of a section (group of tasks), and also receives the self-set time limit transmitted from the user terminal 70 via the network 2, and performs a process to store the received self-set time limit in the training timer setting information storage means 33 (see Figure 4) in association with the user identification information and the problem type number (task type identification information) of that section.

[0125] Furthermore, when the user performing the training (Trg3) has selected the self-setting mode, the timer adjustment means 24 displays an advice message to the user recommending that they shorten the time limit if certain conditions are met. Specifically, the timer adjustment means 24 uses the response result obtained by the training response result determination means 23, that is, the training response result data (including training summary result data) stored in the training response result storage means 32 (see Figure 3), to determine that the conditions for shortening the time limit are met, and when the facial expression recognition means 27A outputs an identification result indicating a relaxed expression (however, this is the overall identification result converted to section units by the training response result determination means 23), it transmits display data (Web page data) for an advice message recommending that the user shorten the time limit used for the timer display to the user terminal 70 via the network 2. As a result, an advice message to that effect is displayed on the screen of the user terminal 70.

[0126] (When the initial setting of the (TrgA) time limit is set to the standard value) Then, if the user performing the training has selected either (Trg1) simple automatic adjustment mode or (Trg2) complex automatic adjustment mode, and has also selected to use the default value for the initial setting of the time limit in (TrgA) (see Figure 6), the timer adjustment means 24 executes a process to set the initial setting of the timer's time limit to a predetermined default value (stored in the test task storage means 61 as shown in Figure 2). Accordingly, the default value is stored in the "current time limit" column of the training timer setting information storage means 33 (see Figure 4).

[0127] (When using (TrgB) test summary result data to automatically set the initial time limit value according to test performance) Furthermore, if the user performing the training has selected (Trg1) simple automatic adjustment mode or (Trg2) complex automatic adjustment mode, and has also selected (TrgB) to automatically set the initial time limit value according to the test performance using the test summary result data (see Figure 6), the timer adjustment means 24 sends a request to the test management server 50 via network 2 to acquire test summary result data for that user, along with user identification information and problem type number (task type identification information). In response to this acquisition request, the test management server 50 receives the test summary result data sent via network 2 by the training management server cooperation data transmission means 55, and uses the received test summary result data to determine the test performance derived from the test summary result data. Based on the determined test performance, the timer adjustment means 24 executes a process to set the training start time limit shorter the better the test performance. Accordingly, the "current time limit" column of the training timer setting information storage means 33 (see Figure 4) stores the time limit according to the test performance.

[0128] Specifically, the timer adjustment means 24 receives test aggregation result data from the test management server 50, including, for example, the accuracy rate (the rate at which the response was correct, or the task completion rate, more generally, the success rate of the response) and the time limit used, which are stored in the aggregation result table of the test response result storage means 62 (see Figure 3), along with user identification information, problem type number (task type identification information), and section number (task group identification information).

[0129] Next, the timer adjustment means 24 calculates the average accuracy rate (success rate of handling) for each section and the average time limit used for each section. For example, if the average accuracy rate (success rate of handling) is 95% or higher, the training start time limit is set to be 2 seconds shorter than the average time limit used (rounded to the nearest second). If the average accuracy rate (success rate of handling) is less than 95% but 80% or higher, the training start time limit is set to be 1 second shorter than the average time limit used (rounded to the nearest second). If the average accuracy rate (success rate of handling) is less than 80%, the training start time limit is set to be the average time limit used (rounded to the nearest second). This enables the automatic setting of the timer's initial time limit value according to the test results. The above algorithms for determining the initial time limit and the thresholds used for judgment, such as 95% and 80%, are merely examples and are not limiting. In short, any algorithm or threshold that can implement a process where the training start time is shortened as the test performance improves is acceptable.

[0130] It should be noted that in this embodiment, the time limit can be changed on a section-by-section basis, regardless of whether it is automatic adjustment or self-setting. In other words, the timing of a section change is the timing at which the time limit can be changed. On the other hand, since the standard time limit is set for each question type (task type), for example, 7 seconds for multiple-choice questions and 15 seconds for rearrangement questions (see Figure 15), the time limit is set for each question type. Therefore, as shown in Figure 4, the set time limit is stored in association with the question type number (task type identification information), and for questions (tasks) belonging to that question type (task type), the same time limit will be used even if the section changes, unless the time limit setting is changed.

[0131] <Configuration of the training management server 20 / test management server data transmission means 25>

[0132] The data transmission means 25 for the test management server receives a request to acquire the current time limit, which is transmitted via network 2 from the timer setting means 54 of the test management server 50 along with user identification information and problem type number (task type identification information) for the user performing the test. In response to this acquisition request, the means executes a process to transmit the current time limit for the user stored in the "current time limit" column of the training timer setting information storage means 33 (see Figure 4), along with the user identification information and problem type number (task type identification information) for the user, to the test management server 50 via network 2.

[0133] <Configuration of training management server 20 / face image acquisition means 26>

[0134] The face image acquisition means 26 acquires the user's face image data captured by the shooting means 75 of the user terminal 70 via the network 2. When logging in for training, it passes the acquired face image data to the login means 21 as user authentication information. Subsequently, when training starts, it passes the acquired face image data to the face expression classifier 27 as input data for identifying the user's facial expressions.

[0135] In this embodiment, the face image acquisition means 26 receives face image data (video) that is sequentially transmitted (streamed) from the user terminal 70 via the network 2, but it may also be configured to receive face image data (still images) that are transmitted from the user terminal 70 at appropriate time intervals.

[0136] In login authentication by the login means 21 and authentication server 40, a comparison is made with the user's face image data (still image) registered in the authentication information storage means 42. Therefore, the face image data passed from the face image acquisition means 26 to the login means 21 can be either one frame from the video data or one still image. However, if authentication is to be performed by integrating multiple comparison results, then multiple frames from the video data or multiple still images may be used.

[0137] On the other hand, the facial image data passed from the facial image acquisition means 26 to the facial expression classifier 27 is adapted to the format of the input data to the facial expression classifier 27 (number of dimensions and input method: see Figures 5 and 11), as will be described in detail later, and will be either video data or multiple still image data. Even when it is video data, it will be video data of multiple extracted sections (see Figure 11) extracted from a long video data with time staggered intervals. The facial image acquisition means 26 also performs the preparation processing of the input data to the facial expression classifier 27, including this extraction process. In this case, the pattern recognition processing (face expression recognition processing) by the face expression classifier 27 is performed during the time period from the presentation of the problem (task) shown in Figure 11, P1, to the answer (action) time, P2. Therefore, the face image acquisition means 26 receives information on the start timing of face expression recognition from the training task presentation means 22 at the presentation of the problem (task) P1, and information on the end timing of face expression recognition from the training action result determination means 23 at the answer (action) time, P2. Based on this, the face image acquisition means 26 grasps the timings of P1 and P2 and performs preparatory processing including extraction processing.

[0138] <Configuration of Training Management Server 20 / Facial Expression Recognition Device 27>

[0139] The facial expression classifier 27 can be implemented, for example, by existing pattern recognition methods such as neural networks, and consists of a facial expression recognition means 27A and a facial expression recognition model storage means 27B. The facial expression recognition means 27A uses the facial expression recognition model stored in the facial expression recognition model storage means 27B to perform pattern recognition processing for facial expression recognition according to an algorithm such as a neural network, and outputs a recognition result of whether the facial expression is tense or relaxed. The facial expression recognition model is a set of parameters for a neural network determined by machine learning using methods such as backpropagation.

[0140] During the learning phase, as shown in Figure 5, a learning dataset is prepared consisting of numerous learning data and corresponding tag information. The learning data consists of facial image data from numerous subjects, and the tag information is set to "tense expression, relaxed expression" for two-class classification and to "tense expression, relaxed expression, other expression" for three-class classification. Note that classification of four or more classes is also possible if tense and relaxed expressions are included.

[0141] When collecting facial image data and corresponding tag information, multiple subjects were presented with numerous problems similar to the various problem types presented in the training system 10 of this embodiment, and were asked to actually answer them under various time limits. Then, the subjects themselves were interviewed about their state while considering the answer to each problem (whether they considered and answered in a state of concentration and tension, or with ease, or, in the case of 3-class classification, in a state other than that), and the objective state assessment results of the system builder who observed them were also taken into consideration, and the facial image data captured while solving each problem was tagged.

[0142] Therefore, this is not a training dataset collected under general circumstances where various things occur, but rather a training dataset collected under the specific circumstances of solving a problem (dealing with a task). Consequently, even when providing "other expressions" as tag information, or when providing expressions with specific names that are neither tense nor relaxed, these are expressions that appeared when dealing with a problem (task), such as the joyful expression "Yay!" when the answer is correct, the "Oh no!" when the answer is wrong, or the regretful / disappointed expressions such as "Ahhh." when the answer is wrong. Thus, although it is not the identification of expressions of the six basic emotions (anger, disgust, fear, joy, sadness, surprise) based on the general Ekman theory, if it is possible to identify facial expressions including tense and relaxed expressions by combining the output of an existing pattern recognition system, the facial expression recognizer in this invention may be configured in such a way.

[0143] Then, using the training dataset prepared as described above, a pre-training process is performed to construct a facial expression recognition model, which is then stored in the facial expression recognition model storage means 27B. The facial image data used for training may be video data or still image data, but it must be the same format (number of dimensions) as the input data to be input to the facial expression recognition classifier 27 during the operational phase. Therefore, when the input data to the facial expression recognition classifier 27 is video data (see Figure 11), video data of the same duration as the input data during the operational phase is prepared as training data, and the training process is performed.

[0144] During the operational phase, as shown in Figure 5, facial image data (video) or facial image data (still images) from multiple different time points in time (see Figure 11) between the presentation time P1 of a problem (task) currently being trained and the answer time (action time) P2 are input to the facial expression classifier 27 (facial expression classifier 27A) as input data. However, multiple facial image data (video or still images) are not input simultaneously, but rather at different times, i.e., sequentially. Therefore, the same number of outputs as the input facial image data (video or still images) are obtained from the facial expression classifier 27 (facial expression classifier 27A).

[0145] When creating a video from facial image data, as shown in Figure 11, multiple facial image data (videos) are created by extracting facial image data of a fixed length from the facial image data between the time P1 when the problem (task) is presented and the time P2 when the solution (action) is answered. Preferably, each extracted section is a shifted section that partially overlaps, as shown in Figure 11, but it may also be extracted without overlapping. If the sections overlap, a large number of facial image data (videos) can be created even if the time length from time P1 to time P2 is short.

[0146] Furthermore, the position of time point P2 shifts depending on whether the problem is answered (the task is addressed) quickly or not, so the time elapsed from time point P1 to time point P2 varies from problem to problem (task to task), and there are problem types (task types) with relatively short standard time limits T1. Therefore, the length of a single extraction interval cannot be set to a very long time. For example, if there is a problem type (such as a multiple-choice question) with a standard time limit T1 of 7 seconds, setting the extraction interval length to 5 seconds would result in a situation where no face image data (video) is created if the answer is given in 4 seconds. However, if a certain length of input data cannot be obtained, input data interpolation may be performed.

[0147] Furthermore, as shown in Figure 5, the facial expression recognition means 27A performs pattern recognition processing using sequentially input facial image data (video or still images), sequentially outputs preliminary identification results, and then performs a process to integrate the obtained preliminary identification results and output the final identification result. This integration includes (integration method 1) and (integration method 2).

[0148] In (integration method 1), the average value of multiple likelihoods output as preliminary identification results is calculated, and by comparing the calculated average value with a predetermined threshold, it is determined whether the expression is tense or relaxed, and this is used as the final identification result.

[0149] In (integration method 2), the likelihood of each output is compared with a predetermined threshold α to determine whether the expression is tense or relaxed, and multiple preliminary identification results are output. Of these multiple preliminary identification results, if a certain percentage (above or below the predetermined threshold β) is a tense expression, the final identification result is determined to be a tense expression, and if a certain percentage (above or below the threshold β) is a relaxed expression, the final identification result is determined to be a relaxed expression.

[0150] The final identification result output from the facial expression recognition means 27A is then stored in the task-unit identification result table of the identification result storage means 34 (see Figure 5), associated with user identification information, problem type number (task type identification information), section number (task group identification information), and problem number (task identification information).

[0151] Specifically, in the case of two-class classification, whether an expression is tense or relaxed is the same as whether it is tense or not, or whether it is relaxed or not. Therefore, the likelihood λkin, which indicates the probability of a tense expression (i.e., not a relaxed expression), and the likelihood λyoy, which indicates the probability of a relaxed expression (i.e., not a tense expression), are output, and λkin + λyoy = 1. In the case of three or more-class classification, the likelihood λkin, which indicates the probability of a tense expression, the likelihood λyoy, which indicates the probability of a relaxed expression, and the likelihood λoth, which indicates the probability of any other expression, are output, and λkin + λyoy + λoth = 1.

[0152] In this case, (integration method 1) outputs multiple (for example, three) λkin(1)=0.86, λkin(2)=0.78, and λkin(3)=0.91. Their average value is calculated as (0.86 + 0.78 + 0.91) / 3 = 0.85. This average value of 0.85 is compared with the threshold of 0.8. Since the average value is greater than or equal to the threshold, the final classification result is determined to be a tense expression. In addition, in two-class classification, λyoy(1)=0.14, λyoy(2)=0.22, and λyoy(3)=0.09 are output simultaneously. Their average value is calculated as (0.14 + 0.22 + 0.09) / 3 = 0.15. Since this average value of 0.15 is less than the threshold of 0.8, the final classification result is determined not to be a relaxed expression, which is equivalent to determining it to be a tense expression. Therefore, in two-class classification, calculating either the mean value of λkin or the mean value of λyoy will yield the same final classification result, so it is sufficient to calculate either one. On the other hand, in classification of three or more classes, even if the final classification result using λkin to determine whether or not the expression is tense, this does not mean that the expression is relaxed. Therefore, the final classification result using λyoy to determine whether or not the expression is relaxed is also output.

[0153] Furthermore, in (integration method 2), if multiple (for example, three) λkin(1)=0.86, λkin(2)=0.78, and λkin(3)=0.91 are output, and each of these λkin values ​​is compared with the threshold α=0.8, λkin(1) is judged to be "tense expression" because it is 0.8 or higher, λkin(2) is judged to be "not tense expression" because it is less than 0.8, and λkin(3) is judged to be "tense expression" because it is 0.8 or higher. These become the multiple (three) preliminary identification results. Therefore, the proportion of "tense expression" among these three preliminary identification results is 2 / 3 = 66.7%, and since the threshold β = 80% is lower, it is judged to be "not tense," and this becomes the final identification result using λkin to determine whether or not it is a "tense expression." Similarly, multiple (three) preliminary identification results are output for λyoy, and the final identification result using λyoy to determine whether or not it is a "relaxed expression" is also output.

[0154] <Configuration of training management server 20 / training task storage means 31: Figure 2>

[0155] As shown in Figure 2, the training task memory means 31 stores the following information in association with each other: the problem type number (task type identification information), the section number (task group identification information), the problem number (task identification information), the path indicating the storage location of the audio data for reading aloud, which is part of the problem data (task data), the answer choices (choices for dealing with the problem, which are candidates for the outcome data), the correct answer data (correct outcome data), and the standard value of the time limit, which are also part of the problem data (task data). Although not shown in the illustration, the system also stores the number and path indicating the storage location of the layout data for the problem presentation screen (task presentation screen) as part of the problem data (task data).

[0156] The example in Figure 2 is for a multiple-choice question. The question data (task data) to be stored differs depending on the question type (task type). For example, in the case of a sentence rearrangement problem (a problem in which you rearrange randomly provided words to create a correct sentence), the word data to be rearranged and the correct answer data (correct solution result data) showing the correct order of those words are stored as question data (task data).

[0157] Furthermore, in the example in Figure 2, the presented task is a problem, so the correct answer data (data for the correct response) is stored. However, if the presented task is not a problem, the expression of whether the response is correct or incorrect (correct / incorrect, or right / wrong) may not be applicable. For example, in a game, for tasks such as dodging bullets fired by enemies, intercepting missiles fired by enemies, a person passing through a gate, or a person or horse jumping over hurdles, whether the task was completed or not, due to being distracted by something else, is expressed as whether the task was completed or not, rather than whether the response was correct or not. Therefore, in such cases, the training task storage means 31 will store data indicating the conditions for completing the task (for example, data for when bullets are dodged, missiles are intercepted, a person passes through a gate, or a person or horse jumps over hurdles) instead of the correct answer data. In short, combining whether the response was correct or not with whether the task was accomplished or not results in the expression of whether the response to the task was successful or not (success or failure). More generally, the training task memory means 31 only needs to store data for cases where the response was successful.

[0158] <Configuration of training management server 20 / training response result storage means 32: Figure 3>

[0159] As shown in Figure 3, the training response result storage means 32 includes a task-unit response result table that stores response result data for each task (problem), and an aggregate result table that stores training aggregate result data aggregated and analyzed for each section (task group).

[0160] The task-unit response result table stores the following information in association with each other: problem type number (task type identification information), section number (task group identification information), problem number (task identification information), user identification information, section implementation date and time (start date and time), correctness judgment result of the answer (judgment result of whether the response was correct or incorrect, or whether the task was completed or not, more generally, judgment result of whether the task was successfully addressed or not), time limit used (time limit T1 in Figure 11), time taken to answer (time taken to address, elapsed time T2 in Figure 11), and remaining time (remaining time T3 at the time of addressing P2 in Figure 11). In addition, the presentation date and time of each task (each problem) is also stored.

[0161] The summary results table stores the following information in association with the problem type number (task type identification information), section number (task group identification information), number of problems in the section (number of tasks), user identification information, the date and time the section was conducted (start date and time), the number of correct answers (number of correct responses or number of tasks completed, more generally the number of successful responses to the tasks), the accuracy rate (percentage of correct responses or task completion rates, more generally the percentage of successful responses to the tasks), and the time limit used. Since the date and time the section was conducted (start date and time) is stored, if training for the same section is conducted multiple times, for example, if the performance in the first training session for a particular section was poor and a second training session is conducted for that section, the performance data for those multiple sessions can be stored separately.

[0162] <Configuration of training management server 20 / training timer setting information storage means 33: Figure 4>

[0163] As shown in Figure 4, the training timer setting information storage means 33 stores user identification information, problem type number (task type identification information), current time limit, previous time limit, time limit two sessions prior, and current selected mode (the training mode selected on the training mode selection screen 80 in Figure 6) in association with each other.

[0164] <Configuration of training management server 20 / identification result storage means 34: Figure 5>

[0165] As shown in Figure 5, the identification result storage means 34 includes a task-unit identification result table that stores the facial expression identification results for each task (problem), and a comprehensive identification result table that stores the facial expression identification results obtained for each section (group of tasks).

[0166] The task unit identification result table stores user identification information, problem type number (task type identification information), section number (task group identification information), problem number (task identification information), and facial expression identification result (tense expression, relaxed expression, other) in association with each other.

[0167] The comprehensive identification result table stores user identification information, problem type number (task type identification information), section number (task group identification information), and facial expression identification result (tense expression, relaxed expression, other) in association with each other.

[0168] <Configuration of Training Management Server 20 / Point Storage Device 35>

[0169] The point storage means 35 stores the quantity of points held by the user, associating it with user identification information.

[0170] <Configuration of Test Management Server 50 / Login Method 51>

[0171] The login means 51 receives the user ID and password transmitted from the user terminal 70 via the network 2 during testing (not in the sense of being in the middle of testing, but in the sense of testing in relation to training), or in addition to these, receives facial image data via the facial image acquisition means 56, transmits this authentication information to the authentication server 40 via the network 2, receives the authentication result from the authentication server 40, and if authentication fails, executes the process of sending notification data to the user terminal 70 to that effect.

[0172] Furthermore, during testing, it is preferable to perform authentication using facial image data not only when logging in using the login means 51, but also while the user is performing the test (this may be done multiple times at time intervals, or continuously) to prevent fraud such as impersonation or substitution by others. Accordingly, the test management server 50 is equipped with a monitoring means 57, which receives the user's facial image data taken during the test via the network 2 using the facial image acquisition means 56, and the monitoring means 57 receives the facial image data of the user during the test from the facial image acquisition means 56 and sends it to the authentication server 40, and uses the authentication result received from the authentication server 40 to determine whether or not there has been any fraudulent activity such as impersonation.

[0173] However, for rigorous tests such as certification exams, it is preferable to perform authentication using facial image data both during login and during the test, as described above. However, since some tests are not rigorous, facial image data authentication may be performed only during login, or it may be omitted both during login and during the test.

[0174] The facial image data used for authentication is captured by the camera 75 of the user terminal 70, transmitted from the user terminal 70 to the test management server 50 via the network 2 by the processing unit 72, received by the facial image acquisition unit 56, and passed from the facial image acquisition unit 56 to the login unit 51.

[0175] The face image data (video) continuously transmitted from the processing means 72 of the user terminal 70 may be processed by the face image acquisition means 56 of the test management server 50 into one or more still image data necessary for authentication processing at the authentication server 40, or the processing may be done on the processing means 72 of the user terminal 70 and received by the face image acquisition means 56.

[0176] Furthermore, if human monitoring (online test proctors) of facial images (videos) is to be performed during the test to prevent cheating (for example, monitoring as shown in Figure 6 of Patent No. 7530630 by the applicant of this application), the facial image data (video) transmitted via streaming from the processing means 72 may be received by the facial image acquisition means 56, and the video may be used directly for monitoring. In this case, the monitoring means 57 should be provided with a function to display the facial image of the user taking the test on the screen of the online test proctor's terminal (not shown).

[0177] <Configuration of Test Management Server 50 / Test Assignment Presentation Means 52>

[0178] The test task presentation means 52, during testing, presents the user with a timed task stored in the test task storage means 61 (see Figure 2) on the screen of the user terminal 70, and also displays a timer on the screen of the user terminal 70 showing the remaining time or elapsed time until the time limit given to the user to complete the task, for the user who is working on the task. This test task presentation means 52 corresponds to the training task presentation means 22 of the training management server 20, and the format of task presentation (problem presentation) is the same as that of the training task presentation means 22. However, since this is a relationship between training and testing, it does not mean that the task data (problem data) to be presented is the same, but rather that the format of presentation is the same. Therefore, the content of the task data (problem data) and the order of presentation (order of questions) may be different.

[0179] Furthermore, the test task presentation means 52 processes the selection of a test mode by the user. Specifically, the test task presentation means 52 transmits the display data (Web page data) for the test mode selection screen 90 shown in Figure 7 to the user terminal 70 via the network 2, and receives the user's test mode selection information transmitted from the user terminal 70 via the network 2. The received selection information is associated with user identification information and stored in the test timer setting information storage means 63 (see Figure 4). This process itself is the same as that of the training task presentation means 22, but as shown in Figure 7, the test mode has different modes available as options from the training mode (see Figure 6) that is selected by the training task presentation means 22.

[0180] In Figure 7, the test mode selection screen 90 is provided with a selection unit 91 for (Tst1) self-configuration mode and a selection unit 92 for (Tst2) standard mode. Tst is an abbreviation for test.

[0181] (Tst1) Self-Setting Mode is a mode in which the user sets the timer limit for the test themselves. When this (Tst1) Self-Setting Mode is selected, the acceptance of the user's input for setting the time limit is performed by the timer setting means 54, so the test task presentation means 52 only accepts the mode selection. The reason why a self-setting mode is provided even though it is a test is as detailed in [Means for Solving the Task], and is because in some tests, a higher score is given to those who deal with many tasks (solve many problems) within a certain time (naturally, the time spent on each task must be shortened), in some tests, a higher score is given to those who choose a shorter time limit, and in other cases, the test may want to finish the test quickly and make effective use of the time afterward. The act of the user setting the time limit themselves in a test leads to a change in the difficulty of the test, so it can be said to be equivalent to, for example, when there are tests of different difficulty levels such as Grade 1, Grade 2, and Grade 3 (Grade 1 is the most difficult test), the act of choosing the grade to take the test.

[0182] □(Tst2) Standard mode is a mode in which the timer limit during testing is set to the standard value set by the system (stored in the test task storage means 61 as shown in Figure 2).

[0183] Furthermore, the test mode selection screen 90 in Figure 7 includes, when the (Tst1) self-configuration mode is selected, a selection unit 93 that selects to automatically set the timer limit at the start of the test (when the first section is executed) if a time limit has been received from the training management server 20 (TstA), and a selection unit 94 that selects to display the received time limit as reference information on the screen of the user terminal 70 if a time limit has been received from the training management server 20 (TstB), and allow the user to set the time limit themselves based on that information (since the received time limit is reference information, a different time limit may be set).

[0184] When the selection unit 93 of (TstA) is selected, the (Tst1) self-setting mode is selected, but only at the start of the test (when the first section is executed), automatic setting will be performed instead of self-setting. This automatic setting process is performed by the timer setting means 54.

[0185] Furthermore, the test mode selection screen 90 in Figure 7 is provided with a "Send" button 95. When the user presses the "Send" button 95 after making a selection in the selection units 91 to 94, the test mode selection information is transmitted from the user terminal 70 to the test management server 50 via the network 2, received by the test task presentation means 52, and stored in the test timer setting information storage means 63 (see Figure 4).

[0186] <Configuration of Test Management Server 50 / Test Response Result Determination Means 53>

[0187] The test response result determination means 53 receives the user's input means 73 on the user terminal 70 during the test, which is performed as a test to address a task (answer a problem), and makes a judgment including evaluation or aggregation to obtain a response result, including whether the response to the task (answer to the problem) was correct or whether the task was achieved or not, and then executes a process to store the obtained user response result data (various information derived from objective judgment from data showing the content of the user's response received from the user terminal 70, including not only information on whether the response was correct or whether the task was achieved or not, but also test aggregation result data obtained by aggregation and analysis) in the test response result storage means 62 (see Figure 3). This test response result determination means 53 corresponds to the training response result determination means 23 of the training management server 20, and the format for processing data showing the content of the user's response to the presented task (problem) (answer data) is the same as that of the training response result determination means 23.

[0188] <Configuration of Test Management Server 50 / Timer Setting Means 54>

[0189] (When (Tst1) Self-configuration mode is selected) When the user performing the test (Tst1) has selected the self-setting mode (see Figure 7), the timer setting means 54 transmits display data (Web page data) for a self-setting time limit screen (not shown) or a confirmation screen (not shown) regarding whether or not to change the time limit setting to the user terminal 70 via the network 2 at the start and / or end of a section (group of tasks), and also receives the self-set time limit transmitted from the user terminal 70 via the network 2, and performs a process to store the received self-set time limit in the test timer setting information storage means 63 (see Figure 4) in association with the user identification information and the problem type number (task type identification information) of the section.

[0190] (When standard mode (Tst2) is selected) Furthermore, if the user performing the test has selected the standard mode (Tst2) (see Figure 7), the timer setting means 54 executes a process to set the timer limit for the test to the standard value set by the system (stored in the test task storage means 61 as shown in Figure 2). Accordingly, the standard value of the time limit is stored in the "Current Time Limit" column of the test timer setting information storage means 63 (see Figure 4).

[0191] (If (Tst1) Self-configuration mode is selected, and (TstA) Automatically setting the time limit received from the training management server 20 as the time limit at the start of the test is selected.) Furthermore, if the user performing the test (Tst1) selects self-setting mode (see Figure 7) and (TstA) selects to automatically set the time limit received from the training management server 20 as the time limit at the start of the test (see Figure 7), the timer setting means 54 sends a request to the training management server 20 via network 2 to obtain the current time limit of the user used in training, along with the user identification information and the problem type number (task type identification information). In response to this request, the training management server 20 receives the current time limit of the user transmitted via network 2 along with the user identification information and the problem type number (task type identification information) from the test management server cooperation data transmission means 25, and uses the received current time limit for training to automatically set the time limit at the start of the test when measuring the user's ability to handle the task. Accordingly, the current time limit of the user received from the training management server 20 is stored in the "Current Time Limit" column of the test timer setting information storage means 63 (see Figure 4), associated with the user identification information and the problem type number (task type identification information) of the user being tested.

[0192] (When (Tst1) Self-configuration mode is selected, and (TstB) Displaying the time limit received from the training management server 20 as reference information for self-configuration is selected.) Furthermore, if the user performing the test has selected (Tst1) self-setting mode (see Figure 7) and (TstB) display the time limit received from the training management server 20 as reference information and the user sets the time limit at the start of the test themselves (see Figure 7), the timer setting means 54 sends a request to the training management server 20 via network 2 to obtain the current time limit used by that user in the training, along with the user identification information and the problem type number (task type identification information). In response to this request, the training management server 20's data transmission means 25 for the test management server transmits the user identification information and the problem type number (task type identification information) via network 2. The system receives the user's current time limit, and transmits the received data for displaying the current training time limit (Web page data) to the user terminal 70 via network 2, displaying it on the user terminal 70's screen as reference information. Subsequently, it receives the user's self-configured test time limit transmitted from the user terminal 70 via network 2 (the current training time limit transmitted to the user terminal 70 is merely reference information and does not need to be the same as that reference information). The system then stores the received self-configured test time limit as the test start time, associating it with the user identification information and the problem type number of that section (task type identification information), in the test timer setting information storage means 63 (see Figure 4).

[0193] <Configuration of the data transmission means 55 for the test management server 50 and training management server>

[0194] The data transmission means 55 for the training management server receives a request to acquire test summary result data transmitted via network 2 from the timer adjustment means 24 of the training management server 20, along with user identification information and problem type number (task type identification information) for the user performing the training. In response to this acquisition request, the means executes the process of transmitting the test summary result data stored in the test response result storage means 62 (see Figure 3) to the training management server 20 via network 2, along with the user identification information and problem type number (task type identification information) for the user in question. The test summary result data to be transmitted includes, for example, the correct answer rate for each section (the rate at which the response was correct, or the task completion rate, and more generally, the success rate of the response) and the time limit used for each section, stored in the summary result table of the test response result storage means 62 (see Figure 3).

[0195] <Configuration of Test Management Server 50 / Face Image Acquisition Means 56>

[0196] The facial image acquisition means 56 acquires the user's facial image data captured by the shooting means 75 of the user terminal 70 via the network 2. When logging in to perform a test, it passes the acquired facial image data to the login means 51 as user authentication information. Subsequently, when the test starts, it performs the process of passing the acquired facial image data of the user during the test to the monitoring means 57 as authentication information to check for fraud such as impersonation or mid-test substitution, or as information for monitoring facial images (videos) by a human (online test proctor).

[0197] <Configuration of Test Management Server 50 / Monitoring Device 57>

[0198] The monitoring means 57 receives facial image data of the user being tested from the facial image acquisition means 56, transmits the facial image data to the authentication server 40 via the network 2, receives the authentication result transmitted from the authentication server 40, uses the received authentication result to determine whether or not there is any fraudulent activity such as impersonation, and stores the determination result in association with user identification information in the task-unit handling result table of the test handling result storage means 62 (see Figure 3) on a task (problem) basis.

[0199] The monitoring means 57 may also have a function to receive face image data (video) streamed from the user terminal 70 via the network 2 from the face image acquisition means 56, and to transmit the received video data as is to the terminal (not shown) of the online exam supervisor for screen display.

[0200] <Configuration of Test Management Server 50 / Test Task Storage Means 61: Figure 2>

[0201] As shown in Figure 2, the test task storage means 61 has a configuration similar to the training task storage means 31 of the training management server 20, which has already been described in detail. However, it is preferable that the content of the problem data (task data) and the order in which the questions are presented differ between training and testing, although the content may be the same depending on the type of test. In other words, since it is a test, even if the content of the problems (tasks) is the same, it is common to present the problems (tasks) in a different order than during training, or to slightly change the content of the questions.

[0202] <Configuration of Test Management Server 50 / Test Response Result Storage Means 62: Figure 3>

[0203] As shown in Figure 3, the test handling result storage means 62 has a configuration similar to the training handling result storage means 32 of the training management server 20, which has already been described in detail.

[0204] <Configuration of test management server 50 / test timer setting information storage means 63: Figure 4>

[0205] As shown in Figure 4, the timer setting information storage means 63 for testing has a configuration that is substantially the same as the timer setting information storage means 33 for training of the training management server 20, which has already been described in detail, the only difference being that the current selected mode is test mode instead of training mode.

[0206] <Processing flow during training: Figures 8 and 9>

[0207] In this embodiment, training is performed by the training system 10 as follows.

[0208] In Figure 8, the user performing the training operates the input means 73 on the user terminal 70 to enter their user ID and password, and then takes a picture of their face using the shooting means 75. The user then transmits their user ID, password, and face image data to the training management server 20 via the network 2 (step S1).

[0209] The training management server 20 receives the user ID and password, as well as facial image data, transmitted from the user terminal 70 via the login means 21, and transmits this authentication information to the authentication server 40 via the network 2. The authentication server 40 receives the user ID and password, as well as facial image data, via the authentication means 41, performs authentication processing using the registration information stored in the authentication information storage means 42, and transmits the authentication result to the training management server 20 via the network 2. The training management server 20 receives the authentication result from the authentication server 40 via the login means 21, and if authentication fails, transmits a message to that effect to the user terminal 70 via the network 2 (step S2).

[0210] Then, the training mode selection screen 80 shown in Figure 6 is displayed on the user terminal 70 by the training task presentation means 22 of the training management server 20. The user selects a training mode on this screen 80 and presses the "Send" button 86 to send the training mode selection information to the training management server 20 via the network 2 (step S3).

[0211] The training management server 20 receives training mode selection information from the user via the network 2 from the user terminal 70 using the training task presentation means 22, and processes the received selection information with user identification information and problem type number (task type). The training timer setting information is stored in the training timer setting information storage means 33 (see Figure 4) in association with the identification information (step S4). Note that the training problem type (task type) is selected by the user on a screen different from Figure 6.

[0212] Furthermore, when (Trg1) simple automatic adjustment mode or (Trg2) complex automatic adjustment mode is selected (see Figure 6), and (TrgB) automatic setting of the timer limit according to the test performance using the test summary result data (linked data) received from the test management server 50 is selected (when the user selects selection section 85 in Figure 6), the timer adjustment means 24 acquires the test summary result data (linked data) from the test management server 50, determines the initial value of the timer limit according to the test performance, and saves it in the "current time limit" column of the training timer setting information storage means 33 (see Figure 4) (step S4). Note that the process of determining the time limit according to the test performance is described in detail in the description of the configuration of the timer adjustment means 24, so a detailed explanation is omitted here.

[0213] Next, the training task presentation means 22 of the training management server 20 transmits task data with a time limit (here, problem data, section number = 04, problem number = 02) related to the user's selection stored in the training task storage means 31 (see Figure 2) to the user terminal 70 via the network 2 (step S5). Note that the next problem will have a number that increases by 1 within the same section, so the section number will be 04 and the problem number will be 03.

[0214] Then, a problem presentation screen 100, which is a task presentation screen as shown in Figure 9, is displayed on the screen of the user terminal 70 (step S6). In Figure 9, the problem presentation screen 100 is provided with display units 101 and 102 for the problem number and section number, a selection unit 103, 104 and 105 for selecting one of three answers, a display unit 106 for displaying the progress of the problems in the section (that the second problem out of 33 problems is being presented), and a display unit 107 that displays a timer showing the remaining time at the current time (T3 as shown in Figure 11).

[0215] When the problem presentation screen 100 shown in Figure 9 is displayed, the problem statement (e.g., "bag") is read aloud in a language other than the user's native language (the language to be learned, for example, English). The user listens to the audio and considers the corresponding native language (Japanese) for the problem statement (e.g., "bag"). That is, they consider which of the three options corresponds to their native language.

[0216] While the user is thinking, the timer display on the display unit 107 decreases by one second at a time from the currently set time limit (for example, 7 seconds) (step S7). That is, it decreases as follows: 7, 6, 5, 4, ... Figure 9 shows an example of the moment when it is 5 seconds.

[0217] Then, when the user performs a selection operation for the answer, such as choosing their native language (Japanese) corresponding to the spoken question (for example, "bag"), by clicking on one of the three selection units 103, 104, or 105, the answer data related to the user's selection (data indicating which of the three options was selected, or more generally, data indicating the content of the action taken) is transmitted to the training management server 20 via the network 2 (step S8). In addition, facial image data (video or still image) of the user while they are thinking is captured by the shooting means 75, and this facial image data is also transmitted to the training management server 20 via the network 2 (step S8).

[0218] The training management server 20 receives data indicating the content of the action transmitted from the user terminal 70 via the network 2 using the training action result determination means 23. The server compares the received data indicating the content of the action with the correct answer data (data for successful actions) stored in the training task storage means 31 (see Figure 2), determines whether they match, and stores the result of this determination (correct if they match, incorrect if they do not) as user action result data, associated with user identification information, in the task-unit action result table of the training action result storage means 32 in Figure 3 (step S9). In this case, if no selection operation (click operation) is performed within the time limit, it is treated as an incorrect answer as shown in Figure 9, and the incorrect answer due to time expiring is stored as user action result data in the task-unit action result table of the training action result storage means 32 in Figure 3.

[0219] Furthermore, the facial image acquisition means 26 receives facial image data of the user dealing with a task (problem) transmitted from the user terminal 70 via the network 2, inputs this facial image data of the user dealing with the task into the facial expression recognition unit 27, and the facial expression recognition means 27A outputs a facial expression recognition result for each task (problem) (as shown in Figure 5, the final recognition result obtained by integrating multiple preliminary recognition results), and stores this facial expression recognition result in the task-unit identification result table of the identification result storage means 34 in Figure 5 (step S9).

[0220] Next, the training response result determination means 23 determines whether the task (problem) for which data indicating the content of the response (answer data) has been received from the user terminal 70 is the last task in the section (group of tasks), that is, whether the section has ended (step S10). If it is not the last task, that is, if the section has not ended, the process returns to step S5 (processing by the training task presentation means 22) described above in order to present the next task (problem). At this time, as shown in Figure 9, if the answer is correct, the screen proceeds to the next problem. If the answer is incorrect or if it is treated as incorrect due to time running out, the correct answer is displayed along with a "Proceed to next problem" button. When the "Proceed to next problem" button is pressed (clicked), the screen proceeds to the next problem.

[0221] On the other hand, when a section is completed, the training response result determination means 23 uses the problem unit (task unit) response result data stored in the task unit response result table of the training response result storage means 32 in Figure 3 to perform section aggregation and analysis processing. The obtained training aggregation result data (number of correct answers and correct answer rate in the section) is then stored in the aggregation result table of the training response result storage means 32 in Figure 3 as user response result data, associated with user identification information and section number (step S11).

[0222] Furthermore, the training response result determination means 23 performs a process to convert the facial expression recognition results obtained by the facial expression recognizer 27 on a problem-by-problem basis (task-by-task basis) into section-by-section basis (task group basis), in addition to the aggregation and analysis processing of the above sections. That is, using the facial expression recognition results (tense expression, relaxed expression, other) for each problem-by-problem basis (task-by-task basis) stored in the task-by-task identification result table of the identification result storage means 34 in Figure 5, the facial expression recognition result (overall) for each section is obtained, and the obtained facial expression recognition result (overall) is stored in the overall identification result table of the identification result storage means 34 in Figure 5, associated with the user identification information and the section number (step S11). Note that the conversion process to section basis has already been described in detail in the explanation of the configuration of the training response result determination means 23, so a detailed explanation is omitted here.

[0223] Furthermore, if (Trg1) Simple Automatic Adjustment Mode (see Figure 6) is selected, the timer adjustment means 24 uses the response results obtained by the training response result determination means 23, that is, the training response result data (including training summary result data) stored in the training response result storage means 32 (see Figure 3), to determine whether the time limit reduction condition is met. If the time limit reduction condition is met, the time limit reduction process is executed. The timer adjustment means 24 also determines whether the time limit extension condition is met, and if the time limit extension condition is met, the time limit extension process is executed. The adjusted time limit is then saved in the "Current Time Limit" column of the training timer setting information storage means 33 (see Figure 4) (step S11). Note that the time limit reduction condition, time limit reduction process, time limit extension condition, and time limit extension process (see Figures 13 and 14) have already been described in detail in the explanation of the configuration of the timer adjustment means 24, so a detailed explanation is omitted here.

[0224] Furthermore, if (Trg2) Combined Automatic Adjustment Mode (see Figure 6) is selected, the Timer Adjustment Means 24 performs a process to adjust the time limit used for timer display (relaxed expression adjustment process, tense expression adjustment process) using the response result obtained by the training response result determination means 23, i.e., the training response result data (including training summary result data) stored in the training response result storage means 32 (see Figure 3), and the identification result by the facial expression identification means 27A (however, the comprehensive identification result converted into section units by the training response result determination means 23), i.e., the facial expression identification result (tense expression or relaxed expression) stored in the comprehensive identification result table of the identification result storage means 34 (see Figure 5). The adjusted time limit is then saved in the "Current Time Limit" column of the training timer setting information storage means 33 (see Figure 4) (Step S11). Note that the relaxed expression adjustment process and tense expression adjustment process have already been described in detail in the description of the configuration of the Timer Adjustment Means 24, so a detailed explanation is omitted here.

[0225] Then, when (Trg3) Self-Setting Mode (see Figure 6) is selected, the Timer Adjustment Means 24, when it determines that certain conditions are met, displays an advice message to the user recommending that they shorten the time limit. That is, it transmits the advice display data to the user terminal 70 via the network 2, and receives the changed time limit transmitted from the user terminal 70 by the user who has referred to this advice display, and stores it in the "Current Time Limit" column of the Training Timer Setting Information Storage Means 33 (see Figure 4) (Step S11). The determination that certain conditions are met is when the system determines that the time limit shortening conditions are met using the response results obtained by the training response result determination means 23, that is, using the training response result data (including training summary result data) stored in the training response result storage means 32 (see Figure 3), and the facial expression recognition means 27A outputs an identification result indicating a relaxed expression (however, this is the overall identification result converted to section units by the training response result determination means 23).

[0226] Furthermore, when (Trg3) Self-Setting Mode (see Figure 6) is selected, if the timer adjustment means 24 of the training management server 20 sends advice display data recommending shortening the time limit to the user terminal 70 via the network 2, an advice display to that effect will be shown on the screen of the user terminal 70. The user will refer to this advice display, and if they decide to shorten the time limit, they will perform an input operation to change the time limit setting and send the changed time limit from the user terminal 70 to the training management server 20 via the network 2 (step S12).

[0227] Subsequently, if the user chooses to end the training, the training process is terminated. If the user chooses to continue the training, the process returns to step S5 (step S13) to begin presenting the problem (task) for the next training section according to the user's instructions.

[0228] <Processing flow by timer adjustment means 24, Figure 12>

[0229] To describe the processes in steps S11 and S12 mentioned above in more detail, the process flow is as shown in Figure 12. Here, we will mainly describe in detail the branching of the process by the timer adjustment means 24.

[0230] In Figure 12, when a section is completed (see step S10 in Figure 8), the training response result determination means 23 performs aggregation and analysis processing of the section, and stores the obtained training aggregation result data (number of correct answers and accuracy rate in the section) in the aggregation result table of the training response result storage means 32 in Figure 3 (step S1101).

[0231] Next, the training response result determination means 23 converts the facial expression recognition results obtained by the facial expression recognition unit 27 on a problem-by-problem basis (task-by-task basis) into section-by-section basis (task-by-task group basis), and stores the resulting facial expression recognition results (overall) in the overall identification result table of the identification result storage means 34 in Figure 5 (step S1102).

[0232] Furthermore, the timer adjustment means 24 determines whether (Trg1) simple automatic adjustment mode (see Figure 6) is selected (step S1103). If (Trg1) simple automatic adjustment mode is not selected, it determines whether (Trg2) complex automatic adjustment mode (see Figure 6) is selected (step S1104). If (Trg2) complex automatic adjustment mode is selected, it determines whether the time limit reduction condition is met (step S1105). If the time limit reduction condition is met, it determines whether the facial expression recognition result (overall) is a relaxed expression (step S1106). If it is determined to be a relaxed expression, it executes the time limit reduction process and saves the changed time limit in the training timer setting information storage means 33 (see Figure 4) (step S1107). If it is determined not to be a relaxed expression, nothing is done (the time limit setting is not changed).

[0233] Furthermore, if the timer adjustment means 24 determines in step S1105 above that the time limit reduction condition is not met, it determines whether the time limit extension condition is met (step S1108). If the time limit extension condition is met, it determines whether the facial expression recognition result (overall) is a tense expression (step S1109). If it is determined to be a tense expression, it executes the time limit extension process and saves the changed time limit in the training timer setting information storage means 33 (see Figure 4) (step S1110). If it is determined not to be a tense expression, nothing is done (the time limit setting is not changed). On the other hand, if it is determined in step S1108 above that the time limit extension condition is not met, neither the time limit reduction condition nor the time limit extension condition is met, so nothing is done (the time limit setting is not changed).

[0234] Furthermore, if the timer adjustment means 24 determines that the (Trg1) simple automatic adjustment mode (see Figure 6) has been selected in step S1103, it determines whether the time limit reduction condition is met (step S1111). If the time limit reduction condition is met, it executes the time limit reduction process and saves the changed time limit in the training timer setting information storage means 33 (see Figure 4) (step S1112). On the other hand, if the time limit reduction condition is not met in step S1111, it determines whether the time limit extension condition is met (step S1113). If it determines that the time limit extension condition is met, it executes the time limit extension process and saves the changed time limit in the training timer setting information storage means 33 (see Figure 4) (step S1114). If it determines that the time limit extension condition is not met, it means that neither the time limit reduction condition nor the time limit extension condition is met, so nothing is done (the time limit setting is not changed).

[0235] Furthermore, if the timer adjustment means 24 determines that the (Trg2) composite automatic adjustment mode is not selected in step S1104 described above, then the (Trg3) self-setting mode (see Figure 6) is selected. If certain conditions are met, it sends advice display data recommending a reduction in the time limit to the user terminal 70 via the network 2 (step S1115). If the user sees this advice display and sends the modified time limit from the user terminal 70, the modified time limit is received and stored in the training timer setting information storage means 33 (see Figure 4) (step S1116).

[0236] <Processing flow during testing: Figure 10>

[0237] In this embodiment, the test is performed by the test system included in the training-test linkage system 1 as follows. The main differences from the processing by the training system 10 in Figure 8 described above will be explained.

[0238] In Figure 10, the processing in steps S21 to S24 during testing corresponds to the processing in steps S1 to S4 during training in Figure 8. The difference is that the mode selection is the selection of a test mode (see Figure 7) rather than the selection of a training mode (see Figure 6). Also, in the processing of step S4 during training in Figure 8, the timer adjustment means 24 acquires the test aggregation result data as linked data from the test management server 50, determines the initial setting value of the training time limit according to the test performance, and saves it. In contrast, in the processing of step S24 during testing in Figure 10, the timer setting means 54 acquires the user's own training time limit (the time limit used by the user performing the test during training) as linked data from the training management server 20, and saves it as the time limit to be used at the start of the test (when the first section is performed).

[0239] Furthermore, the processes in steps S25-S29 during testing in Figure 10 correspond to the processes in steps S5-S9 during training in Figure 8. In step S9 during training in Figure 8, facial image data of the user while working on the task is received from the user terminal 70, pattern recognition processing is performed by the facial expression recognition device 27 to identify facial expressions, and the identification result is stored in the identification result storage means 34 (see Figure 5). In contrast, in step S29 during testing in Figure 10, the facial image data of the user received from the user terminal 70 is not used for facial expression identification, but rather for authentication during testing to prevent fraud such as impersonation, and for monitoring by online test proctors.

[0240] Furthermore, the processes in steps S30 and S32 during the test in Figure 10 correspond to the processes in steps S10 and S13 during training in Figure 8. Also, the process in step S31 during the test in Figure 10 corresponds to the process in step S11 during training in Figure 8. However, in step S11 during training in Figure 8, the timer adjustment means 24 performs an automatic adjustment of the time limit, whereas in step S31 during the test in Figure 10, since it is a test and not training, the timer adjustment means 24 does not perform an automatic adjustment of the time limit. Furthermore, the processes during the test in Figure 10 do not correspond to the process in step S12 during training in Figure 8.

[0241] <Effects of this embodiment>

[0242] This embodiment provides the following advantages. Specifically, since the training system 10 is equipped with a timer adjustment means 24, it can use the results obtained from the judgment process by the training response result determination means 23 to determine whether or not the time limit reduction condition is met, and if this time limit reduction condition is met, it can execute a time limit reduction process (see Figure 13). It can also determine whether or not the time limit extension condition is met, and if this time limit extension condition is met, it can execute a time limit extension process (see Figure 14).

[0243] Therefore, training can be conducted in a way that is suitable for improving each user's problem-solving abilities, allowing for appropriate and efficient training for each user. In other words, if it is determined that the user is handling the task (answering the problem) with ease, the time limit can be automatically shortened, bringing them closer to a state where they are always handling the task (answering the problem) with tension, or in other words, a state where they are handling the task while making the most of their current problem-solving abilities. On the other hand, if it is determined that the user's problem-solving ability (language skills in the case of language problems) is not keeping up with the presentation of the task (problem), the time limit can be automatically extended, bringing the user closer to a state where they can fully utilize their problem-solving abilities.

[0244] Then, the timer adjustment means 24 uses the response results obtained from the judgment process by the training response result judgment means 23 to determine whether or not the time limit reduction condition is met, and if this time limit reduction condition is met, the time limit reduction process (see Figure 13) is executed. This avoids the inconveniences that occur when the user does not realize that they can respond faster (does not realize the high level of their current response ability) or when they do not actively try to improve their response speed, as in the rapid response training system described in Patent Document 1 mentioned above.

[0245] Furthermore, in the training system 10, when the (Trg2) combined automatic adjustment mode (see Figure 6) is selected, the timer adjustment means 24 can perform a combined time limit adjustment process using the response results obtained by the training response result determination means 23 and the identification results by the facial expression identification means 27A (however, the comprehensive identification results converted into section units by the training response result determination means 23). Therefore, it is possible to determine whether or not to shorten the time limit using the facial expressions of the user while they are dealing with the task (problem), so that information about the user that is closer to their true state can be captured and used in the time limit adjustment process by the timer adjustment means 24. As a result, the user can perform training more reliably and efficiently.

[0246] Furthermore, in the training system 10, by selecting (Trg3) Self-Setting Mode (see Figure 6), the user can freely set the time limit themselves. Then, when the timer adjustment means 24 determines that certain conditions have been met, an advice display recommending shortening the time limit is shown. The user can then refer to this advice display and change the time limit setting to perform training more reliably and efficiently.

[0247] Furthermore, the training outcome determination means 23 can award points to a user if the user sets a shorter time limit than the standard value (see Figure 2) and performs well in the training conducted under that self-set time limit. By informing users that they can earn points for completing tasks within a short time limit and performing well, users become aware that they can earn points when their problem-solving abilities (e.g., language skills) improve, increasing their motivation to improve their problem-solving abilities and boosting their motivation to train. Consequently, the speed of improvement in problem-solving abilities can be accelerated even further.

[0248] Furthermore, in the training-test linkage system 1, the training management server 20 transmits the user's training time limit as linkage data to the test management server 50. As shown in the selection section 93 for (TstA) and the selection section 94 for (TstB) in Figure 7, the user's training time limit received from the training management server 20 can be automatically set as the test start time limit used by the test management server 50, or it can be displayed on the screen of the user terminal 70 as reference information for the user to set the test start time limit themselves. Therefore, the test management server 50 can set an appropriate time limit using the linkage data.

[0249] Furthermore, in the training-test linkage system 1, test summary result data is transmitted as linkage data from the test management server 50 to the training management server 20. The training management server 20, using the timer adjustment means 24, can use the received test summary result data to set a shorter training start time for better test performance. As a result, users can perform training that is appropriate and efficient for them.

[0250] <Form of transformation>

[0251] It should be noted that the present invention is not limited to the embodiments described above, and any modifications that can achieve the objectives of the present invention are included within the scope of the present invention.

[0252] For example, in the above embodiment, the training system of the present invention was mainly described as a language learning system as well as a game system (in particular, a system that plays an achievement-type game in which the user progresses by clearing difficult scenes or situations (multiple stages) that appear in sequence). However, the present invention can be applied to any training system that improves the ability to deal with tasks for which setting a time limit is meaningful.

[0253] More specifically, in games such as shogi, go, and mahjong, where the objective is to defeat the opponent and time is irrelevant, setting a time limit is meaningless in such game systems. However, the present invention can be applied to training in various situations (ranging from relatively easy to difficult) where a time limit is set to practice how to move shogi pieces, place go stones, or discard mahjong tiles. In this case, although it is part of a software-based game system, it is training for individual practice before playing against an opponent, and can therefore be called a learning system or acquisition system for shogi, go, and mahjong.

[0254] Furthermore, in the case of systems for acquiring life skills such as cooking, sewing, flower arranging, and calligraphy, any system that presents questions to learn the knowledge necessary for these skills and requires users to answer them within a time limit can be called a question system. Also, any system that requires users to perform input operations corresponding to the assigned actions of cooking, sewing, flower arranging, or calligraphy within a time limit in a virtual space created by software can be called a training system for improving life skills to acquire those skills, and the present invention can be applied to both of these.

[0255] Furthermore, the same applies to vocational skill acquisition systems for learning professional skills such as surgical techniques, how to operate vehicles such as airplanes and ships, and how to operate heavy machinery such as cranes and forklifts; pet care learning systems for learning how to care for pets such as cats and dogs; and musical instrument playing learning systems for pianos and guitars. Any system that presents questions to learn the necessary knowledge and requires users to answer them within a time limit can be called a question system. Additionally, any system that allows users to perform input operations corresponding to the assigned actions such as surgical procedures, vehicle operation, heavy machinery operation, pet handling, and musical instrument playing within a time limit in a virtual space created by software can be called a training system for acquiring skills, and the present invention can be applied to all of these. [Industrial applicability]

[0256] As described above, the training system, training-test linkage system, and program of the present invention are suitable for use in, for example, language learning systems, game systems, life skills acquisition systems such as cooking, sewing, flower arranging, and calligraphy, occupational skills acquisition systems, pet care learning systems, musical instrument playing acquisition systems, and the like. [Explanation of Symbols]

[0257] 1. Training and Test Integration System 2 Network 20 Training Management Server 22. Methods for presenting training tasks 23. Method for determining the results of training response 24 Timer adjustment means 25. Data transmission method for the test management server 26. Method for acquiring facial images 27. Facial expression recognition device 27A Facial expression recognition means constituting the facial expression recognition device 27B Facial expression recognition model storage means constituting the facial expression recognition device 31 Training task memory means 32. Training Result Memory System 33. Timer setting information storage means for training. 34 Identification Result Storage Means 50 Test Management Servers 52. Methods for Presenting Test Assignments 53. Test Response Result Determination Method 54 Timer setting means 55. Data transmission method for the training management server 61. Test task memory means 62. Test Response Result Storage Method 63. Timer setting information storage means for testing 70 User terminals

Claims

1. A training system comprising a computer that provides training to improve the user's ability to deal with given tasks, The system includes a training management server that is connected to a user terminal operated by the user via a network and performs the process of managing the training, The aforementioned training management server A training task presentation means that presents the user with the time-limited task that the user must complete on the screen of the user terminal, and displays a timer on the screen of the user terminal showing the remaining time or elapsed time until the time limit given to the user to complete the task, while the user is working on the task. A training response result determination means that receives the user's operation of the user terminal's input means as training to address the aforementioned problem, and makes a determination including evaluation or aggregation to obtain a response result including whether the response to the problem was correct or whether the problem was achieved, This system includes a timer adjustment means that uses the results obtained from the training response result determination means to determine whether the response was correct or whether the percentage or number of consecutive tasks achieved, or other predetermined time limit reduction conditions that the response results must satisfy, meets, and if these time limit reduction conditions are met, executes a time limit reduction process to shorten the time used for the timer display; and also determines whether the percentage or number of consecutive tasks that were not achieved or other predetermined time limit extension conditions that the response results must satisfy, meets, and if these time limit extension conditions are met, executes a time limit extension process to extend the time used for the timer display. A training system characterized by the following features.

2. The aforementioned user terminal is The system includes a means for capturing an image of the user's face, The aforementioned training management server A face image acquisition means for acquiring face image data of the user in the process of dealing with the problem, captured by the shooting means of the user terminal, via a network; The facial expression recognition means includes a facial image acquisition means that inputs the user's facial image data acquired by this facial image acquisition means, and uses a facial expression recognition model that has been previously trained and stored in the facial expression recognition model storage means to identify whether the user's facial expression while dealing with the task is a tense expression indicating concentration and tension in dealing with the task, or a relaxed expression indicating a relaxed attitude in dealing with the task, and outputs this identification result, thereby performing a pattern recognition process. The timer adjustment means is Using the response result obtained by the training response result determination means and the identification result by the facial expression recognition means, a process is executed to adjust the time limit used for the timer display. In this case, if the judgment process by the training response result determination means determines that the response result obtained satisfies the predetermined conditions for shortening the time limit, and the facial expression recognition means outputs an identification result that indicates a relaxed expression, then the relaxed expression-incorporating process to shorten the time limit used for the timer display is executed. Alternatively, the system is configured to perform the relaxed expression processing, and if the decision processing by the training response result determination means determines that the response result obtained satisfies the predetermined conditions for extending the time limit, and the facial expression identification means outputs an identification result that indicates tension, then it performs the tension expression processing to extend the time limit used for the timer display. The training system according to feature 1.

3. The aforementioned training task presentation means is If the user selects the self-setting mode to set the time limit themselves, the system accepts the user's operation of the input means on the user terminal to set and input the time limit, and uses the accepted time limit to display the timer on the user terminal's screen. The timer adjustment means is If the user has selected the self-configuration mode, and the training response result determination means determines that the response result obtained satisfies the predetermined time limit reduction condition, and the facial expression recognition means outputs an identification result indicating a relaxed expression, the system is configured to display an advice message on the user terminal screen recommending that the user shorten the time limit used for the timer display. The training system according to feature 2.

4. The aforementioned training management server The system includes a point storage means that stores the number of points held by the user, associated with user identification information, which can be used to pay service fees for the training or other services provided to the user by the training management server operator or another company affiliated with the operator. The aforementioned training response result determination means is If the user has set a time limit shorter than the predetermined standard value, the system determines whether the numerical value of the data indicating the user's performance obtained by aggregating the results of the actions is equal to or greater than a predetermined threshold. If it is equal to or greater than the threshold, the system awards the user a predetermined number of points, or an amount of points corresponding to the difference between the standard value of the time limit and the time limit set by the user. The system then adds the number of points awarded to the number of points held by the user, which is stored in the point storage means in association with the user's user identification information. The training system according to feature 1.

5. A training and test integration system comprising a computer that performs training to improve a user's ability to deal with a given task, and tests to measure the user's ability to deal with the said task, through data linkage, The training system comprises a training management server that constitutes the training system according to any one of claims 1 to 4, and a test management server that is connected via a network to a user terminal operated by the user and performs processing to manage the test in the same format as the time-limited task presentation and response format by the training management server, The aforementioned training management server A timer setting information storage means stores the adjusted time limit set for each of the aforementioned users in association with user identification information, This system includes a data transmission means for sending data to the test management server, which transmits the time limit for the user stored in the timer setting information storage means, along with the user identification information of the user, to the test management server via the network. The aforementioned test management server is To address the aforementioned issues, the system executes a process that manages tests in which the user can set the time limit during testing. The system includes a timer setting means that either performs a process to automatically set the time limit at the start of the test when measuring the user's ability to handle the task, using the time limit of the user received from the data transmission means for the test management server along with the user identification information, or displays the time limit received from the data transmission means for the test management server along with the user identification information on the screen of the user terminal as reference information for the user to set the time limit at the start of the test, and performs a process to accept the user's operation of the user terminal for the user to set the time limit. A training and test integration system characterized by the following features.

6. A training and test integration system comprising a computer that performs training to improve a user's ability to deal with a given task, and tests to measure the user's ability to deal with the said task, through data linkage, The training system comprises a training management server that constitutes the training system according to any one of claims 1 to 4, and a test management server that is connected via a network to a user terminal operated by the user and performs processing to manage the test in the same format as the time-limited task presentation and response format by the training management server, The aforementioned test management server is A test response result storage means stores the results of how the user who took the test handled the tasks in the test, including test summary result data showing aggregated results obtained by aggregating the results of how multiple tasks were handled, in association with user identification information. This system includes a training management server data transmission means that transmits the test aggregation result data stored in the test handling result storage means, along with user identification information, to the training management server via the network. The timer adjustment means of the training management server is The system is configured to receive the test summary result data transmitted via the network by the data transmission means for the training management server, and to use the received test summary result data to set the time limit at the start of training to be shorter the better the test performance shown in the test summary result data. A training and test integration system characterized by the following features.

7. The training system according to any one of claims 1 to 4, characterized in that the aforementioned problem is a language learning problem, the user's ability to cope is language ability, and the training outcome determination means performs a process to determine whether or not the answer to the problem was correct.

8. A program for causing a computer to function as a training system according to any one of claims 1 to 4.

Images