State determination device and state determination program

Abstract

To provide a state determination device capable of determining a degree of a driver state suitable for driving from driver reaction to an instruction.SOLUTION: A state determination device includes: a task execution instruction unit (11) that outputs an execution instruction for causing a driver of a vehicle to execute a plurality of target tasks having different degree of difficulty selected from a plurality of candidate tasks having different degree of difficulty set based on task instruction conditions in order; a reaction information acquisition unit (12) that acquires reaction information to the execution instruction; an execution possibility determination unit (13) that determines whether or not a driver has been able to execute the target task according to the execution instruction based on the reaction information; a state determination unit (14) that determines a degree of normality that is a degree of suitability of a state of the driver for driving based on determination results of whether or not the driver has been able to execute the target task according to the execution instruction; and a state determination results output unit (15) that outputs information regarding the degree of normality determined by the state determination unit (14).SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a state determination device and a state determination program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a technique for determining the state of a driver of a vehicle from the driver's reaction to an instruction given to the driver, in order to perform some control related to the driving of the vehicle. For example, Patent Document 1 discloses a vehicle control device that outputs simple instructions to the driver (such as shaking the head from side to side) and detects the driver's responsiveness and alertness by observing the driver's reaction to the instructions, and determines whether or not to switch from automatic driving mode to manual driving mode based on the detected driver's responsiveness and alertness. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-21229 Summary of the Invention [Problem to be solved by the invention]

[0004] When some control relating to vehicle driving is performed in consideration of the driver's condition, it is preferable to change the content of the control depending on the degree to which the driver's condition is suitable for driving. In contrast, when only simple instructions are output, as in the technology disclosed in Patent Document 1, there is a problem in that even if the driver follows the instructions accurately, it is not possible to determine the degree to which the driver's condition is suitable for driving.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a state determination device that can determine the degree to which a driver's state is suitable for driving based on the driver's response to instructions. [Means for solving the problem]

[0006] The state determination device disclosed herein includes a task execution instruction unit that outputs execution instructions to a driver of a vehicle to sequentially execute a plurality of target tasks of varying difficulty levels selected from a plurality of candidate tasks of varying difficulty levels that have been set based on task instruction conditions; a reaction information acquisition unit that acquires reaction information regarding the driver's reaction to the execution instructions output by the task execution instruction unit; an execution feasibility determination unit that determines whether or not the driver was able to execute the target tasks in accordance with the execution instructions based on the reaction information acquired by the reaction information acquisition unit; a state determination unit that determines the normality, which is the degree to which the driver's state is suitable for driving, based on the determination result of whether or not the driver was able to execute the target tasks in accordance with the execution instructions determined by the execution feasibility determination unit; and a state determination result output unit that outputs information regarding the normality determined by the state determination unit. [Effects of the Invention]

[0007] According to the present disclosure, the state determination device is configured as described above, and is therefore able to determine the degree to which the driver's state is suitable for driving from the driver's response to instructions. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of the configuration of a state determination device according to a first embodiment. [Figure 2] FIG. 4 is a diagram illustrating an example of the contents of candidate task information stored in a storage unit in the first embodiment. [Figure 3] 2 is a diagram for explaining an example of a plurality of target tasks and their execution order selected and determined by a task execution instruction unit based on task instruction conditions in the first embodiment. FIG. [Figure 4] 10 is a diagram for explaining a specific example of determination of normality by the state determination unit in accordance with the state determination conditions based on the executability determination result determined by the executability determination unit in the first embodiment. FIG. [Figure 5]10 is a diagram for explaining another specific example of the determination of the normality by the state determination unit in accordance with the state determination conditions based on the executability determination result determined by the executability determination unit in the first embodiment. FIG. [Figure 6] 10 is a diagram for explaining another specific example of the determination of the normality by the state determination unit in accordance with the state determination conditions based on the executability determination result determined by the executability determination unit in the first embodiment. FIG. [Figure 7] 10 is a diagram for explaining another specific example of the determination of the normality by the state determination unit in accordance with the state determination conditions based on the executability determination result determined by the executability determination unit in the first embodiment. FIG. [Figure 8] 10 is a diagram for explaining another specific example of the determination of the normality by the state determination unit in accordance with the state determination conditions based on the executability determination result determined by the executability determination unit in the first embodiment. FIG. [Figure 9] 10 is a diagram for explaining another specific example of the determination of the normality by the state determination unit in accordance with the state determination conditions based on the executability determination result determined by the executability determination unit in the first embodiment. FIG. [Figure 10] 4 is a flowchart for explaining the operation of the state determination device according to the first embodiment. [Figure 11] 11 is a flowchart for explaining in more detail the operation of the state determination device shown in the flowchart of FIG. 10. [Figure 12] 12A and 12B are diagrams illustrating an example of a hardware configuration of the state determining device according to the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0010] Embodiment 1 FIG. 1 is a diagram illustrating an example of the configuration of a state determining device 1 according to the first embodiment. The state determination device 1 according to the first embodiment is mounted on, for example, a vehicle. The state determination device 1 according to the first embodiment outputs an execution instruction to the driver of the vehicle to sequentially execute tasks of different difficulty levels selected from a plurality of tasks of different difficulty levels set based on the task instruction conditions, and determines the normality of the driver's state, which indicates the degree to which the driver is suitable for driving, based on the determination result of whether the driver was able to execute the task according to the execution instruction. In the first embodiment, the plurality of tasks with different set levels of difficulty that are instructed to the driver by the state determination device 1 are referred to as "target tasks," and the plurality of tasks with different set levels of difficulty that are used to select the "target tasks" are referred to as "candidate tasks." That is, in the first embodiment, the state determination device 1 outputs execution instructions to the driver of the vehicle to cause the driver to execute, in order, a plurality of target tasks with different levels of difficulty that are selected from a plurality of candidate tasks with different set levels of difficulty based on the task instruction conditions, and determines the normality of the driver's state, which is the degree to which the driver is suitable for driving, based on the determination result of whether the driver was able to execute the target tasks in accordance with the execution instructions.

[0011] In the first embodiment, it is assumed that the higher the normality level, the more suitable the driver's state is for driving, i.e., the driver is estimated to be in a state that is suitable for driving. Details of the task instruction conditions when the state determination device 1 determines the normality level will be described later with specific examples. In the first embodiment, a "task" refers to an operation that the driver of the vehicle is made to perform in order to judge the normality based on how the driver reacts when instructed to perform the operation.

[0012] The state determination device 1 is connected to an imaging device 2. The state determination device 1 acquires, from the imaging device 2, a captured image of the driver captured by the imaging device 2, and determines, based on the captured image, whether the driver has been able to execute the target task in accordance with the execution instruction. That is, in the first embodiment, the imaging device 2 can also be said to be a reaction acquisition device for acquiring information relating to the driver's reaction to the execution instruction output by the state determination device 1 to cause the driver to execute the target task (hereinafter referred to as "reaction information"). The state determination device 1 acquires the captured image as reaction information from the imaging device 2, which is the reaction acquisition device.

[0013] The image capturing device 2 is, for example, a near-infrared camera or a visible light camera. The image capturing device 2 may be, for example, a device shared with a so-called "Driver Monitoring System (DMS)" that is mounted in a vehicle to monitor the status of passengers in the vehicle cabin. The image capturing device 2 is installed so as to be able to capture an image of at least the range within the vehicle cabin that includes the area where the upper half of the driver's body should be located, which is, for example, the area corresponding to the seat back and the space in front of the headrest. The imaging device 2 outputs the captured image to the state determination device 1.

[0014] The state determination device 1 includes a task execution instruction unit 11, a reaction information acquisition unit 12, an execution possibility determination unit 13, a state determination unit 14, a state determination result output unit 15, and a storage unit 16.

[0015] The task execution instruction unit 11 outputs an execution instruction to the driver of the vehicle to sequentially execute a plurality of target tasks of different difficulty levels selected from a plurality of candidate tasks of different difficulty levels based on the task instruction conditions.

[0016] The task instruction conditions are conditions for the procedure of selecting and issuing instructions to execute multiple target tasks of varying difficulty levels that the driver is to execute in order to determine normality. For example, the task instruction conditions specify which candidate tasks are to be selected as multiple target tasks of varying difficulty levels from among multiple candidate tasks of varying difficulty levels that have been set, in what order the selected multiple target tasks are to be executed by the vehicle driver, and under what output conditions the execution instructions to the vehicle driver to execute the selected multiple target tasks are to be output. The plurality of candidate tasks with different levels of difficulty are determined in advance by an administrator, etc. The administrator, etc. determines the plurality of candidate tasks with different levels of difficulty in advance, generates information about the plurality of candidate tasks with different levels of difficulty (hereinafter referred to as "candidate task information"), and stores it in storage unit 16. Furthermore, the task instruction conditions are set in advance by an administrator, etc. The administrator, etc. sets the task instruction conditions in advance, generates information indicating the set task instruction conditions (hereinafter referred to as “task instruction condition information”), and stores the information in the storage unit 16. The manager or the like generates the task candidate information and the task instruction condition information, for example, when the vehicle is shipped from the factory, and stores them in the storage unit 16. Note that multiple candidate tasks with different levels of difficulty and task instruction conditions may be determined for each vehicle. In this case, the multiple candidate tasks with different levels of difficulty and task instruction conditions are assumed to be common to all vehicles.

[0017] FIG. 2 is a diagram illustrating an example of the contents of candidate task information stored in storage unit 16 in the first embodiment. The candidate task information is information that associates the content of the candidate task with the difficulty level of the candidate task for each candidate task. A task number may be assigned to each candidate task. The task number is an arbitrary number, but the same number is not assigned to multiple candidate tasks.

[0018] The administrator or the like sets multiple candidate tasks and, for example, estimates how difficult it would be to execute the candidate tasks as instructed while driving, and sets the difficulty level corresponding to each candidate task. The difficulty level is expressed in appropriate levels. For example, the difficulty level may be expressed numerically, such as "Level 1," "Level 2," etc., or may be expressed as "easy," "normal," "difficult," etc. The number of difficulty levels can be set appropriately. The administrator or the like can set any task as a candidate task, but it is essential that multiple tasks with different levels of difficulty be set as candidate tasks. In other words, it is essential that the candidate task information contains information that associates at least a level of difficulty for each of multiple candidate tasks with different levels of difficulty. Note that even if multiple tasks with the same level of difficulty are set as candidate tasks, it is sufficient that a candidate task with a different level of difficulty is set in addition to the multiple candidate tasks with the same level of difficulty.

[0019] As an example, Figure 2 shows task candidate information in which the task "open your mouth" with a difficulty level of "Level 1" and the task "wink" with a difficulty level of "Level 4" are set. The higher the level number, the higher the difficulty level. In the following first embodiment, it is assumed that the storage unit 16 stores candidate task information having contents as shown in FIG. 2, as an example. In the following description, when the term "plurality of candidate tasks" is used, it means "plurality of candidate tasks with different levels of difficulty set."

[0020] The task instruction conditions include conditions that specify candidate tasks to be selected from among a plurality of candidate tasks as multiple target tasks of different difficulty levels (hereinafter referred to as "task selection conditions"), conditions that specify the order in which the selected multiple target tasks of different difficulty levels are to be executed (hereinafter referred to as "order conditions"), and conditions that output execution instructions to execute the selected multiple target tasks of different difficulty levels (hereinafter referred to as "instruction output conditions"). That is, the task instruction condition information generated in advance and stored in the memory unit 16 includes information regarding task selection conditions (hereinafter referred to as "task selection condition information"), information regarding order conditions (hereinafter referred to as "order condition information"), and information regarding instruction output conditions (hereinafter referred to as "instruction output condition information"). In the following description, when the term "plurality of target tasks" is used, it means "plurality of target tasks with different levels of difficulty set."

[0021] For example, the task selection condition may be set as follows: "Task Selection Condition Example (1): Among the candidate tasks, select candidate tasks No. 1 to No. 5 as target tasks." Note that this is just one example, and the task selection condition may be set as long as it specifies which candidate tasks are to be selected as multiple target tasks from among multiple candidate tasks. The number of target tasks is determined appropriately by an administrator or the like. However, it is essential that the task selection conditions are such that multiple candidate tasks associated with different levels of difficulty are selected as multiple target tasks. Even if the task selection conditions are such that multiple target tasks of the same level of difficulty are selected, it is sufficient that the task selection conditions are such that target tasks of different levels of difficulty are selected in addition to the multiple target tasks of the same level of difficulty. The administrator or the like sets the task selection conditions so that multiple candidate tasks of different levels of difficulty are selected as multiple target tasks. For example, a task selection condition may be set to select all of the multiple candidate tasks set in the candidate task information as target tasks. In the first embodiment, as an example, it is assumed that the task selection conditions are set to the above-mentioned task selection condition example (1).

[0022] The order condition may be set, for example, as follows: "Order condition example (1): Execute target tasks in order from lowest to highest difficulty. If there are multiple target tasks with the same difficulty, execute the target tasks in order of task number." Note that this is just one example, and the order condition may be set to any condition that specifies the order in which multiple target tasks are to be executed by the driver of the vehicle. In the first embodiment, as an example, it is assumed that the task selection conditions are set to the above-mentioned order condition example (1).

[0023] The instruction output condition may be set, for example, as follows: "Instruction output condition example (1): If it is determined that the driver was able to execute a certain target task as a result of outputting an execution instruction to the driver to execute the certain target task, an execution instruction to execute the next target task in order is output, and if it is determined that the driver was unable to execute the certain target task, execution instructions to execute the remaining target tasks of the multiple target tasks are not output." Note that this is merely an example, and it is sufficient that the instruction output condition is set to specify under what output condition execution instructions to execute multiple target tasks to the driver of the vehicle are to be output. In the first embodiment, as an example, it is assumed that the instruction output condition is set to the condition of the content of the instruction output condition example (1) described above.

[0024] The task execution instruction unit 11 selects multiple target tasks from multiple task candidates by referring to task candidate information based on task instruction conditions, more specifically, task selection conditions included in the task instruction conditions. The task execution instruction unit 11 can grasp the task selection conditions from task selection condition information included in the task instruction condition information stored in the storage unit 16. The task execution instruction unit 11 then determines that the driver should execute the selected multiple target tasks in an order based on the task instruction conditions, more specifically, the order conditions included in the task instruction conditions. The task execution instruction unit 11 can determine the order conditions from the order condition information included in the task instruction condition information stored in the storage unit 16. The task execution instruction unit 11 selects multiple target tasks and determines the order in which the selected multiple target tasks are to be executed by the driver, and then outputs an instruction to execute each target task based on the task instruction conditions, more specifically, the instruction output conditions included in the task instruction conditions. The task execution instruction unit 11 can grasp the instruction output conditions from the instruction output condition information included in the task instruction condition information stored in the storage unit 16.

[0025] 3 is a diagram illustrating an example of a plurality of target tasks and their execution order selected and determined by the task execution instruction unit 11 based on task instruction conditions in embodiment 1. Note that in FIG. 3, the difficulty level is also shown associated with each target task. Here, the content of the candidate task information is as shown in Fig. 2, the task selection conditions are the conditions of the content of the above-mentioned example task selection conditions (1), and the order conditions are the conditions of the content of the above-mentioned example order conditions (1). Therefore, as shown in Fig. 3, the task execution instruction unit 11 selects as multiple target tasks the candidate task "open mouth" of difficulty level "level 1", the candidate task "close mouth" of difficulty level "level 1", the candidate task "make scissors with left hand" of difficulty level "level 2", the candidate task "touch chin with right hand" of difficulty level "level 3", and the candidate task "touch ear with left hand" of difficulty level "level 3", and determines to have the driver execute these tasks in this order.

[0026] Then, the task execution instruction unit 11 outputs an execution instruction to the driver to execute each target task in the determined order, in accordance with the instruction output condition, here, the example instruction output condition (1).

[0027] More specifically, the task execution instruction unit 11 outputs an execution instruction to execute the nth (n is an integer) target task (hereinafter referred to as the "nth target task") among multiple target tasks, where n=1 to 5. The execution instruction may be, for example, information for outputting a voice instruction to execute the target task from a speaker (not shown), or information for displaying a message instruction to execute the target task on a display device (not shown). The speaker and display device are provided, for example, in a navigation device (not shown) mounted on the vehicle. The display device is, for example, a touch panel display. For example, when n=1, the task execution instruction unit 11 outputs a voice message saying "Please open your mouth" from the speaker to instruct execution of the first target task having the content "Open your mouth." The task execution instruction unit 11 may also display the message "Please open your mouth" on the display device.

[0028] The above example is merely an example, and the execution instruction may be information that instructs the driver to execute the target task in a manner that allows the driver to understand the content of the target task (for example, "open your mouth"). When an instruction to execute a target task is outputted by voice outputted from a speaker or a message displayed on a display device, the driver attempts to execute the target task in accordance with the execution instruction. At this time, whether the driver can execute the target task in accordance with the execution instruction depends on the state of the driver. The state determination device 1 determines the normality based on whether the driver was able to execute the instructed target task. This determination of the normality is performed by the execution feasibility determination unit 13. Details of the execution feasibility determination unit 13 will be described later, and we will return to the description of the task execution instruction unit 11.

[0029] After outputting the execution instruction to execute the n-th target task, the task execution instruction unit 11 then obtains from the execution possibility determination unit 13 the determination result as to whether or not the driver was able to execute the n-th target task. When the task execution instruction unit 11 receives a determination result from the execution possibility determination unit 13 indicating that the driver was able to execute the n-th target task, it counts up n by 1. Then, the task execution instruction unit 11 outputs an execution instruction to execute the n-th target task after n has been counted up by 1, i.e., the next target task in order. The task execution instruction unit 11 repeats outputting the execution instructions as described above until the execution possibility determination unit 13 determines that the driver was unable to execute the target task, or until execution instructions for all of the target tasks have been output. Note that the instruction output conditions also include a condition for ending output of the execution instructions. When the task execution instruction unit 11 obtains a determination result indicating that the driver was unable to execute the target task, or when it has output execution instructions for all of the multiple target tasks, in other words, when there is no target task for which to output an execution instruction next, it outputs information indicating that all output of execution instructions has been completed (hereinafter referred to as ``task completion information'') to the execution feasibility determination unit 13 via the response information acquisition unit 12.

[0030] Every time the task execution instruction unit 11 outputs an execution instruction, it outputs information indicating that the execution instruction has been output (hereinafter referred to as “instruction output completion information”) to the response information acquisition unit 12. The instruction output information includes information that can identify the content of the target task that has been instructed to be executed. The information that can identify the target task that has been instructed to be executed may be, for example, the task number assigned to the target task.

[0031] The reaction information acquisition unit 12 acquires reaction information relating to the driver's reaction to the execution instruction output by the task execution instruction unit 11. Here, the reaction information acquisition unit 12 acquires a captured image of the driver captured by the imaging device 2. The reaction information acquiring unit 12 may acquire the captured image output from the imaging device 2 for the first time after the instruction output completion information is output from the task execution instructing unit 11, or may acquire the captured image output from the imaging device 2 after a preset time (hereinafter referred to as "execution waiting time") has elapsed after the instruction output completion information is output from the task execution instructing unit 11. The execution waiting time is set in advance by an administrator or the like, assuming a grace period from when the driver is instructed to execute the target task until when the driver actually starts executing the task, and is stored in a buffer or the like inside the reaction information acquiring unit 12.

[0032] The reaction information acquisition unit 12 outputs the acquired reaction information, in this case, the captured image, to the execution feasibility determination unit 13. At this time, the reaction information acquisition unit 12 outputs to the execution feasibility determination unit 13, together with the reaction information, instruction output information output from the task execution instruction unit 11 that triggered the acquisition of the reaction information.

[0033] The execution possibility determination unit 13 determines, based on the response information acquired by the response information acquisition unit 12, whether or not the driver was able to execute the target task in response to the execution instruction. The response information acquisition unit 12 outputs instruction output information together with the response information. The execution feasibility determination unit 13 can identify the target task instructed to the driver based on the instruction output information. For example, if the information included in the instruction output information that can identify the target task instructed to be executed is a task number, the execution feasibility determination unit 13 can identify the content of the target task instructed to the driver by referring to the task instruction conditions stored in the storage unit 16. Note that in FIG. 1, the arrow connecting the execution feasibility determination unit 13 and the storage unit 16 is omitted.

[0034] The execution possibility determination unit 13 may determine whether or not the driver has been able to execute the target task in accordance with the execution instruction, using a known technique such as a known image recognition technique. For example, the execution possibility determination unit 13 may use a technique such as pattern matching to determine whether the driver has been able to execute the target task in response to the execution instruction. Furthermore, for example, the execution feasibility determination unit 13 may determine whether or not the driver was able to execute the target task in accordance with the execution instruction using a trained model in machine learning (hereinafter referred to as a "machine learning model"). The machine learning model is a model that receives, for example, a captured image and information that can identify the content of the target task as input, and outputs information indicating whether or not the person captured in the captured image was able to execute the target task. An administrator or the like may generate a machine learning model in advance, for example, when the vehicle is shipped from a factory, by learning captured images of people who were able to execute multiple candidate tasks and people who were not able to execute them, and store the model in the storage unit 16. The execution feasibility determination unit 13 can determine whether the driver was able to execute the target task in accordance with the execution instruction from information indicating whether the person captured in the captured image was able to execute the target task, which is obtained by inputting the captured image acquired from the reaction information acquisition unit 12 and information that can identify the content of the target task contained in the instruction output information into a machine learning model.

[0035] The execution feasibility determination unit 13 outputs a determination result (hereinafter referred to as the "executability determination result") as to whether the driver was able to execute the target task in accordance with the execution instruction to the state determination unit 14 and the task execution instruction unit 11. The execution feasibility determination result includes information that can identify the content of the target task that was instructed to be executed, and information indicating whether the driver was able to execute the target task. The execution feasibility determination unit 13 acquires the instruction output completion information from the reaction information acquisition unit 12, and includes in the execution feasibility determination result information that can identify the content of the target task that was instructed to be executed, which information is included in the instruction output completion information.

[0036] The state determination unit 14 determines the normality based on the execution possibility determination result output from the execution possibility determination unit 13 . More specifically, the state determination unit 14 determines the normality in accordance with the state determination conditions based on the determination result of the execution feasibility determination unit 13 as to whether or not the driver was able to execute the target task in response to the execution instruction.

[0037] The condition for determining the state defines the normality level based on the level of difficulty of the target task that the driver is able to execute in response to the execution instruction, and the condition for determining the state defines the correspondence between the difficulty of the target task that the driver was able to execute and the normality level. The state determination conditions are set in advance by an administrator, etc. The administrator, etc. sets the state determination conditions in advance, generates information indicating the set state determination conditions (hereinafter referred to as "state determination condition information"), and stores the information in the storage unit 16. The administrator or the like sets the condition for state determination and generates the condition information for state determination, taking into consideration, for example, the difficulty of the target task and the control content to be executed depending on the determined normality. For example, the administrator or the like sets the state determination conditions at the same time as setting the task instruction conditions.

[0038] For example, the following condition is set as the condition for determining the state: "Example condition (1) for determining the state: The normality level is set according to the highest difficulty level (hereinafter referred to as "maximum difficulty level") among the difficulty levels of the target tasks that have been executed, and if multiple target tasks with different contents are executed at the same maximum difficulty level, the normality level is set higher than when only one target task of the same maximum difficulty level is executed." Note that the normality level for each difficulty level and how much the normality level should be increased when multiple target tasks with different contents are executed at the same maximum difficulty level (how high the normality level should be) may be determined in advance by an administrator or the like, and may be defined in accordance with the condition for determining the state, or may be stored, for example, in the storage unit 16.

[0039] For example, each time the execution possibility determination unit 13 determines whether the driver was able to execute the target task in response to the execution instruction, it outputs the execution possibility determination result to the state determination unit 14, and the state determination unit 14 stores the execution possibility determination result in an internal buffer or the like. When task completion information is output from the task execution instruction unit 11, the state determination unit 14 determines the normality based on the execution possibility determination result stored in the internal buffer or the like. In this case, the task execution instruction unit 11 outputs the task completion information to the state determination unit 14 via the reaction information acquisition unit 12 and the execution possibility determination unit 13, for example. After determining the normality, the state determination unit 14 deletes the execution possibility determination result stored in the internal buffer or the like. It should be noted that this is merely an example, and for example, the execution feasibility determination unit 13 may store the execution feasibility determination result in an internal buffer or the like of the execution feasibility determination unit 13 until task completion information is output from the task execution instruction unit 11, and when task completion information is output from the task execution instruction unit 11, output the stored execution feasibility determination result to the state determination unit 14. When the execution feasibility determination unit 13 outputs the execution feasibility determination result to the state determination unit 14, it deletes the execution feasibility determination result that was stored in the internal buffer or the like.

[0040] In the first embodiment, as an example, it is assumed that the condition for state determination is set to the content of the above-mentioned example condition for state determination (1).

[0041] Here, a specific example will be given to explain how the state determining unit 14 determines the normality in accordance with the state determination conditions based on the execution possibility determination result determined by the execution possibility determining unit 13. Figures 4, 5, 6, 7, 8, and 9 are diagrams for explaining specific examples of the determination of normality by the state determination unit 14 in accordance with the state determination conditions based on the execution feasibility determination result determined by the execution feasibility determination unit 13 in embodiment 1. In the following specific example, the task execution instruction unit 11 determines multiple target tasks and their execution order as shown in Figure 3 based on the above-mentioned task selection condition example (1) and order condition example (1), and outputs an execution instruction in accordance with the execution instruction output condition example (1). In the following specific example, the normality level is judged on six levels: "0: Unfit," "1: Caution Required," "2: Distracted," "3: Normal," "4: Concentration," and "5: Alert." "0: Unfit" indicates a state in which the driver is completely unfit to drive, "1: Caution Required" indicates a state in which the driver has a serious problem with driving ability, "2: Distracted" indicates a state in which the driver lacks concentration while driving and is not paying sufficient attention to the surrounding situation, "3: Normal" indicates a state in which the driver has average driving ability and is paying appropriate attention, "4: Concentration" indicates a state in which the driver is highly concentrated while driving, pays attention to the surrounding situation, and is in a state in which the driver is able to make appropriate decisions, and "5: Alert" indicates a state in which the driver is fully alert, has high concentration and reaction speed, and is in a state in which the driver is able to demonstrate the best driving ability.

[0042] <Example of judgment (1)> For example, suppose that the task execution instruction unit 11 outputs an instruction to execute the first target task, "open your mouth," and as a result, the execution feasibility determination unit 13 determines that the driver was unable to execute the target task, "open your mouth." In this case, the task execution instruction unit 11 does not output instructions to execute the second or subsequent target tasks. The state determination unit 14 determines the normality based on the execution feasibility determination result that the driver was unable to execute the target task, "open your mouth." Since the driver was unable to perform the target task "open mouth" with a difficulty level of "level 1," the state determination unit 14 determines the normality level as "0: unsuitable" (see FIG. 4). Here, it is assumed that the normality level according to the difficulty level is set as "0: unsuitable" in the state determination conditions when the driver is unable to perform even the target task with a difficulty level of "level 1." The state determination unit 14 may, for example, identify the difficulty level of the target task by comparing the task completion information with the task instruction conditions stored in the storage unit 16.

[0043] <Example of judgment (2)> For example, suppose that the task execution instruction unit 11 outputs an execution instruction for the first target task, "open your mouth," and as a result, the execution feasibility determination unit 13 determines that the driver was able to execute the target task, "open your mouth." In this case, the task execution instruction unit 11 then outputs an execution instruction for the second target task, "close your mouth." As a result, the execution feasibility determination unit 13 determines that the driver was unable to execute the target task, "close your mouth." In this case, the task execution instruction unit 11 does not output execution instructions for the third and subsequent target tasks. The state determination unit 14 determines the normality based on the execution feasibility determination result that the driver was able to execute the target task, "open your mouth," but was unable to execute the target task, "close your mouth." The state determination unit 14 determines the normality level as "1: caution required" because the driver was able to perform the target task "open mouth" with a difficulty level of "level 1" but was unable to perform the target task "close mouth" with the same difficulty level of "level 1" (see FIG. 5). Here, it is assumed that the normality level according to the difficulty level is set as "1: caution required" in the state determination conditions when only one target task with a difficulty level of "level 1" is performed.

[0044] <Example of judgment (3)> For example, suppose that the task execution instructing unit 11 outputs execution instructions for the first and second target tasks, "open your mouth" and "close your mouth," and as a result, the execution feasibility determining unit 13 determines that the driver was able to execute the target tasks, "open your mouth" and "close your mouth." In this case, the task execution instructing unit 11 subsequently outputs an execution instruction for the third target task, "make your left hand a scissors." As a result, the execution feasibility determining unit 13 determines that the driver was unable to execute the target task, "make your left hand a scissors." In this case, the task execution instructing unit 11 does not output execution instructions for the fourth and subsequent target tasks. The state determining unit 14 determines the normality level based on the execution feasibility determination result that the driver was able to execute the target tasks, "open your mouth" and "close your mouth," but was unable to execute the target task, "make your left hand a scissors." The state determination unit 14 determines that the driver's normality level is "2: Distracted" because the driver was able to perform the target tasks "open mouth" and "close mouth" with a difficulty level of "Level 1" but was unable to perform the target task "make scissors with left hand" with a difficulty level of "Level 2" (see FIG. 6). Here, it is assumed that the normality level according to the difficulty level is set to "2: Distracted" in the state determination conditions when the driver is able to perform multiple target tasks with different contents with a difficulty level of "Level 1". In the above-mentioned <Determination Example (2)>, the driver was able to execute only one target task of "Level 1", whereas in the present <Determination Example (3)>, the driver was able to execute two target tasks with different contents of "Level 1". Therefore, the state determination unit 14 determines a higher normality level compared to the case of <Determination Example (2)>.

[0045] <Example of judgment (4)> For example, suppose that the task execution instructing unit 11 outputs execution instructions for the first, second, and third target tasks, "open your mouth," "close your mouth," and "make a scissors sign with your left hand," and as a result, the execution feasibility determining unit 13 determines that the driver was able to execute all of the target tasks, "open your mouth," "close your mouth," and "make a scissors sign with your left hand." In this case, the task execution instructing unit 11 then outputs an execution instruction for the fourth target task, "touch your chin with your right hand." As a result, the execution feasibility determining unit 13 determines that the driver was unable to execute the target task, "touch your chin with your right hand." In this case, the task execution instructing unit 11 does not output an execution instruction for the fifth target task. The state determining unit 14 determines the normality level based on the execution feasibility determination result that the driver was able to execute the target tasks, "open your mouth," "close your mouth," and "make a scissors sign with your left hand," but was unable to execute the target task, "touch your chin with your right hand." The state determination unit 14 determines that the driver's normality level is "3: normal" (see FIG. 7) because the driver was able to perform the target tasks "open mouth" and "close mouth" at "Level 1" difficulty level and the target task "make scissors with left hand" at "Level 2" difficulty level, but was unable to perform the target task "touch chin with right hand" at "Level 3" difficulty level (see FIG. 7). Here, it is assumed that the normality level according to the difficulty level is set to "3: normal" in the state determination conditions when the driver is able to perform the target task at "Level 2" difficulty level. It is determined that the driver was able to execute the target task with a difficulty level of "Level 1" and the target task with a difficulty level of "Level 2." However, since the state determination unit 14 sets the normality as the normality corresponding to the highest difficulty among the multiple target tasks that the driver was determined to be able to execute in the state determination conditions, the normality here is determined to be the normality corresponding to "Level 2."

[0046] <Example of judgment (5)> For example, suppose that the task execution instructing unit 11 outputs execution instructions for the first, second, third, and fourth target tasks, namely, "open your mouth," "close your mouth," "make scissors with your left hand," and "touch your chin with your right hand," and as a result, the execution feasibility determining unit 13 determines that the driver was able to execute all of the target tasks, namely, "open your mouth," "close your mouth," "make scissors with your left hand," and "touch your chin with your right hand." In this case, the task execution instructing unit 11 then outputs an execution instruction for the fifth target task, namely, "touch your ear with your left hand." As a result, the execution feasibility determining unit 13 determines that the driver was unable to execute the target task, namely, "touch your ear with your left hand." In this case, the state determining unit 14 determines the normality level based on the execution feasibility determination result that the driver was able to execute the target tasks, namely, "open your mouth," "close your mouth," "make scissors with your left hand," and "touch your chin with your right hand," but was unable to execute the target task, namely, "touch your ear with your left hand." The state determination unit 14 determines the normality level as "4: Concentration" because the driver was able to perform the target task "touch chin with right hand" with a difficulty level of "Level 3" but was unable to perform the target task "touch ear with left hand" with the same difficulty level of "Level 3" (see FIG. 8). Here, it is assumed that the normality level according to the difficulty level is set as "4: Concentration" in the state determination conditions when only one target task with a difficulty level of "Level 3" is performed.

[0047] <Example of judgment (6)> For example, suppose that the task execution instruction unit 11 outputs execution instructions for the first to fifth target tasks, namely, "open mouth," "close mouth," "make scissors with left hand," "touch chin with right hand," and "touch ear with left hand," and as a result, the execution feasibility determination unit 13 determines that the driver was able to execute all of the target tasks, "open mouth," "close mouth," "make scissors with left hand," "touch chin with right hand," and "touch ear with left hand." In this case, the state determination unit 14 determines the normality based on the execution feasibility determination result that the driver was able to execute all of the target tasks. Since the driver was able to perform both of the target tasks, "touch chin with right hand" and "touch ear with left hand," which have a difficulty level of "level 3," the state determination unit 14 determines the normality level as "5: Awake" (see FIG. 9). Here, it is assumed that the normality level according to the difficulty level is set as "5: Awake" in the state determination conditions when the driver is able to perform multiple target tasks with different contents, each of which has a difficulty level of "level 3." In the above-mentioned <Determination Example (5)>, the driver was able to execute only one target task of "Level 3", whereas in the present <Determination Example (6)>, the driver was able to execute two target tasks with different contents of "Level 3". Therefore, the state determination unit 14 determines a higher normality level compared to the case of <Determination Example (5)>.

[0048] The above-described specific example is merely an example, and the normality level may be determined in two stages, for example, as "high" or "low." For example, the state determination unit 14 may set a condition for determining the normality level to be "low" if the driver is unable to execute all of the target tasks of "Level 1," and to be "high" if the driver can execute all of the target tasks of "Level 1." Furthermore, for example, the normality level may be determined in three stages: "high," "normal," and "low." For example, the condition for state determination may be set so that the state determination unit 14 determines the normality level as "low" if the driver cannot perform all of the "Level 1" target tasks and the "Level 2" target tasks, the normality level as "normal" if the driver can perform all of the "Level 1" target tasks and the "Level 2" target tasks and cannot perform at least one of the "Level 3" target tasks, and the normality level as "high" if the driver can perform target tasks of all difficulty levels.

[0049] After determining the normality, the state determination unit 14 outputs the determined normality to the state determination result output unit 15.

[0050] The state determination result output unit 15 outputs information relating to the normality (hereinafter referred to as "normality information") determined by the state determination unit 14. The normality information includes at least information indicating the normality. The state determination result output unit 15 may output, as normality information, automatic driving control information for controlling automatic driving of the vehicle according to the normality determined by the state determination unit 14, to an automatic driving control device (not shown) that controls automatic driving of the vehicle. For example, when the normality is "0: Unsuitable," "1: Caution Required," or "2: Normal," the state determination result output unit 15 outputs automatic driving control information to the automatic driving control device that temporarily suspends switching from the automatic driving mode to the manual driving mode or transitions the automatic driving mode to an emergency stop mode. For example, when the normality is "4: Concentration" or "5: Alert," the state determination result output unit 15 outputs automatic driving control information to the automatic driving control device that permits switching from the automatic driving mode to the manual driving mode. For example, the state determination result output unit 15 may output to the automatic driving control device automatic driving control information in which an automatic driving level corresponding to the normality level is set. By setting the automatic driving level corresponding to the normality level, the state determination result output unit 15 can enable more detailed levels of automatic driving control, such as outputting automatic driving control information that temporarily suspends switching from automatic driving mode to manual driving mode or transitions the automatic driving mode to emergency stop mode when the normality level is "0: unsuitable" or "1: caution required," outputting automatic control information that follows the driver's input for accelerator operation but automatically controls steering operation when the normality level is "2: normal," and outputting automatic driving control information that permits switching from automatic driving mode to manual driving mode when the normality level is "4: concentration" or "5: alert."

[0051] Furthermore, the state determination result output unit 15 may output, as normality information, warning control information for issuing a warning to the driver according to the normality determined by the state determination unit 14, to a speaker, for example. For example, when the normality level is "0: unsuitable," "1: caution required," or "3: normal," the state determination result output unit 15 outputs to the speaker warning control information for generating a warning sound to wake up the driver. In this case, the state determination result output unit 15 may output to the speaker warning control information for changing the volume of the warning sound depending on the normality level, such as making the warning sound quieter when the normality level is "3: normal" than when the normality level is "1: caution required," and making the warning sound quieter when the normality level is "1: caution required" than when the normality level is "0: unsuitable."

[0052] The state determination result output unit 15 may, for example, store the normality information in the storage unit 16. Note that the arrow from the state determination result output unit 15 to the storage unit 16 is omitted in Fig. 1 .

[0053] The storage unit 16 stores various types of information. 1, the storage unit 16 is provided in the state determination device 1, but this is merely an example. The storage unit 16 may be provided in a location outside the state determination device 1 that can be referenced by the state determination device 1.

[0054] The operation of the state determining device 1 according to the first embodiment will be described. FIG. 10 is a flowchart for explaining the operation of the state determining device 1 according to the first embodiment. For example, when the vehicle power is turned on, the state determination device 1 starts the operation shown in the flowchart of FIG. 10, and repeats the operation shown in the flowchart of FIG. 10 at predetermined intervals until the vehicle power is turned off. Furthermore, the state determination device 1 may perform the operation shown in the flowchart of Fig. 10, for example, when switching between autonomous driving modes or whenever it becomes necessary to detect the normality. Note that the conditions under which it is determined that it becomes necessary to detect the normality are predetermined.

[0055] The task execution instruction unit 11 outputs an execution instruction to the driver of the vehicle to sequentially execute a plurality of target tasks of different difficulty levels selected from a plurality of candidate tasks of different difficulty levels based on the task instruction conditions (step ST1). When the task execution instruction unit 11 outputs execution instructions for a plurality of target tasks, it outputs instruction output completion information to the reaction information acquisition unit 12 every time it outputs an execution instruction.

[0056] The reaction information acquisition unit 12 acquires reaction information relating to the driver's reaction to the execution instruction output by the task execution instruction unit 11 in step ST1 (step ST2). Here, the reaction information acquisition unit 12 acquires an image of the driver captured by the imaging device 2. The reaction information acquisition unit 12 outputs the acquired reaction information, in this case, the captured image, to the execution feasibility determination unit 13. At this time, the reaction information acquisition unit 12 outputs to the execution feasibility determination unit 13, together with the reaction information, instruction output information output from the task execution instruction unit 11 that triggered the acquisition of the reaction information.

[0057] The execution possibility determination unit 13 determines whether or not the driver was able to execute the target task in response to the execution instruction, based on the reaction information acquired by the reaction information acquisition unit 12 in step ST2 (step ST3). The execution possibility determination unit 13 outputs the execution possibility determination result to the state determination unit 14 and the task execution instruction unit 11. Here, as an example, the execution possibility determination unit 13 outputs the execution possibility determination result to the state determination unit 14 every time it determines whether or not the driver has been able to execute the target task.

[0058] The state determination unit 14 determines the normality based on the execution possibility determination result output from the execution possibility determination unit 13 in step ST3 (step ST4). More specifically, the state determination unit 14 determines the normality in accordance with the state determination conditions based on the determination result of the execution feasibility determination unit 13 as to whether or not the driver was able to execute the target task in response to the execution instruction. After determining the normality, the state determination unit 14 outputs the determined normality to the state determination result output unit 15.

[0059] The state determination result output unit 15 outputs the normality information (step ST5).

[0060] FIG. 11 is a flowchart for explaining in more detail the operation of the state determining device 1 shown in the flowchart of FIG. The specific operations of steps ST1, ST2, ST4, and ST5 in FIG. 11 are similar to the specific operations of steps ST1, ST2, ST4, and ST5 in FIG. 10, respectively, and therefore will not be described again.

[0061] Based on the reaction information acquired by the reaction information acquisition unit 12 in step ST2, the execution feasibility determination unit 13 determines whether the driver was able to execute the target task in accordance with the execution instruction, and if it determines that the driver was able to execute the target task in accordance with the execution instruction (if "YES" in step ST31), it outputs the execution feasibility determination result indicating that the driver was able to execute the target task to the state determination unit 14 and the task execution instruction unit 11. The task execution instruction unit 11 determines whether or not execution instructions for all target tasks have been output (step ST32), and if it is determined that execution instructions for all target tasks have not been output ("NO" in step ST32), the operation of the state determination device 1 returns to the processing of step ST1, and the task execution instruction unit 11 outputs an execution instruction for the next target task in the order.

[0062] In step ST31, if the execution possibility determination unit 13 determines that the driver was unable to execute the target task in accordance with the execution instruction (in the case of "NO" in step ST31), it outputs an execution possibility determination result indicating that the driver was unable to execute the target task to the state determination unit 14 and the task execution instruction unit 11. When the task execution instruction unit 11 acquires an execution possibility determination result indicating that the driver was unable to execute the target task from the execution possibility determination unit 13, it outputs task completion information to the state determination unit 14 via the reaction information acquisition unit 12 and the execution possibility determination unit 13. Then, the operation of the state determination device 1 proceeds to the processing of step ST4.

[0063] In step ST32, if the execution feasibility determination unit 13 determines that execution instructions for all target tasks have been output (if "YES" in step ST32), it outputs task completion information to the state determination unit 14 via the reaction information acquisition unit 12 and the execution feasibility determination unit 13.

[0064] In this way, the state determination device 1 has the vehicle driver sequentially execute multiple target tasks of varying difficulty levels selected from multiple candidate tasks of varying difficulty levels based on the task instruction conditions, and acquires response information regarding the driver's response to the execution instructions.The state determination device 1 determines whether the driver was able to execute the target tasks in accordance with the execution instructions based on the acquired response information, and determines the normality level based on the determination result of whether the driver was able to execute the target tasks in accordance with the execution instructions.The state determination device 1 then outputs information regarding the determined normality level. Therefore, the state determination device 1 can determine the degree to which the driver's state is suitable for driving from the driver's reaction to the instruction.

[0065] When performing some control related to vehicle driving taking into consideration the driver's condition, it is preferable to change the control content depending on the degree to which the driver's condition is suitable for driving. When trying to determine the driver's condition, issuing instructions to the driver and judging based on the driver's reaction to the instructions leads to a more reliable determination. However, if only simple instructions are output to the driver, even if the driver accurately follows the instructions, it is not possible to determine to what extent the driver's condition is suitable for driving. In contrast, the state determination device 1 according to the first embodiment has the driver of the vehicle sequentially execute a plurality of target tasks of varying difficulty levels selected from a plurality of candidate tasks of varying difficulty levels based on a task instruction condition, and acquires response information regarding the driver's response to the execution instruction. Based on the acquired response information, the state determination device 1 determines whether the driver was able to execute the target tasks in accordance with the execution instruction, and determines the normality level based on the determination result of whether the driver was able to execute the target tasks in accordance with the execution instruction. This allows the state determination device 1 to determine the degree to which the driver's state is suitable for driving from the driver's reaction to the instruction.

[0066] In the above-described first embodiment, the state determination conditions are set to the state determination condition example (1) as an example, but this is merely an example. For example, the condition for determining the status may be set as follows: "Example condition for determining the status (2): weight the difficulty of the target task that has been executed, and determine the normality level from the difficulty level after weighting." It is assumed that the weighting to be applied to which target task of which difficulty level is to be applied is predetermined. The state determination unit 14 determines the normality in accordance with the example condition (2) for state determination based on the determination result of the execution feasibility determination unit 13 as to whether the driver was able to execute the target task in accordance with the execution instruction, and thereby the state determination device 1 can determine the normality by prioritizing, for example, whether the target task can be executed, which is important in determining the normality. The state determination conditions may be set to conditions other than the state determination condition example (1) and the state determination condition example (2). The administrator or the like can set the state determination conditions as appropriate.

[0067] In addition, in the above embodiment 1, the task selection conditions, order conditions, and instruction output conditions included in the task instruction conditions are set to the conditions of task selection condition example (1), order condition example (1), and instruction output condition example (1), respectively, but this is merely an example. As described above, the task selection condition, order condition, and instruction output condition may be set to the conditions of task selection condition example (1), order condition example (1), and instruction output condition example (1), respectively. Another example of the instruction output condition will be shown below. For example, the instruction output condition may be set as follows: "Instruction output condition example (2): As a result of outputting an execution instruction to the driver to execute a certain target task, an execution instruction to execute the next target task in order is output regardless of whether it is determined that the driver has been able to execute the certain target task." In this case, the task execution instruction unit 11 outputs all execution instructions for executing each target task. The state determination unit 14 determines the normality based on whether it is determined that the driver has been able to execute all the target tasks. In this case, the state determination condition may include a condition for determining the normality based on whether it is determined that the driver has been able to execute all the target tasks. The state determination unit 14 determines the normality level based on whether it is determined that the driver was able to perform all target tasks, so that the state determination device 1 can determine the normality level taking into account individual differences. For example, there may be a driver who is not good at target tasks of a certain level of difficulty, but is good at target tasks of a higher level of difficulty than the target task of that level of difficulty. The state determination unit 14 can determine the normality level taking this into account.

[0068] As another example, another example of the order condition will be shown. For example, the order condition may be set as follows: "Order condition example (2): Execute the target tasks in order of increasing difficulty. If there are multiple target tasks of the same difficulty, execute the target tasks in the order of task number." By setting the order condition example (2), the state determination device 1 can shorten the time until determining the normality, for example, when the control related to the vehicle based on the normality is an emergency control that requires a high normality.

[0069] Furthermore, for example, the task execution instruction unit 11 may randomly change the order in which the driver executes a plurality of target tasks. That is, for example, the order condition may be set to "Order Condition Example (3): The execution order of the target tasks is changed randomly each time." By enabling the task execution instruction unit 11 to randomly change the order in which the driver is to execute multiple target tasks, the state determination device 1 can prevent the driver from becoming accustomed to the multiple target tasks to be executed, and can more accurately determine the normality.

[0070] In the first embodiment, for example, the driver may be able to set the task instruction conditions. For example, the task instruction conditions include task selection conditions, order conditions, and instruction output conditions, as well as designated task conditions, which are set by the driver and specify which candidate tasks are to be executed or not executed from among multiple candidate tasks. For example, when a driver gets into a vehicle, he or she operates a display device, which is a touch panel display, to input an instruction to display a list of multiple candidate tasks (hereinafter referred to as a "candidate task list call instruction"). The task execution instruction unit 11 accepts the candidate task list call instruction from the driver and displays a list of candidate tasks on the display device by referring to the candidate task information in the storage unit 16. The driver checks the display device and operates the display device to specify candidate tasks to be executed or candidate tasks not to be executed. For example, some drivers are unable to wink even when fully awake. In this case, the driver can specify the candidate task of winking as a candidate task not to be executed so that the candidate task of winking is not set as a target task. The driver can also specify candidate tasks that they want to be set as target tasks. The task execution instruction unit 11 receives information specifying a candidate task that is not to be executed or a candidate task that is to be executed, sets specified task conditions such that the candidate task specified as a candidate task that is not to be executed is a candidate task that should be excluded (hereinafter referred to as an "excluded task"), and the candidate task specified as a candidate task that is to be executed is a candidate task that should be selected (hereinafter referred to as a "selected task"), generates information indicating the specified task conditions (hereinafter referred to as "specified task condition information"), and stores it in the memory unit 16.

[0071] When selecting a plurality of target tasks from a plurality of candidate tasks, the task execution instruction unit 11 selects the plurality of target tasks based on the specified task conditions. For example, if there is an excluded task among the candidate tasks that should be selected as the target task according to the task selection conditions, the task execution instruction unit 11 does not select the excluded task as the target task. Since the total number of target tasks decreases when an excluded task is not selected, the task execution instruction unit 11 may select any candidate task with other content as the target task to replace the excluded task, so that the total number of target tasks does not change. For example, if the selected task is not included in the candidate tasks that should be selected as the target task according to the task selection conditions, the task execution instruction unit 11 selects the selected task as the target task. Since the total number of target tasks increases by adding the selected task, the task execution instruction unit 11 may exclude any target tasks with other contents from the target tasks (do not set them as target tasks) by the amount of the selected task.

[0072] Furthermore, for example, the task instruction conditions may include a designated order condition in addition to the task selection condition, the order condition, and the instruction output condition. The designated order condition is a condition set by the driver that specifies the order in which multiple target tasks are to be executed. For example, when the driver gets into the vehicle, he operates a display device, which is a touch panel display, to input an instruction to display a list of multiple target tasks (hereinafter referred to as a "target task list call instruction"). The task execution instruction unit 11 accepts the target task list call instruction from the driver and refers to the task instruction condition information in the memory unit 16 to cause the display device to display the list of target tasks. The driver checks the display device and operates the display device to specify the execution order of the target tasks. The task execution instruction unit 11 accepts the information specifying the execution order of the target tasks, sets a specified order condition that specifies the execution order of the multiple target tasks set by the driver, and generates information indicating the specified order condition (hereinafter referred to as "specified order condition information") and stores it in the memory unit 16. Then, the task execution instruction unit 11 selects a plurality of target tasks from a plurality of candidate tasks, and when determining the execution order of the selected plurality of target tasks, if a specified order condition is set, determines the execution order of the plurality of target tasks based on the specified order condition.

[0073] Furthermore, in the above-described first embodiment, the reaction information is the captured image captured by the imaging device 2. In other words, the reaction acquisition device is the imaging device 2. However, this is merely an example, and the reaction acquisition device may be a device other than the imaging device 2. For example, if the target task is some kind of speech, the reaction acquisition device may be a microphone (not shown) mounted on the vehicle. In this case, the reaction information will be the speech collected by the microphone.

[0074] Furthermore, in the first embodiment described above, in the task selection conditions included in the task instruction conditions, a candidate task to be selected as one of a plurality of target tasks may be associated with information indicating that the candidate task is a candidate task that can determine whether a specific abnormal state exists. To give a specific example, if the task selection conditions specify "opening one's mouth" as a candidate task to be selected as one of a plurality of target tasks, information indicating "cerebral infarction" may be associated with "opening one's mouth." It is generally assumed that a person who has suffered a cerebral infarction has difficulty opening their mouth properly. In this case, for example, if the execution feasibility determination unit 13 determines that the driver was unable to execute "open mouth," the state determination unit 14 outputs information indicating the possibility of cerebral infarction together with the determined normality to the state determination result output unit 15, and the state determination result output unit 15 may output information indicating the possibility of cerebral infarction together with the normality information to an external device.

[0075] In the first embodiment, multiple task instruction conditions may be set according to the age group of the driver. For example, the administrator or the like may conduct an experiment in advance to identify multiple target tasks or the execution instruction order of the target tasks that are more likely to be appropriately determined as to the normality level for each age group. Then, the administrator or the like may set the task instruction conditions based on the results of the experiment. This allows the state determining device 1 to determine the normality level taking the driver's age into consideration.

[0076] In the first embodiment described above, the state determination device 1 is an in-vehicle device mounted on a vehicle, and the task execution instruction unit 11, the reaction information acquisition unit 12, the executability determination unit 13, the state determination unit 14, and the state determination result output unit 15 are provided in the in-vehicle device. However, this is merely an example. For example, some of the task execution instruction unit 11, the reaction information acquisition unit 12, the executability determination unit 13, the state determination unit 14, and the state determination result output unit 15 may be mounted in the in-vehicle device, and the rest may be provided in a server connected to the in-vehicle device via a network, thereby forming a system with the in-vehicle device and the server. Furthermore, the task execution instruction unit 11, the reaction information acquisition unit 12, the executability determination unit 13, the state determination unit 14, and the state determination result output unit 15 may all be provided in the server.

[0077] 12A and 12B are diagrams illustrating an example of a hardware configuration of the state determining device 1 according to the first embodiment. In the first embodiment, the functions of the task execution instruction unit 11, the reaction information acquisition unit 12, the execution feasibility determination unit 13, the state determination unit 14, and the state determination result output unit 15 are realized by the processing circuit 101. That is, the state determination device 1 includes the processing circuit 101 for performing control to determine the degree to which the driver's state is suitable for driving (normality) based on the driver's reaction to instructions. The processing circuit 101 may be dedicated hardware as shown in FIG. 12A, or may be a processor 104 that executes a program stored in memory as shown in FIG. 12B.

[0078] When the processing circuitry 101 is dedicated hardware, the processing circuitry 101 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0079] When the processing circuit is a processor 104, the functions of the task execution instruction unit 11, reaction information acquisition unit 12, executability determination unit 13, state determination unit 14, and state determination result output unit 15 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in memory 105. The processor 104 reads and executes the program stored in memory 105, thereby executing the functions of the task execution instruction unit 11, reaction information acquisition unit 12, executability determination unit 13, state determination unit 14, and state determination result output unit 15. In other words, the state determination device 1 includes memory 105 for storing a program that, when executed by the processor 104, results in the execution of steps ST1 to ST5 of FIG. 10 described above. It can also be said that the program stored in memory 105 causes the computer to execute the processing procedures or methods of task execution instruction unit 11, reaction information acquisition unit 12, execution feasibility determination unit 13, state determination unit 14, and state determination result output unit 15. Here, memory 105 corresponds to, for example, non-volatile or volatile semiconductor memory such as RAM, ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disc), etc.

[0080] It is also possible to realize some of the functions of the task execution instruction unit 11, the reaction information acquisition unit 12, the executability determination unit 13, the state determination unit 14, and the state determination result output unit 15 with dedicated hardware and some with software or firmware. For example, the functions of the reaction information acquisition unit 12 and the state determination result output unit 15 can be realized by the processing circuit 101 as dedicated hardware, and the functions of the task execution instruction unit 11, the executability determination unit 13, and the state determination unit 14 can be realized by the processor 104 reading and executing programs stored in the memory 105.

[0081] The storage unit 16 is composed of a memory 105 and the like. The state determination device 1 also includes an input interface device 102 and an output interface device 103 that perform wired or wireless communication with devices such as an imaging device 2, an automatic driving control device (not shown), or an audio output device (not shown).

[0082] As described above, according to the first embodiment, the state determination device 1 is configured to include a task execution instruction unit 11 that outputs an execution instruction to the driver of the vehicle to sequentially execute a plurality of target tasks of different difficulty levels selected from a plurality of candidate tasks of different difficulty levels that have been set based on task instruction conditions; a reaction information acquisition unit 12 that acquires reaction information regarding the driver's reaction to the execution instruction output by the task execution instruction unit 11; an execution feasibility determination unit 13 that determines whether or not the driver was able to execute the target task in accordance with the execution instruction based on the reaction information acquired by the reaction information acquisition unit 12; a state determination unit 14 that determines the normality, which is the degree to which the driver's state is suitable for driving, based on the determination result determined by the execution feasibility determination unit 13 as to whether or not the driver was able to execute the target task in accordance with the execution instruction; and a state determination result output unit 15 that outputs information regarding the normality determined by the state determination unit 14. Therefore, the state determination device 1 can determine the degree to which the driver's state is suitable for driving (normality) from the driver's reaction to the instruction. The state determination device 1 can determine the degree to which the driver's state is suitable for driving (normality) so as to enable more precise control of any control relating to vehicle driving, taking into account the driver's state.

[0083] In the state determination device 1, the state determination unit 14 determines the normality based on the determination result of the execution feasibility determination unit 13 as to whether or not the driver was able to execute the target task in accordance with the execution instruction, in accordance with the state determination conditions that define the correspondence between the difficulty of the target task that was able to be executed and the normality. Therefore, the state determination device 1 can determine the degree to which the driver's state is suitable for driving (normality) from the driver's reaction to the instruction. The state determination device 1 can determine the degree to which the driver's state is suitable for driving (normality) so as to enable more precise control of any control relating to vehicle driving, taking into account the driver's state.

[0084] Furthermore, in the state determination device 1, the task execution instruction unit 11 can randomly change the order in which the driver is made to execute a plurality of target tasks. Therefore, the state determination device 1 prevents the driver from getting used to the plurality of target tasks to be executed, and can more accurately determine the normality.

[0085] Furthermore, the driver can set the task instruction conditions. Therefore, the state determination device 1 can select, for example, tasks that take into consideration the physical characteristics of the driver as the plurality of target tasks, and determine the normality level more accurately.

[0086] Any of the components of the embodiments may be modified or omitted. [Explanation of symbols]

[0087] 1 Status determination device, 2 Imaging device, 11 Task execution instruction unit, 12 Response information acquisition unit, 13 Execution feasibility determination unit, 14 Status determination unit, 15 Status determination result output unit, 16 Storage unit, 101 Processing circuit, 102 Input interface device, 103 Output interface device, 104 Processor, 105 Memory.

Claims

1. a task execution instruction unit that outputs an execution instruction to a driver of the vehicle to sequentially execute a plurality of target tasks having different levels of difficulty selected from a plurality of candidate tasks having different levels of difficulty based on a task instruction condition; a reaction information acquisition unit that acquires reaction information regarding a reaction of the driver to the execution instruction output by the task execution instruction unit; an execution possibility determination unit that determines whether the driver was able to execute the target task in response to the execution instruction based on the reaction information acquired by the reaction information acquisition unit; a state determination unit that determines a normality level of the driver, which is a degree of suitability for driving, based on a determination result of whether or not the driver was able to execute the target task in accordance with the execution instruction, as determined by the execution feasibility determination unit; and a state determination result output unit that outputs information about the normality determined by the state determination unit A state determination device comprising:

2. The state determination unit determines the normality level based on a determination result of whether the driver was able to execute the target task in accordance with the execution instruction, as determined by the execution possibility determination unit, in accordance with a state determination condition that defines a correspondence relationship between the difficulty level of the target task that was able to be executed and the normality level.

2. The state determination device according to claim 1.

3. In the state determination conditions, The normality level is determined according to the greatest maximum difficulty level among the difficulty levels of the target tasks that have been successfully executed, and if multiple target tasks with different contents have been successfully executed at the same maximum difficulty level, the normality level is increased compared to when only one target task of the same maximum difficulty level has been successfully executed. The condition is set 3. The state determination device according to claim 2.

4. In the state determination conditions, The degree of difficulty of the target task that has been successfully executed is weighted, and the normality is determined based on the degree of difficulty after the weighting. The condition is set 3. The state determination device according to claim 2.

5. the task instruction condition includes an instruction output condition that is an output condition of the execution instruction, The task execution instruction unit outputs the execution instruction based on the instruction output condition.

2. The state determination device according to claim 1.

6. In the instruction output condition, If it is determined that the driver was able to execute the target task as a result of outputting the execution instruction to the driver to execute the target task, the execution instruction to execute the next target task in order is output, and if it is determined that the driver was unable to execute the target task, the execution instruction to execute the remaining target tasks among the plurality of target tasks is not output. The condition is set 6. The state determination device according to claim 5.

7. In the instruction output condition, As a result of outputting the execution instruction to the driver to execute a certain target task, regardless of whether it is determined that the driver has been able to execute the certain target task, the execution instruction to execute the next target task in order is output. The condition is set 6. The state determination device according to claim 5.

8. The reaction information acquisition unit acquires an image of the driver as the reaction information.

2. The state determination device according to claim 1.

9. The task execution instruction unit randomly changes the order in which the driver executes the plurality of target tasks.

2. The state determination device according to claim 1.

10. The driver can set the task instruction conditions.

2. The state determination device according to claim 1.

11. the task instruction condition includes a designated task condition set by the driver, which designates a candidate task to be executed or a candidate task not to be executed among the plurality of candidate tasks; The task execution instruction unit selects the target task based on the specified task condition.

11. The state determination device according to claim 10.

12. the task instruction condition includes a specified order condition set by the driver that specifies an execution order of the plurality of target tasks; The task execution instruction unit causes the driver to execute the plurality of target tasks in an order according to the specified order condition.

11. The state determination device according to claim 10.

13. The state determination result output unit outputs automatic driving control information for controlling automatic driving of the vehicle in accordance with the normality determined by the state determination unit.

2. The state determination device according to claim 1.

14. The state determination result output unit outputs warning control information for issuing a warning to the driver according to the normality determined by the state determination unit.

2. The state determination device according to claim 1.

15. Computer, a task execution instruction unit that outputs an execution instruction to a driver of the vehicle to sequentially execute a plurality of target tasks having different levels of difficulty selected from a plurality of candidate tasks having different levels of difficulty based on a task instruction condition; a reaction information acquisition unit that acquires reaction information regarding a reaction of the driver to the execution instruction output by the task execution instruction unit; an execution possibility determination unit that determines whether the driver was able to execute the target task in response to the execution instruction based on the reaction information acquired by the reaction information acquisition unit; a state determination unit that determines a normality level of the driver, which is a degree of suitability for driving, based on a determination result of whether or not the driver was able to execute the target task in accordance with the execution instruction, as determined by the execution feasibility determination unit; and a state determination result output unit that outputs information about the normality determined by the state determination unit A status determination program to function as a

Images