State estimation device, state estimation method, and program

Abstract

To provide an apparatus for appropriately estimating a state of a user.SOLUTION: A state estimation apparatus 1 for estimating a state of a user comprises: a dazzling degree estimation unit 10 which estimates a dazzling degree and its certainty degree on the basis of sensing data acquired by a camera 30 and an illuminance sensor 31; a drowsiness degree estimation unit 12 which estimates a drowsiness degree and its certainty degree; an occupant state estimation unit 14 which estimates a state of an occupant and its certainty degree obtained by adjusting the dazzling degree and the drowsiness degree; and an ambiguity elimination interference unit 16 which interferes the state of the occupant on the basis of an ambiguity elimination model when the certainty degree of the state of the occupant has a predetermined threshold or less, the apparatus then updating determination parameters 11 and 13 to use for estimation by the dazzling degree estimation unit 10 and the drowsiness degree estimation unit 12 on the basis of a result of the interference by the ambiguity elimination interference unit 16.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This relates to a system for monitoring the status of people in vehicles, facilities, etc. [Background technology]

[0002] Conventionally, there are known technologies that detect people from images and estimate their posture and behavior by applying facial orientation and skeletal models. However, when the estimation results depend on the environment and individual differences, there is a problem that passive sensors such as images alone cannot accurately estimate the state. There are also issues such as the inability to estimate internal states such as boredom with content or anxiety from appearance.

[0003] One application of technology for estimating human behavior is driver status detection. Non-Patent Document 1 proposes a technology for detecting abnormalities in the driver's status, in which if the reliability of the estimation result is low, feedback from the driver is received through voice dialogue to confirm the estimation result.

[0004] Other proposed technologies include one that uses dialogue to find out about a driver's physical condition, collects and stores the information, and then shares it with others by conducting a health check based on lifestyle habits, such as for lack of exercise (Patent Document 1), and another that uses dialogue stimuli to estimate a driver's level of alertness and comfort (Patent Document 2). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-24481 [Patent Document 2] Patent Publication No. 2021-12409 [Non-patent literature]

[0006] [Non-Patent Document 1] Hiroaki Hayashi et al., "Driver Abnormal Sign Detection System Using Multimodal Monitoring and Voice Dialogue," Society of Automotive Engineers of Japan 2021 Spring Meeting Summary of the Invention [Problem to be solved by the invention]

[0007] The technology proposed in Non-Patent Document 1 is a dialogue system that allows the user to answer yes / no about a single state. Therefore, it does not resolve ambiguity between the driver's internal state or multiple states. Furthermore, when ambiguity occurs, constantly engaging in dialogue may increase the driver's operational burden. Patent Document 1 does not have a mechanism for inferring health status from accumulated data in a manner that is adapted to the environment and individual needs, and estimating health status without dialogue, so the user may find it cumbersome to engage in dialogue every time. The technology described in Patent Document 2 merely estimates emotional status through dialogue, and does not resolve ambiguity between multiple estimation results obtained by other sensors.

[0008] In view of the above background, an object of the present invention is to provide a device that appropriately estimates the state of a user. [Means for solving the problem]

[0009] The state estimation device of the present invention is a device for estimating a user state, and includes a first state estimation unit that estimates a first user state and its certainty for individual evaluation criteria based on sensing data acquired by a sensor; a second state estimation unit that estimates a second user state obtained by adjusting a plurality of the first user states and its certainty; an inference unit that infers the second user state based on a user-specific model for disambiguation when the certainty of the second user state is equal to or less than a predetermined threshold; and updates parameters used for estimation in the first state estimation unit based on the inference results of the inference unit.

[0010] In this way, by using the result of inferring the second user state based on a user-specific disambiguation model to update the parameters used in the first state estimation unit, the estimation by the first state estimation unit can be made closer to an estimation result that reflects the user's specific state.

[0011] In the state estimation device of the present invention, the inference unit may infer the second user state based on the model when the first user state conflicts with respect to a plurality of evaluation criteria.

[0012] When the first user states are contradictory in this way, even if the certainty of each first state is high, it is considered that the user state is erroneous overall. By performing inference using a user-specific disambiguation model, it is possible to infer the second user state appropriately.

[0013] The state estimation device of the present invention may include a dialogue processing unit that asks a question to a user and acquires an answer to the question, and the inference unit may infer the second user state based on the answer acquired by the dialogue processing unit.

[0014] With this configuration, the second user state can be appropriately inferred based on the answer obtained through the dialogue processing.

[0015] In the state estimation device of the present invention, the dialogue processing unit may perform dialogue processing when the estimation result of the second state estimation unit satisfies a predetermined condition.

[0016] This configuration reduces the hassle of the dialogue process because the dialogue process is not performed every time. Here, the "predetermined condition" is a condition that indicates that the second user state is highly ambiguous. For example, the confidence level is below a predetermined threshold, or the user states of multiple evaluation criteria are contradictory.

[0017] The state estimation device of the present invention includes: a storage unit that stores data on the state of an occupant inferred by the inference unit through processing by the dialogue processing unit and various data sensed at that time; The apparatus may further include a learning unit that uses the data stored in the storage unit to learn a model for disambiguation.

[0018] This configuration allows the disambiguation model to be updated to be unique to the user.

[0019] Another aspect of the state estimation device of the present invention is a device for estimating a user state, and includes a first state estimation unit that estimates a first user state and its certainty for individual evaluation criteria based on sensing data acquired by a sensor, a second state estimation unit that estimates a second user state obtained by adjusting a plurality of the first user states and its certainty, and an inference unit that infers the second user state based on a user-specific model for disambiguation when the first user states for the plurality of evaluation criteria are contradictory.

[0020] When the first user states are contradictory in this way, even if the certainty of each first state is high, it is considered that the user state is erroneous overall. By performing inference using a user-specific disambiguation model, it is possible to infer the second user state appropriately.

[0021] The state estimation method of the present invention is a method for estimating a user state by a state estimation device, and includes the steps of: the state estimation device estimating a first user state and its certainty for individual evaluation criteria based on sensing data acquired by a sensor; the state estimation device estimating a second user state obtained by adjusting a plurality of the first user states and its certainty; the state estimation device inferring the second user state based on a user-specific model for disambiguation when the certainty of the second user state is equal to or less than a predetermined threshold; and the state estimation device updating parameters used for estimating the first user state based on a result of inferring the second user state.

[0022] The state estimation method of the present invention is a method for estimating a user state by a state estimation device, and includes the steps of: the state estimation device estimating a first user state and its certainty for individual evaluation criteria based on sensing data acquired by a sensor; the state estimation device estimating a second user state obtained by adjusting a plurality of the first user states and its certainty; and the state estimation device inferring the second user state based on a user-specific model for disambiguation when the first user states for a plurality of evaluation criteria are contradictory.

[0023] The program of the present invention is a program for estimating a user state, and causes a computer to execute the following steps: estimating a first user state and its certainty for individual evaluation criteria based on sensing data acquired by a sensor; estimating a second user state and its certainty by adjusting a plurality of the first user states; inferring the second user state based on a user-specific model for disambiguation when the certainty of the second user state is equal to or less than a predetermined threshold; and updating parameters used to estimate the first user state based on the inference result of the second user state.

[0024] The program of the present invention is a program for estimating a user state, and causes a computer to execute the steps of estimating a first user state and its certainty for individual evaluation criteria based on sensing data acquired by a sensor, estimating a second user state and its certainty by adjusting multiple first user states, and inferring the second user state based on a user-specific model for disambiguation when the first user states for multiple evaluation criteria are contradictory. [Effects of the Invention]

[0025] According to the present invention, the state of the user can be appropriately estimated. [Brief explanation of the drawings]

[0026] [Figure 1] FIG. 1 is a diagram illustrating a configuration of a state estimation device. [Figure 2] 1A and 1B are diagrams illustrating examples of determination parameters for dazzle level estimation and drowsiness level estimation, respectively; [Figure 3] 2(a) and 2(b). [Figure 4] FIG. 1 is a diagram conceptually illustrating a disambiguation model. [Figure 5] FIG. 10 is a diagram illustrating an example of an ambiguity pattern and a dialogue plan. [Figure 6] 1 shows an example of inference and learning of a disambiguation model. (a) shows the discretization of each variable. (b) shows the factor graph corresponding to (a). [Figure 7] FIG. 10 is a schematic diagram showing an overview of updating of a judgment parameter. [Figure 8] 1A and 1B are diagrams illustrating the decision boundary before and after updating of the decision parameter, respectively. [Figure 9] FIG. 10 is a diagram for explaining updating of the certainty factor of an estimation result. [Figure 10] FIG. 2 illustrates an operation of a state processing device according to an embodiment; [Figure 11] FIG. 10 is a diagram illustrating detailed operations of the ambiguity resolution dialogue process. DETAILED DESCRIPTION OF THE INVENTION

[0027] The state estimation device according to the present embodiment will be described below with reference to the drawings. Note that the following description merely shows an example of a preferred embodiment and is not intended to limit the scope of the invention as defined in the claims.

[0028] 1 is a diagram showing the configuration of a state estimation device 1 according to an embodiment. The state estimation device 1 is a device that estimates the state of an occupant (user) based on sensing data acquired from a camera 30, an illuminance sensor 31, etc. The camera 30 and the illuminance sensor 31 are merely examples of sensors, and the state estimation device 1 can acquire sensing data from various other sensors and use it to estimate the state of the occupant.

[0029] In this embodiment, a state estimation device 1 that estimates the state of a passenger in a vehicle will be described as an example. For convenience of explanation, an example will be given in which the state of the passenger is estimated based on the degree of dazzle, which indicates whether the passenger feels drowsy, and the degree of drowsiness, which indicates whether the passenger feels drowsy.

[0030] The state estimation device 1 includes a dazzle level estimation unit 10, a drowsiness level estimation unit 12, an occupant state estimation unit 14, and an output unit 15. The dazzle level estimation unit 10 estimates the dazzle level of the occupant based on sensing data transmitted from an illuminance sensor 31 and image data of the occupant transmitted from a camera 30. The drowsiness level estimation unit 12 estimates the drowsiness level of the occupant based on the image data of the occupant transmitted from the camera 30.

[0031] The dazzle level estimation unit 10 acquires image data of the occupant from the camera 30, identifies the area of ​​the occupant's eyes from the image based on the acquired image data of the occupant, and determines the degree to which the eyes are open (referred to as "eye opening degree"). The dazzle level estimation unit 10 determines the presence or absence of dazzle based on the eye opening degree calculated from the image data and the illuminance data acquired from the illuminance sensor 31.

[0032] FIG. 2(a) is a diagram showing an example of a judgment parameter for estimating the level of dazzle. As shown in FIG. 2(a), when the data on illuminance and eye opening level fall within the shaded area, it is estimated that dazzle is present. The dazzle level estimation unit 10 performs linear discrimination on the illuminance data and eye opening level data to estimate the presence or absence of dazzle. The discrimination boundary used here is the judgment parameter 11 for estimating the level of dazzle. The judgment parameter 11 is initially set to a value that averages out individual and situational differences.

[0033] The dazzle level estimation unit 10 estimates the presence or absence of dazzle and the confidence level of the judgment result. The confidence level is estimated to be higher as the illuminance moves away from the decision boundary, and lower when the illuminance is near the decision boundary. Note that, although the present embodiment has described an example in which the dazzle level is estimated using illuminance data and eye opening level, the dazzle level estimation unit 10 may estimate the dazzle level using only illuminance data.

[0034] FIG. 2(b) is a diagram showing an example of a judgment parameter for drowsiness level estimation. The drowsiness level estimation unit 12 acquires image data of the occupant from the camera 30 and estimates the drowsiness level of the occupant based on the acquired image data of the occupant. Specifically, the drowsiness level estimation unit 12 identifies the area of ​​the occupant's eyes from the image and determines the degree of eye opening. The drowsiness level estimation unit 12 compares the degree of eye opening with a threshold, and estimates that the occupant is drowsy if the degree of eye opening is equal to or less than the threshold, and estimates that the occupant is not drowsy if the degree of eye opening is greater than the threshold. The threshold used here is a judgment parameter 13 for drowsiness level estimation. The judgment parameter 13 is initially set to a value that averages individual differences and situational differences.

[0035] The drowsiness level estimation unit 12 estimates the degree of certainty of the determination result together with the presence or absence of drowsiness. The degree of certainty is estimated to be high as the illuminance is farther from the threshold, and low when the illuminance is near the threshold.

[0036] The occupant state estimation unit 14 estimates the overall occupant state by adjusting the dazzle level state calculated by the dazzle level estimation unit 10 and the drowsiness level state calculated by the drowsiness level estimation unit 12. When the reliability of the estimation results of the dazzle level estimation unit 10 and the drowsiness level estimation unit 12 is high, the occupant state estimation unit 14 estimates the occupant state in accordance with the estimation results of the dazzle level estimation unit 10 and the drowsiness level estimation unit 12. When the reliability of both the dazzle level and the drowsiness level is low, the ambiguity resolution inference unit 16 resolves the ambiguity. The ambiguity resolution inference unit 16 will be described later.

[0037] Figure 3 is a graph that overlaps the graphs shown in Figures 2(a) and 2(b). As shown in Figure 3, there is a region where the eye opening degree is small and the illuminance is high, where it is determined that there is both dazzle and drowsiness. Although there may be cases where an occupant feels drowsy at the same time as feeling dazzled, this is thought to be a rare case. In other words, dazzle and drowsiness can be said to be contradictory states.

[0038] In a state where both dazzle and drowsiness are present, if the certainty of one of the dazzle level or the drowsiness level is high and the certainty of the other is low, the occupant state estimation unit 14 estimates that the state with the higher certainty is present. If the certainty of the presence of dazzle is high and the certainty of the presence of drowsiness is also high, the occupant state estimation unit 14 resolves the contradictory states in the disambiguation inference unit 16.

[0039] The occupant state estimation unit 14 also includes a disambiguation inference unit 16 and a dialogue processing unit 18. The disambiguation inference unit 16 resolves ambiguity by applying data to a disambiguation model stored in a storage unit 17.

[0040] FIG. 4 is a conceptual diagram of the disambiguation model. The disambiguation model is a Bayesian network model in which the dependencies between nodes are defined by conditional probabilities. The variables of each node are generally vector values ​​consisting of multiple values. For example, sensor data S consists of data from multiple sensors. The node of observed value Z includes the observed value and its confidence level. The disambiguation model is a model customized for each occupant. The disambiguation inference unit 16 applies the traffic situation V, time period T, and sensor data S to each node of the disambiguation model, and applies the observed dazzle level, drowsiness level, and their confidence level data to the node of observed value Z to infer the true state X. In this way, the disambiguation inference unit 16 resolves ambiguity and infers the true state when the confidence levels of dazzle level and drowsiness level are low or when the confidence levels of the contradictory states of both dazzle and drowsiness are high.

[0041] The dialogue processing unit 18 has a function of having a dialogue with the occupant through a dialogue processing interface. When the ambiguity resolution inference unit 16 does not resolve the ambiguity, the dialogue processing unit 18 conducts a dialogue with the occupant to confirm the state of the occupant.

[0042] The dialogue processing unit 18 transmits speech data to the dialogue interface 32, and receives and analyzes the speech data acquired by the dialogue interface 32. The dialogue interface 32 is a device for dialogue between the apparatus and the user. Examples of the output interface include a speaker and a monitor, and examples of the input interface include a microphone and a touch panel.

[0043] Since passengers may feel annoyed if they are asked questions such as "Isn't it too bright?" or "Aren't you sleepy?" every time, the dialogue processor 18 devise ways to ask the questions.

[0044] FIG. 5 is a diagram showing examples of ambiguity patterns and dialogue plans. The dialogue processing unit 18 stores ambiguity patterns and corresponding dialogue plans in association with each other. In the example shown in FIG. 5, when the state is ambiguous but the certainty is high, the dialogue plan is provided to propose that the state is above a threshold. As an example, assuming that the drowsiness level is above a threshold, the dialogue "Shall we take a break?" is performed. In the case of conflicting states such as dazzled and drowsy, when the certainty of each state is high, the dialogue plan is provided to directly confirm one state. As an example, the dialogue "Isn't it dazzling?" is performed.

[0045] The storage unit 19 stores data on the occupant state estimated through the dialogue processing and various data sensed at that time as learning data. Here, the various data correspond to each node of the disambiguation model. The state estimation device 1 accumulates the learning data obtained from the occupant in the storage unit 19. The model learning unit 20 uses the accumulated learning data to learn the disambiguation model.

[0046] FIG. 6 is a diagram showing an example of how the ambiguity resolution model infers and learns. FIG. 6 shows an approximate model that does not consider the dependencies of the traffic situation V, time period T, and sensor data S in FIG. 4, assuming that currently, information other than the output from the occupant state estimation unit 14 is unavailable. In FIG. 6(a), Z1, Zn indicate observed values, and R1, Rn indicate certainty factors. X1, Xn indicate the true state of the occupant (unknown). As shown in FIG. 6(a), each variable {Zn, Rn} is discretized.

[0047] To be more precise, the factorization in the model in Figure 6 can be written as an additive equation. These factors are determined by learning, which will be explained below. To convert it into a probability, we simply normalize the equation obtained using the proportional relationship in the following equation:

number

[0048] Next, the Bayesian network model shown in Figure 6(a) is trained by assuming a Dirichlet distribution for the prior distribution of each random variable and applying EAP (Expected A Posteriori) estimation. To perform inference, the Bayesian network model shown in Figure 6(a) is first converted into a factor model equivalent to a Bayesian network, as shown in Figure 6(b). A factor graph is a general-purpose probability model. Inference is performed by using the message passing method in the converted factor graph. Note that the message passing method itself can be used as a Bayesian network, but converting it into a factor graph for processing provides better visibility and greater versatility.

[0049] Next, the updating of the judgment parameters 11, 13 of the dazzle level estimation unit 10 and the drowsiness level estimation unit 12 will be described. As described above, values ​​that average individual differences and situational differences are initially set as the judgment parameters of the dazzle level estimation unit 10 and the drowsiness level estimation unit 12 (see FIG. 2). The state estimation device 1 of this embodiment tunes the judgment parameters 11, 13 for estimating the dazzle level and drowsiness level, which are individual evaluation criteria, to suit the individual characteristics of the occupant.

[0050] 7 is a schematic diagram showing an overview of updating of the judgment parameters. The state estimation device 1 feeds back the occupant state estimated by the occupant state estimation, or the occupant state disambiguated by the ambiguity resolution inference as necessary, to the dazzle level estimation unit 10 and the drowsiness level estimation unit 12, and updates the judgment parameters.

[0051] FIG. 8 shows the decision boundary before and after updating the decision parameter 11. FIG. 8(a) shows the decision boundary B before updating. In FIG. 8(a), data D1 and D2 indicated by "x" are below the decision boundary and were estimated to be dazzling-free, but were determined to be dazzling by the occupant state estimation unit 14. The result of this occupant state estimation is fed back to the dazzle level estimation unit 10, which then updates the decision parameter 11 to the decision boundary Bnew. The decision parameter 11 may be updated by online learning each time the occupant state estimation unit 14 estimates the state, or by batch learning when a certain amount of data has been accumulated or at a certain timing.

[0052] Fig. 9 is a diagram for explaining updating of the certainty factor of the estimation result. Fig. 9 shows an example of the drowsiness level estimated by the drowsiness level estimation unit 12. In Fig. 9, p1(x) indicates the probability of the drowsiness level in terms of the eye opening level. The probability of the drowsiness level decreases as the eye opening level increases, and becomes 0 at the threshold for determining whether or not there is a drowsiness level.

[0053] p0(x) indicates the probability of the degree of dazzle from the degree of eye opening. Under the assumption that dazzle and drowsiness are mutually exclusive, the probability of the degree of dazzle is the probability of false detection of the degree of drowsiness. The probability of the degree of dazzle also decreases as the degree of eye opening increases. The drowsiness level estimation unit 12 obtains information on the probability of the degree of eye opening and calculates the certainty of the degree of drowsiness using the following formula: where ρ is the original certainty of the degree of drowsiness.

number

[0054] By receiving feedback of information on the degree of eye opening in this way, the drowsiness level estimation unit 12 can update the certainty factor of the drowsiness level.

[0055] Fig. 10 is a flowchart showing the operation of the state estimating device 1 of this embodiment, and Fig. 11 is a flowchart showing the operation of the ambiguity resolution dialogue process (step S20). The process at each step will be described below.

[0056] (Step S10) The state estimation device 1 acquires sensor data from a camera 30 and an illuminance sensor 31 .

[0057] (Step S11) The state estimation device 1 estimates the occupant's dazzle level and drowsiness level based on the acquired sensor data. The dazzle level estimation unit 10 estimates the occupant's dazzle level and its reliability based on data from the camera 30 and the illuminance sensor 31, and the drowsiness level estimation unit 12 estimates the occupant's drowsiness level and its reliability based on data from the illuminance sensor 31. The dazzle level and drowsiness level estimated here represent the occupant's state with respect to each individual evaluation criterion.

[0058] (Step S12) The state estimation device 1 estimates the overall state of the occupant and its certainty factor based on the dazzle level and drowsiness level estimated in step S11. The occupant state obtained in step S12 is a state in which the dazzle level and drowsiness level have been adjusted.

[0059] (Step S13) The state estimation device 1 determines whether or not there is ambiguity in the occupant state estimated in step S12. The state estimation device 1 determines the ambiguity based on the degree of certainty of the occupant state and whether or not there is a contradiction between the multiple occupant states. If it is determined that there is ambiguity (YES in step S13), the process proceeds to step S14, and if there is no ambiguity (NO in step S13), the process proceeds to step S16.

[0060] (Step S14) The state estimation device 1 performs processing to resolve ambiguity of the occupant state by inference using the ambiguity resolution model.

[0061] (Step S15) The state estimation device 1 determines whether the ambiguity has been resolved as a result of the ambiguity resolution inference. If the certainty of the occupant state is equal to or greater than a predetermined threshold, or if the conflicting states of multiple occupant states are resolved, it is determined that the ambiguity has been resolved. If the ambiguity has been resolved (YES in S15), the process proceeds to step S16, and if the ambiguity has not been resolved (NO in S15), the process proceeds to step S20. Step S20 will be described with reference to FIG. 11.

[0062] (Step S16) The state estimation device 1 updates the occupant state based on the estimated occupant state or the occupant state after the ambiguity has been resolved.

[0063] (Step S17) The state estimation device 1 feeds back the updated occupant state to the dazzle level estimation unit 10 and the drowsiness level estimation unit 12, and tunes the discrimination boundary of the dazzle level estimation unit 10 and the discrimination threshold of the drowsiness level estimation unit 12 to parameters specific to the occupant.

[0064] (Step S18) The state estimation device 1 outputs data on the estimated occupant state to various in-vehicle devices and performs processing based on the occupant state. For example, if the occupant state is drowsy, the in-vehicle devices output a message urging the occupant to take a rest.

[0065] (Step S20) The state estimation device 1 performs a process to resolve the ambiguity of the occupant's state through a dialogue with the occupant, as will be described in detail below with reference to FIG.

[0066] (Step S21) The state estimation device 1 selects a dialogue plan according to the ambiguity. An example of the correspondence between the ambiguity pattern and the dialogue plan to be selected is as explained in Fig. 5. The state estimation device 1 also stores data of specific dialogue messages according to the dialogue plan.

[0067] (Step S22) The state estimating device 1 carries out a dialogue with the occupant based on the dialogue plan.

[0068] (Step S23) The state estimation device 1 estimates the state of the occupant from the occupant's answer. When the occupant is directly asked a question about their state, the answer indicates the occupant's state. When a proposal is made based on an assumption of the occupant's state, if the proposal is accepted, it is estimated that the occupant's state was correct, and if it is not accepted, it is estimated that the occupant's state was incorrect.

[0069] (Step S24) The state estimation device 1 stores the data on the estimated occupant states and various sensed data as learning data. Specifically, the state estimation device 1 stores the data in association with the acquired data from among the traffic situation V, time period T, sensor data S, observation values ​​Z, etc. The state estimation device 1 can use this learning data to learn the disambiguation model.

[0070] The configuration of the state estimating device 1 of this embodiment has been described above, but an example of the hardware of the above-mentioned state estimating device 1 is a computer equipped with a CPU, RAM, ROM, a hard disk, a display, a keyboard, a mouse, a communication interface, etc. A program having modules that realize each of the above-mentioned functions is stored in RAM or ROM, and the above-mentioned state estimating device 1 is realized by executing the program by the CPU. Such programs are also included in the scope of the present invention.

[0071] The state estimation device 1 of the embodiment estimates the occupant state based on the sensor values ​​of the camera 30 and the illuminance sensor 31, but when the occupant state is highly ambiguous, the ambiguity can be resolved by performing ambiguity resolution inference, and the occupant state can be appropriately determined. That is, according to the embodiment, not only can a state that can be understood by anyone regardless of the person be estimated, but also subtle states that differ depending on the occupant.

[0072] In addition, the state estimation device 1 feeds back the estimated occupant state to the dazzle level estimation unit and the drowsiness level estimation unit, and by tuning the judgment parameters for dazzle level estimation and drowsiness level estimation, it becomes possible to perform dazzle level estimation and drowsiness level estimation that are appropriate for the occupant.

[0073] When the state estimation device 1 cannot resolve the ambiguity of the occupant state by the ambiguity resolution inference, the state estimation device 1 resolves the ambiguity through a dialogue with the occupant, thereby enabling highly accurate estimation of the occupant state. Furthermore, this dialogue is performed only when the ambiguity cannot be resolved, and the state is not confirmed with the occupant every time, so the occupant is less likely to feel bothered.

[0074] The state estimation device 1 stores data obtained through dialogue with the occupant as learning data and uses the learning data to learn the disambiguation model, so that the disambiguation model is updated to suit the occupant. This increases the number of cases where ambiguity is resolved by disambiguation inference, reducing the need for dialogue with the user.

[0075] Although the state estimation device of the present invention has been described in detail above using an embodiment, the state estimation device of the present invention is not limited to the above embodiment. In the above embodiment, the camera 30 and the illuminance sensor 31 are used as sensors, and the degree of dazzle and drowsiness are used as individual evaluation criteria to be estimated. However, it is possible to estimate the occupant state based on various evaluation criteria using various sensor values. The sensor may acquire the temperature, temperature, and time inside the vehicle. Furthermore, the evaluation criteria may be the comfort of the room temperature, interest in content, or other conditions to be estimated. [Explanation of symbols]

[0076] 1. State Estimation Device 10 Dazzle level estimation section 11 Decision parameters 12 Drowsiness level estimation unit 13 Decision parameters 14 Occupant state estimation unit 15 Output section 16 Disambiguation reasoning unit 17 Ambiguity resolution model memory 18 Dialogue processing section 19 Learning data storage unit 20 Model Learning Section 30 Camera 31 Illuminance sensor 32 Interactive Interface

Claims

1. An apparatus for estimating a user state, comprising: a first state estimation unit that estimates a user state and a certainty factor for each evaluation criterion of the degree of dazzle and the degree of drowsiness based on sensing data acquired by the camera and / or the illuminance sensor, a parameter for estimating the presence or absence of dazzle, and a parameter for estimating the presence or absence of drowsiness; a storage unit that stores a model for disambiguating states regarding a plurality of evaluation criteria, the model being generated by learning using data of a user's state acquired in advance and sensing data sensed at that time; and an inference unit that infers the user state based on the model for disambiguation when the user state conflicts with respect to the evaluation criteria of the dazzle level and the drowsiness level, The state estimation device updates the parameters for estimating the presence or absence of dizziness and the parameters for estimating the presence or absence of drowsiness based on the inference result of the inference unit.

2. a dialogue processing unit that asks a question to a user and acquires a response thereto; The state estimation device according to claim 1 , wherein the inference unit infers the user state based on a response acquired by the dialogue processing unit.

3. The state estimation device according to claim 2 , wherein the dialogue processing unit performs dialogue processing when the user state conflicts with respect to the evaluation criteria of the dazzle level and the drowsiness level.

4. a storage unit that stores data on the state of the occupant inferred by the inference unit through processing by the dialogue processing unit and various data sensed at that time; a learning unit that learns a model for disambiguating the ambiguity using a combination of the data on the occupant state stored in the storage unit and various sensed data as training data; The state estimation device according to claim 2 or 3, comprising:

5. An apparatus for estimating a user state, comprising: a first state estimation unit that estimates a user state and a certainty factor thereof with respect to each of evaluation criteria of a dazzle level and a drowsiness level based on sensing data acquired by a camera and / or an illuminance sensor; a storage unit that stores a model for disambiguating states regarding a plurality of evaluation criteria, the model being generated by learning using data of a user's state acquired in advance and sensing data sensed at that time; and an inference unit that, when the user state is contradictory with respect to evaluation criteria of a dazzle level and a drowsiness level, infers the user state based on the model for disambiguation; A state estimation device comprising:

6. A method for estimating a user state by a state estimation device, comprising: a step in which the state estimation device estimates a user state and a certainty factor thereof with respect to each evaluation criterion of a degree of dazzle and a degree of drowsiness, based on sensing data acquired by a camera and / or an illuminance sensor, a parameter for estimating the presence or absence of dazzle, and a parameter for estimating the presence or absence of drowsiness; When the user state conflicts with respect to the evaluation criteria of the dazzle level and the drowsiness level, inferring the user state based on a model for disambiguating states with respect to a plurality of evaluation criteria, the model being generated by learning using data of the user state previously acquired and sensing data sensed at that time; a step of updating a parameter for estimating the presence or absence of dizziness and a parameter for estimating the presence or absence of drowsiness based on an inference result of the user state by the state estimation device; A state estimation method comprising:

7. A method for estimating a user state by a state estimation device, comprising: a step in which the state estimation device estimates a user state and a certainty factor thereof with respect to each of evaluation criteria of a dazzle level and a drowsiness level based on sensing data acquired by a camera and / or an illuminance sensor; a step in which the state estimation device infers the user state based on a model for disambiguating states with respect to a plurality of evaluation criteria, the model being generated by learning using data of the user state previously acquired and sensing data sensed at that time, when the user state with respect to the evaluation criteria of dazzle level and drowsiness level conflicts with each other; A state estimation method comprising:

8. A program for estimating a user state, the program comprising: a step of estimating a user state and a certainty factor for each evaluation criterion of the degree of dazzle and the degree of drowsiness based on sensing data acquired by the camera and / or the illuminance sensor, a parameter for estimating the presence or absence of dazzle, and a parameter for estimating the presence or absence of drowsiness; When the user's state is contradictory with respect to the evaluation criteria of dazzle level and drowsiness level, inferring the user's state based on a model for disambiguating states with respect to a plurality of evaluation criteria, the model being generated by learning using data of the user's state acquired in advance and sensing data sensed at that time; updating a parameter for estimating the presence or absence of dizziness and a parameter for estimating the presence or absence of drowsiness based on the inference result of the user state; A program that executes the following.

9. A program for estimating a user state, the program comprising: A step of estimating a user state and a certainty factor thereof with respect to each of the evaluation criteria of the dazzle level and the drowsiness level based on sensing data acquired by the camera and / or the illuminance sensor; When the user's state is contradictory with respect to the evaluation criteria of dazzle level and drowsiness level, inferring the user's state based on a model for disambiguating states with respect to a plurality of evaluation criteria, the model being generated by learning using data of the user's state acquired in advance and sensing data sensed at that time; A program that executes the following.

Images