Inner surface state estimation device, inner surface state estimation method, and program

Abstract

An internal state estimation device (1X) comprises: an observation data acquisition means (15X) for acquiring observation data resulting from observation of a subject; a first internal state acquisition means (16X) for acquiring the result of estimation of a first internal state of the subject; and a second internal state estimation means (17X) for estimating a second internal state of the subject, which is an internal state correlated with the first internal state, on the basis of the observation data and the result of the estimation.

Description

Inner surface state estimation device, inner surface state estimation method, and storage medium

[0001] The present disclosure relates to the technical fields of an inner surface state estimation device, an inner surface state estimation method, and a storage medium that perform processing related to estimation of an inner surface state.

[0002] There are known devices or systems that estimate the internal state of a subject. For example, Patent Literature 1 discloses a system that can determine the concentration level of a person in an image from image data by learning the relationship between image data of the user and the concentration level of the user.

[0003] Japanese Patent Application Laid-Open No. 2022-014473

[0004] When learning and estimating internal states such as concentration levels using subjective values ​​perceived by people as correct answers, the subjective values ​​used as correct answer data contain bias. It is difficult to prepare a large amount of correct answer data for such subjective values, and a small amount of data makes it difficult to perform sufficient learning and therefore makes it difficult to obtain the desired estimation accuracy.

[0005] In view of the above-mentioned problems, one object of the present disclosure is to provide an inner state estimation device, an inner state estimation method, and a storage medium that are capable of accurately estimating an inner state.

[0006] One aspect of an inner state estimation device is an inner state estimation device comprising: an observation data acquisition means for acquiring observation data obtained by observing a subject; a first inner state acquisition means for acquiring an estimation result of a first inner state of the subject; and a second inner state estimation means for estimating a second inner state of the subject, which is an inner state correlated with the first inner state, based on the observation data and the estimation result.

[0007] One aspect of the inner state estimation method is a method of estimating an inner state, in which a computer acquires observation data of a subject, acquires an estimation result of a first inner state of the subject, and estimates a second inner state of the subject, which is an inner state correlated with the first inner state, based on the observation data and the estimation result. Note that the term "computer" includes any electronic device (which may be a processor included in an electronic device) and may be composed of multiple electronic devices.

[0008] One aspect of the storage medium is a storage medium that stores a program that causes a computer to execute a process of acquiring observation data of a subject, acquiring an estimation result of a first inner state of the subject, and estimating a second inner state of the subject, which is an inner state that is correlated with the first inner state, based on the observation data and the estimation result.

[0009] According to the present disclosure, the inner state of a subject can be estimated with high accuracy.

[0010] 1 shows a schematic configuration of an internal state estimation system according to a first embodiment; FIG. 2 shows a hardware configuration of an information processing device; FIG. 3 shows (A) a distribution map of a second internal state, with drowsiness as the first internal state as an axis, when the first internal state is drowsiness and the second internal state is concentration; FIG. 4 shows (B) a distribution map of a second internal state, with tension as the first internal state as an axis, when the first internal state is tension and the second internal state is pleasantness / unpleasantness; FIG. 5 shows an example of functional blocks of an internal state estimation device; FIG. 6 shows an example of detailed functional block diagrams of a first internal state estimation unit and a second internal state estimation unit; FIG. 7 shows an example of functional blocks of a processor of the internal state estimation device related to learning an estimation model; FIG. 8 shows an example of detailed functional blocks of a second learning unit; FIG. 9 shows an example of a flowchart showing a processing procedure related to inner state estimation by the internal state estimation device according to the first embodiment; FIG. 10 shows an example of detailed functional blocks of a second learning unit according to a first modification; FIG. 11 shows an example of detailed functional blocks of a second internal state estimation unit according to a first modification;

[0011] Hereinafter, embodiments of an inner surface state estimation device, an inner surface state estimation method, and a storage medium will be described with reference to the drawings.

[0012] <First Embodiment> (1) System Configuration Fig. 1 shows a schematic configuration of an inner state estimation system 100 according to a first embodiment. The inner state estimation system 100 estimates with high accuracy the inner state of a subject, such as the concentration level, which cannot be accurately determined by others just by looking, and presents the estimation results. Here, the "subject" may be a person whose inner state is managed by an organization, or may be an individual user.

[0013] The inner surface state estimation system 100 mainly includes an inner surface state estimation device 1, an input device 2, an output device 3, a storage device 4, and a sensor 5.

[0014] The internal state estimation device 1 communicates data with the input device 2, output device 3, and sensor 5 via a communication network or by direct wireless or wired communication. The internal state estimation device 1 estimates the subject's internal state based on the input signal "S1" provided by the input device 2, the sensor (detection) signal "S3" provided by the sensor 5, and information stored in the storage device 4. The internal state estimation device 1 then estimates the second internal state with high accuracy based on the estimation result of the internal state in which the subject's physiological phenomenon is observed (also referred to as the "first internal state"). The "second internal state" is an internal state in which a state in which the subject's physiological phenomenon is observed is confused with a state in which the subject's physiological phenomenon is not observed, and is an internal state that cannot or is difficult to determine by observation alone (in other words, can only be determined by an expert). Specific examples of the first internal state and the second internal state will be described below. The inner state estimation device 1 generates an output signal “S2” relating to the estimation result of the second inner state of the subject, and supplies the generated output signal S2 to the output device 3. The inner state estimation device 1 may also execute a process relating to learning of an estimation model of the inner state, which will be described later.

[0015] The input device 2 is a user interface that accepts manual input (external input) of information about each subject. The user who inputs information using the input device 2 may be the subject himself / herself, or a person who manages or supervises the subject's activities. The input device 2 may be, for example, a variety of user input interfaces, such as a touch panel, buttons, a keyboard, a mouse, or a voice input device. The input device 2 supplies the generated input signal S1 to the internal state estimation device 1. The output device 3 displays predetermined information or outputs sound based on the output signal S2 supplied from the internal state estimation device 1. The output device 3 is, for example, a display, a projector, a speaker, etc.

[0016] The sensor 5 measures the subject's biosignals and supplies the measured biosignals to the internal state estimation device 1 as a sensor signal S3. In this case, the sensor signal S3 may be any biosignal (including vital information) of the subject, such as the subject's heart rate, brain waves, pulse waves, sweat rate (electrodermal activity), hormone secretion, cerebral blood flow, blood pressure, body temperature, electromyography, respiratory rate, and acceleration. The sensor 5 may also be a device that analyzes blood collected from the subject and outputs a sensor signal S3 indicating the analysis results. The sensor 5 may also be a wearable device worn by the subject, such as a camera that captures images of the subject or a microphone that generates audio signals of the subject's speech, or a device such as a personal computer or smartphone operated by the subject. The wearable device may include, for example, a GNSS (Global Navigation Satellite System) receiver, an acceleration sensor, or other sensors that detect biosignals, and outputs the output signals of these sensors as the sensor signal S3. Furthermore, the sensor 5 may supply information corresponding to the amount of operation of a personal computer, a smartphone, or the like as a sensor signal S3 to the inner state estimation device 1. Furthermore, the sensor 5 may output a sensor signal S3 representing biological data (including sleeping hours) from the subject while the subject is sleeping.

[0017] The storage device 4 is a memory that stores various information necessary for the processing executed by the inner surface state estimating device 1. The storage device 4 may be an external storage device such as a hard disk connected to or built into the inner surface state estimating device 1, or may be a storage medium such as a flash memory. The storage device 4 may also be a server device that performs data communication with the inner surface state estimating device 1. The storage device 4 may also be composed of multiple devices.

[0018] The storage device 4 functionally includes a first estimation model information storage unit 41 and a second estimation model information storage unit 42 .

[0019] The first estimation model information storage unit 41 stores information about a first estimation model, which is a model for estimating a first inner state. The first estimation model is a model that, when data (e.g., a feature vector) expressed in a predetermined tensor format representing observation results of a subject is input, outputs a first inner state estimate value representing an estimation result of the first inner state of the subject. This model is, for example, a model based on any machine learning method such as a neural network or a support vector machine (including a statistical model; the same applies hereinafter), and parameters required for the model are stored in the first estimation model information storage unit 41. For example, if the above-mentioned first estimation model is a model based on a neural network such as a convolutional neural network, the first estimation model information storage unit 41 stores information on various parameters such as the layer structure, the neuron structure of each layer, the number and filter size of filters in each layer, and the weight of each element of each filter. Note that the first inner state estimate value does not need to be a scalar value but may be a vector value including multiple values.

[0020] The second estimation model information storage unit 42 stores information about a second estimation model, which is a model for estimating a second inner state. The second estimation model is a model that, when data (e.g., a feature vector) expressed in a predetermined tensor format representing observation results of a subject is input, outputs a second inner state estimation value representing an estimation result of the subject's second inner state. This model is, for example, a model based on any machine learning method such as a neural network or a support vector machine, and parameters required for the model are stored in the second estimation model information storage unit 42. Here, the data input to the second estimation model includes the first inner state estimation value output by the first estimation model. Note that the second inner state estimation value does not need to be a scalar value and may be a vector value including multiple values.

[0021] In addition, the data input into the first estimation model and the second estimation model is not limited to data based on observation results of the subject at a certain time, but may also be data based on time-series observation results of the subject over a specified period of time.

[0022] Furthermore, when the inner state estimating device 1 learns at least one of the first estimation model and the second estimation model, training data necessary for learning the first estimation model and / or the second estimation model is further stored in the storage device 4. This training data will be described later.

[0023] The configuration of the inner state estimation system 100 shown in FIG. 1 is an example, and various modifications may be made to the configuration. For example, the input device 2 and the output device 3 may be configured as an integrated unit. In this case, the input device 2 and the output device 3 may be configured as a tablet terminal that is integrated with or separate from the inner state estimation device 1. The input device 2 and the sensor 5 may also be configured as an integrated unit. The inner state estimation device 1 may also be configured as a plurality of devices. In this case, the plurality of devices that make up the inner state estimation device 1 exchange information necessary to execute pre-assigned processing between these plurality of devices. In this case, the inner state estimation device 1 functions as a system.

[0024] (2) Hardware Configuration Fig. 2 shows the hardware configuration of the inner surface state estimation device 1. The inner surface state estimation device 1 includes, as hardware components, a processor 11, a memory 12, and an interface 13. The processor 11, the memory 12, and the interface 13 are connected via a data bus 10.

[0025] The processor 11 executes programs stored in the memory 12 to function as a controller (arithmetic unit) that performs overall control of the inner surface state estimation device 1. The processor 11 is, for example, a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a TPU (Tensor Processing Unit). The processor 11 may be composed of multiple processors. The processor 11 is an example of a computer.

[0026] The memory 12 is composed of various types of volatile and non-volatile memory, such as RAM (Random Access Memory), ROM (Read Only Memory), and flash memory. The memory 12 also stores programs for executing the processes performed by the inner state estimation device 1. Note that some of the information stored in the memory 12 may be stored in one or more external storage devices capable of communicating with the inner state estimation device 1, or may be stored in a storage medium that is detachable from the inner state estimation device 1.

[0027] The interface 13 is an interface for electrically connecting the inner surface state estimation device 1 to other devices. These interfaces may be wireless interfaces such as network adapters for wirelessly transmitting and receiving data to and from other devices, or may be hardware interfaces for connecting to other devices via cables or the like.

[0028] The hardware configuration of the inner state estimating device 1 is not limited to the configuration shown in Fig. 2. For example, the inner state estimating device 1 may include at least one of an input device 2 or an output device 3. Furthermore, the inner state estimating device 1 may be connected to or built into a sound output device such as a speaker.

[0029] (3) Specific Examples of the First Inner State and the Second Inner State The first inner state and the second inner state are correlated. The first inner state is an expressed inner state that can be seen by others, and the second inner state is a state in which the expressed inner state that can be seen by others is confused with an inner state that can only be understood subjectively by the person.

[0030] 3A and 3B are diagrams conceptually illustrating the relationship between a first internal state and a second internal state. FIG. 3A shows a distribution diagram of the second internal state, with the first internal state's "sleepiness" as the axis, when the first internal state is "sleepiness" and the second internal state is "concentration." FIG. 3B shows a distribution diagram of the second internal state, with the first internal state's "tension" as the axis, when the first internal state is "tension" and the second internal state is "pleasure / unpleasantness." As shown in FIG. 3A, states of concentration occur when sleepiness is low, while states of unconcentration (i.e., states of low concentration) are distributed over a wide range along the drowsiness axis of the first internal state. As a result, states of concentration and states of unconcentration coexist in states of low sleepiness. As shown in FIG. 3B, states of unconcentration occur when tension (i.e., states of high tension) occurs, while states of pleasure are distributed over a wide range along the tension axis of the first internal state. As a result, in a state of high tension, pleasant and unpleasant states are mixed together. In this way, the first internal state and the second internal state are correlated, and the second internal state is a state that includes all or part of the first internal state.

[0031] Here, a specific example of the first internal state when the concentration level is estimated as the second internal state will be described. For example, the first internal state may be "drowsiness." In this case, the first estimation model is a model that estimates drowsiness and outputs a NEDO index (drowsiness label) as the first internal state estimated value. In another example, the first internal state may be "emotion." In this case, the first estimation model is a model that estimates emotions of joy, anger, sadness, and happiness and outputs an index value indicating joy, anger, sadness, and happiness as the first internal state estimated value. In yet another example, the first internal state may be a combination of "alertness" and "pleasantness / unpleasantness." In this case, the first estimation model outputs, for example, an index value based on Russell's circumplex model as the first internal state estimated value.

[0032] Next, examples of combinations of the first internal state and the second internal state when the second internal state is other than concentration will be described. For example, when the first internal state is "tension," the second internal state is "pleasantness / unpleasantness." In another example, when the first internal state is "drowsiness," the second internal state is one of "alertness," "tension," "health," "activity," and "anxiety." In yet another example, when the first internal state is "emotion," the second internal state is one of "health," "activity," and "mood." In yet another example, when the first internal state is "pleasantness / unpleasantness," the second internal state is one of "mood" and "emotion."

[0033] The correct answer data used to train the first estimation model to estimate the first inner state indicates an objective evaluation value (objective evaluation value) annotated (i.e., labeled as correct) by a third party other than the subject observed in generating the input data for the first estimation model. Meanwhile, the correct answer data used in the second estimation model to estimate the second inner state indicates a subjective evaluation value (subjective evaluation value) by the subject observed in generating the input data for the first estimation model. For example, the correct answer data used in the second estimation model indicates the results of a subjective evaluation questionnaire on the second inner state. For example, if the second inner state is concentration, the correct answer data indicates the level of concentration indicated by the questionnaire results (e.g., a five-point scale). Furthermore, as described below, the input data for the first estimation model may be part of the input data for the second estimation model.

[0034] (4) Functional Blocks Fig. 4 shows an example of functional blocks of the inner state estimating device 1. Functionally, the processor 11 of the inner state estimating device 1 includes an observation data acquisition unit 15, a first inner state estimating unit 16, a second inner state estimating unit 17, and an output control unit 18. Note that in Fig. 4, blocks between which data is exchanged are connected by solid lines, but the combination of blocks between which data is exchanged is not limited to Fig. 4. The same applies to other functional block diagrams described later.

[0035] The observation data acquisition unit 15 receives a sensor signal S3 from the sensor 5 via the interface 13, which observes the subject at a target time or period for estimating the subject's inner state, and acquires observation data based on the received sensor signal S3. The target time or period for estimation may be the latest time or period at a predetermined time interval, or a pre-specified time or period. The observation data may be the sensor signal S3 itself, or data obtained by performing predetermined signal processing on the sensor signal S3. The observation data acquisition unit 15 then extracts data used to estimate the first inner state (also referred to as "first observation data") from the observation data and supplies the first observation data to the first inner state estimation unit 16. The observation data acquisition unit 15 also extracts data used to estimate the second inner state (also referred to as "second observation data") from the observation data and supplies the second observation data to the second inner state estimation unit 17. The observation data acquisition unit 15 may generate observation data using the input signal S1 supplied from the input device 2 in addition to the sensor signal S3 supplied from the sensor 5.

[0036] Here, the first observation data is data used to estimate the first internal state, and is, for example, physiological information representing index values ​​related to physiological phenomena observed in the subject, such as heart rate and sweating. In another example, the first observation data is various data (non-contact sensor data) obtained from a sensor that is not in contact with the subject. In this case, the first observation data may be face data, which is information related to the subject's face (e.g., video data showing the subject's face or index values ​​calculated from the data, such as facial expression and eyelid opening), or audio data representing the sound emitted by the subject (including index values ​​calculated from audio signals). In yet another example, the first observation data may be motion data related to the subject's movements. The motion data may be, for example, video data showing the subject's walking or index values ​​calculated from the video data, or the subject's observed acceleration. Alternatively, the first observation data may be a combination of the above-mentioned information. Alternatively, the first observation data may be any type of observation data used to estimate the first internal state to be estimated. In addition, when the first observation data is a predetermined index value, etc., the observation data acquisition unit 15 performs processing to generate the first observation data from the sensor signal S3 based on a program, etc. stored in the memory 12 or the storage device 4.

[0037] The second observation data is data used to estimate the second internal state, and may be, for example, physiological information about the subject, non-contact sensor data such as face data and voice data, motion data, or a combination thereof. The second observation data may be any type of observation data used to estimate the second internal state to be estimated. Note that, when the second observation data is a predetermined index value, the observation data acquisition unit 15 performs processing to generate the second observation data from the sensor signal S3 based on a program stored in the memory 12 or the storage device 4.

[0038] Furthermore, the first observation data and the second observation data may be partially overlapping data or may be different data. For example, if the first inner state is “drowsiness” and the second inner state is “concentration,” the first observation data is eyelid opening and blinking information based on the facial image of the subject, and the second observation data is eyelid opening, blinking information, facial expression information, and gaze information based on the facial image of the subject.

[0039] The first inner state estimation unit 16 estimates a first inner state of the subject based on the first observation data supplied from the observation data acquisition unit 15, and supplies a first inner state estimation value, which is a value representing the estimation result of the first inner state, to the second inner state estimation unit 17. In this case, the first inner state estimation unit 16 inputs the first observation data or feature amounts (which may be feature amount vectors; the same applies hereinafter) based on the first observation data to a first estimation model configured with reference to the first estimation model information storage unit 41, and acquires the first inner state estimation value output by the first estimation model in response to the input.

[0040] The second inner state estimation unit 17 estimates a second inner state of the subject based on the second observation data supplied from the observation data acquisition unit 15 and the first inner state estimation value supplied from the first inner state estimation unit 16. The second inner state estimation unit 17 then supplies the second inner state estimation value, which is a value representing the estimation result of the second inner state, to the output control unit 18. In this case, the first inner state estimation unit 16 inputs the second observation data or feature amounts based on the second observation data and the first inner state estimation value into a second estimation model configured with reference to the second estimation model information storage unit 42, and obtains the second inner state estimation value output by the second estimation model in response to the input.

[0041] The output control unit 18 outputs information related to the estimation result of the subject's inner state. For example, the output control unit 18 displays the estimation result of the subject's inner state (more specifically, the second inner state) by the second inner state estimation unit 17 on the display unit of the output device 3, or outputs it as audio via the audio output unit of the output device 3. In this case, the output control unit 18 may output the estimation result of the first inner state together with the estimation result of the second inner state.

[0042] Furthermore, the output control unit 18 may, for example, compare the estimation result of the inner state with a reference value for determining whether or not there is an abnormality in the inner state, and issue a predetermined notification to the subject or their manager based on the comparison result. For example, when the estimation result of the inner state is a numerical value that is less favorable than the reference value, the output control unit 18 may output predetermined warning information or information regarding advice.

[0043] Fig. 5 is an example of a detailed functional block diagram of the first inner state estimating unit 16 and the second inner state estimating unit 17. In the example of Fig. 5, the first inner state estimating unit 16 functionally includes a first observation data acquiring unit 61, a first feature generating unit 62, and a first estimation model applying unit 63. Furthermore, the second inner state estimating unit 17 functionally includes a second observation data acquiring unit 71, a second feature generating unit 72, and a second estimation model applying unit 73.

[0044] The first observation data acquisition unit 61 acquires first observation data from the observation data acquisition unit 15 and supplies the acquired first observation data to the first feature generation unit 62. The first feature generation unit 62 generates first features, which are features to be input to the first estimation model. In this case, the first feature generation unit 62 generates the features of the first observation data acquired by the first observation data acquisition unit 61 as the first features. The first features are data in a predetermined tensor format, which is the input format of the first estimation model used by the first estimation model application unit 63. Note that the feature extraction process performed by the first feature generation unit 62 may be a process based on any feature extraction technology. For example, the feature extraction process performed by the first feature generation unit 62 may be a process of calculating statistics such as the mean and variance of index values ​​included in the first observation data, or a process of inputting the first observation data into a feature extraction model trained based on deep learning or the like to acquire features output by the model.

[0045] The first estimation model application unit 63 inputs the first feature supplied from the first feature generation unit 62 into the first estimation model configured with reference to the first estimation model information storage unit 41, and acquires a first inner surface state estimation value output by the first estimation model in response to the input. The first estimation model application unit 63 then supplies the acquired first inner surface state estimation value to the second feature generation unit 72.

[0046] The second observation data acquisition unit 71 acquires second observation data from the observation data acquisition unit 15 and supplies the acquired second observation data to the second feature generation unit 72. The second feature generation unit 72 generates second feature quantities, which are feature quantities to be input to the second estimation model. In this case, the second feature generation unit 72 first generates feature quantities of the second observation data acquired by the second observation data acquisition unit 71. The second feature generation unit 72 then adds the generated feature quantities to the first inner state estimation value supplied from the first estimation model application unit 63 to generate the second feature quantities. The second feature quantities are data in a predetermined tensor format that is consistent with the input format of the second estimation model used by the second estimation model application unit 73. Note that the feature extraction process performed by the second feature generation unit 72 on the second observation data may be a process based on any feature extraction technology.

[0047] The second estimation model application unit 73 inputs the second feature amounts supplied from the second feature amount generation unit 72 into a second estimation model configured with reference to the second estimation model information storage unit 42, and acquires a second inner surface state estimation value output by the second estimation model in response to the input. The second estimation model application unit 73 then supplies the acquired second inner surface state estimation value to the output control unit 18.

[0048] 4 and 5, the internal state estimation device 1 can estimate the second internal state of the subject with high accuracy based on the observation data of the subject and the estimation result of the first internal state. By outputting the estimation result of the second internal state, such as the concentration level, estimated simply and with high accuracy, the internal state estimation device 1 can preferably encourage the subject to engage in self-management aimed at improving productivity, etc.

[0049] The components of the observation data acquisition unit 15, the first inner state estimation unit 16, the second inner state estimation unit 17, and the output control unit 18 described in FIG. 4 can be realized, for example, by the processor 11 executing a program. Alternatively, the necessary programs may be recorded on any non-volatile storage medium and installed as needed to realize the components. At least some of these components are not limited to being realized by software programs, but may also be realized by any combination of hardware, firmware, and software. At least some of these components may also be realized using a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcontroller. In this case, the integrated circuit may be used to realize a program consisting of the above components. Furthermore, at least a portion of each component may be configured by an ASSP (Application Specific Standard Product), an ASIC (Application Specific Integrated Circuit), or a quantum processor (quantum computer control chip). In this way, each component may be realized by various hardware. The same applies to other embodiments described below. Furthermore, each of these components may be realized by the cooperation of multiple computers, for example, using cloud computing technology.

[0050] (5) Model Learning Next, a method for learning the first estimation model and the second estimation model will be described. Hereinafter, as an example, a case where the internal state estimation device 1 learns the first estimation model and the second estimation model will be described. However, a device other than the internal state estimation device 1 may also learn the first estimation model and the second estimation model. Furthermore, the learning of the first estimation model and the learning of the second estimation model may be performed by different devices. For example, the internal state estimation device 1 may refer to parameters of the first estimation model that have already been learned by another device and learn the second estimation model (i.e., the processing of the second learning unit 22, described below). Hereinafter, a person who is the object of observation in generating training data will also be referred to as a "subject." There may be multiple subjects, and the subject may or may not include the target.

[0051] 6 shows an example of functional blocks of the processor 11 of the inner state estimation device 1 related to learning the first estimation model and the second estimation model. With regard to learning the first estimation model and the second estimation model, the processor 11 functionally includes a first learning unit 21 and a second learning unit 22. In addition, first training data 31 and second training data 32 are stored in the storage device 4, etc.

[0052] The first training data 31 is training data used for learning the first estimation model, and includes input data 311 and supervised data 312. The input data 311 corresponds to a predetermined number of first observation data prepared in advance, and the supervised data 312 indicates a supervised first inner state estimation value that should be output by the first estimation model for each of the predetermined number of input data 311. Here, since the first inner state is an expressed inner state that can be seen by others, the supervised data 312 has been generated by an annotation work performed by a third party other than the subject of the input data 311. In other words, the supervised data 312 is data generated based on objective evaluation, and indicates an objective evaluation value of the subject's first inner state at the time of observation of each input data 311.

[0053] The first learning unit 21 learns a first estimation model based on the input data 311 and the supervised data 312, and stores the parameters of the first estimation model obtained by learning in the first estimation model information storage unit 41. In this case, the first learning unit 21 acquires a first internal state estimate output by the first estimation model when the selected piece of input data 311 or its feature amount (i.e., the feature amount corresponding to the first feature amount) is input to the first estimation model. The first learning unit 21 then updates the parameters of the first estimation model so as to minimize the error (loss) between the first internal state estimate output by the first estimation model and the supervised data 312 corresponding to the used input data 311. The first learning unit 21 then performs this process for all input data 311 or until a predetermined learning termination condition is satisfied. The algorithm for determining the above parameters so as to minimize the loss may be any learning algorithm used in machine learning, such as gradient descent or backpropagation. Note that, before the first estimation model is learned, the first estimation model information storage unit 41 may store initial parameters of the first estimation model.

[0054] The second training data 32 is training data used to learn the second estimation model and includes first input data 321, second input data 322, and correct answer data 323. The first input data 321 corresponds to a predetermined number of first observation data prepared in advance. The second input data 322 corresponds to a predetermined number of second observation data prepared in advance, each of which is data obtained by observing the same subject at the same timing (i.e., the same time) as the first input data 321. The correct answer data 323 indicates a correct second inner state estimate that should be output by the second estimation model for each pair of first input data 321 and second input data 322 that has the same subject and observation timing. Here, the second inner state is an inner state of a subject that cannot be accurately determined by others simply by looking at it, and therefore the correct answer data 323 is generated by the subject annotating the first input data 321 and the second input data 322 (e.g., questionnaire results, etc.). In other words, the correct answer data 323 is data generated based on a subjective evaluation, and indicates the subjective evaluation value of the subject's second inner state at the time of observation of each pair of the first input data 321 and the second input data 322.

[0055] The second learning unit 22 learns a second estimation model based on the parameters of the learned first estimation model stored in the first estimation model information storage unit 41 and the second training data 32, and stores the parameters of the second estimation model obtained by learning in the second estimation model information storage unit 42.

[0056] Fig. 7 shows an example of detailed functional blocks of the second learning unit 22. Functionally, the second learning unit 22 shown in Fig. 7 includes a first input data acquisition unit 221, a first feature amount generation unit 222, a first estimation model application unit 223, a second input data acquisition unit 224, a second feature amount generation unit 225, and a second estimation model learning unit 226.

[0057] The first input data acquisition unit 221 extracts any one piece of first input data 321 to which the first estimation model is to be applied from the second training data 32. The first feature generation unit 222 generates first features that are consistent with the input format of the first estimation model from the one piece of first input data 321 acquired by the first input data acquisition unit 221. The feature extraction process performed by the first feature generation unit 222 is the same algorithm as the feature extraction process performed by the first feature generation unit 62. The first estimation model application unit 223 inputs the first feature generated by the first feature generation unit 222 to a first estimation model configured based on parameters of the trained first estimation model stored in the first estimation model information storage unit 41. The first estimation model application unit 223 then supplies a first inner state estimation value output by the first estimation model to which the first feature is input to the second feature generation unit 225.

[0058] The second input data acquisition unit 224 acquires second input data 322 corresponding to one piece of first input data 321 acquired by the first input data acquisition unit 221 (i.e., the subject and observation timing are the same). The second feature generation unit 225 generates second features that are consistent with the input format of the second estimation model, based on the second input data 322 acquired by the second input data acquisition unit 224 and the first inner inner state estimation value supplied from the first estimation model application unit 223. The feature extraction processing executed by the second feature generation unit 225 uses the same algorithm as the feature extraction processing executed by the second feature generation unit 72.

[0059] The second estimation model training unit 226 inputs the second feature quantities generated by the second feature quantity generation unit 225 into a second estimation model configured based on the parameters stored in the second estimation model information storage unit 42, and acquires a second inner state estimate value output by the second estimation model. The second estimation model training unit 226 then updates the parameters of the second estimation model so as to minimize the error (loss) between the second inner state estimate value acquired by the second estimation model and the correct answer indicated by the correct answer data 323 corresponding to the first input data 321 and second input data 322 used to generate the second feature quantities. The second learning unit 22 then performs this process for all pairs of the first input data 321 and the second input data 322, or until a predetermined learning termination condition is satisfied. The algorithm for determining the parameters so as to minimize the loss may be any learning algorithm used in machine learning, such as gradient descent or backpropagation. Note that initial parameters of the second estimation model may be stored in the second estimation model information storage unit 42 before the second estimation model is trained.

[0060] In this way, in the above-described learning method, the internal state estimation device 1 uses, as part of the input to the second estimation model, the output of the first estimation model learned based on first training data 31 that can be generated by objective evaluation by a third party other than the subject (i.e., obtained in a relatively large amount). This allows the internal state estimation device 1 to learn a second estimation model that estimates the second internal state with high accuracy, using second training data 32 that requires subjective evaluation by the subject (i.e., a relatively small amount).

[0061] (6) Processing Flow FIG. 8 is an example of a flowchart showing the processing procedure for estimating the inner surface state by the inner surface state estimating device 1 after learning the first estimation model and the second estimation model.

[0062] The inner state estimation device 1 determines that it is time to estimate the inner state when, for example, a predetermined inner state estimation execution condition is satisfied, and executes the process of the flowchart in FIG. 8 . The inner state estimation device 1 determines that the above-described estimation execution condition is satisfied when, for example, the input device 2 receives an input signal S1 instructing execution of an inner state estimation process. Alternatively, the inner state estimation device 1 may determine whether the estimation execution condition is satisfied by referring to an estimation execution condition pre-stored in the storage device 4 or the like, or may determine that the estimation execution condition is satisfied when a predetermined date and time (e.g., a predetermined time every day) arrives. In yet another example, the inner state estimation device 1 may determine that the estimation execution condition is satisfied when a sensor signal S3 for generating observation data necessary for estimating the inner state is acquired.

[0063] First, the internal state estimation device 1 acquires observation data of the subject at the timing for estimating the internal state (step S11). In this case, the internal state estimation device 1 acquires observation data based on the sensor signal S3 acquired from the sensor 5 via the interface 13. Furthermore, the internal state estimation device 1 extracts, from the acquired observation data, the observation data to be used for the first estimation model as the first observation data, and the observation data to be used for the second estimation model as the second observation data. Note that the first observation data and the second observation data may overlap.

[0064] The inner state estimating device 1 then calculates a first inner state estimation value based on the first observation data acquired in step S11 (step S12). In this case, the inner state estimating device 1 calculates the first inner state estimation value based on the first observation data and a first estimation model configured with reference to the learned parameters stored in the first estimation model information storage unit 41.

[0065] Next, the inner state estimation device 1 calculates a second inner state estimation value based on the second observation data acquired in step S11 and the first inner state estimation value calculated in step S12 (step S13). In this case, the inner state estimation device 1 calculates the second inner state estimation value based on the second estimation model constructed with reference to the learned parameters stored in the second estimation model information storage unit 42, the second observation data, and the first inner state estimation value.

[0066] Next, the inner state estimation device 1 outputs the estimation result of the inner state (step S14). The estimation result of the inner state is not limited to the second inner state estimation value, and may include the first inner state estimation value. In this case, the inner state estimation device 1 supplies an output signal S2 to the output device 3 so that the output device 3 displays or outputs audio representing the estimation result of the inner state. In this case, for example, the output control unit 18 may compare the estimation result of the inner state with a predetermined reference value and, based on the comparison result, notify the subject or their manager of the estimation result of the inner state. This allows the inner state estimation device 1 to preferably present information regarding the estimation result of the subject's inner state to the subject or their manager.

[0067] Furthermore, when the second inner state is a concentration level, the inner state estimating device 1 may output whether the task performed by the subject was a task performed with concentration. Furthermore, the inner state estimating device 1 may treat a task performed with a concentration level below a predetermined level differently from a task performed with a concentration level equal to or higher than the predetermined level (for example, invalidating the test results if the task is a test).

[0068] (7) Modifications Suitable modifications of the above-described embodiment will be described. The modifications may be applied to the above-described embodiment in any combination.

[0069] (Variation 1) The inner state estimation device 1 may use the correct answer data 312 of the first training data 31 as part of the input to the second estimation model, instead of using the first inner state estimation value output by the first estimation model.

[0070] 9 shows an example of detailed functional blocks of the second learning unit 22 according to Modification 1. Functionally, the second learning unit 22 according to Modification 1 includes a second input data acquisition unit 224, a second feature amount generation unit 225, and a second estimation model learning unit 226. Here, the second input data 322 corresponds to second observation data observed from the same subject at the same timing as the correct state indicated by the correct answer data 312.

[0071] The second input data acquisition unit 224 extracts one piece of second input data 322 to be input to the second estimation model from the second training data 32. The second feature generation unit 225 generates second feature amounts based on the second input data 322 acquired by the second input data acquisition unit 224 and correct answer data 312 indicating the correct value of the first inner state at the same time point as the observation of the second input data 322. The second estimation model training unit 226 inputs the second feature amounts generated by the second feature generation unit 225 to a second estimation model configured based on parameters stored in the second estimation model information storage unit 42. The second estimation model training unit 226 then updates the parameters of the second estimation model so as to minimize the error (loss) between the second inner state estimated value output by the second estimation model and the correct answer indicated by the correct answer data 323 corresponding to the second input data 322 used to generate the second feature amounts.

[0072] In this case, preferably, when estimating the subject's inner state, the inner state estimation device 1 also estimates the second inner state using correct answer data 312 that indicates the correct answer for the subject's first inner state at the same time as the second observation data was observed, instead of the first inner state estimation value output by the first estimation model.

[0073] 10 shows an example of detailed functional blocks of the second inner state estimation unit 17 according to Modification 1. Functionally, the second inner state estimation unit 17 according to Modification 1 includes a second observation data acquisition unit 71, a second feature amount generation unit 72, and a second estimation model application unit 73.

[0074] The second observation data acquisition unit 71 acquires the second observation data from the observation data acquisition unit 15, and supplies the acquired second observation data to the second feature generation unit 72. Note that this second observation data is data obtained by observing the subject in the past, and is data observed at the same time as the first inner state of the subject indicated by the ground truth data 312.

[0075] The second feature amount generation unit 72 generates second feature amounts by adding the correct value of the first inner state indicated by the correct data 312 to the feature amounts extracted from the second observation data acquired by the second observation data acquisition unit 71. The second estimation model application unit 73 inputs the second feature amounts supplied from the second feature amount generation unit 72 to a second estimation model configured with reference to the second estimation model information storage unit 42, and acquires a second inner state estimation value output by the second estimation model in response to the input. The second estimation model application unit 73 then supplies the acquired second inner state estimation value to the output control unit 18. Note that the correct value of the first inner state indicated by the correct data 312 is an example of an estimation result of the subject's first inner state.

[0076] In this way, the inner surface state estimating device 1 according to the first modification can preferably learn the second estimation model using the correct answer data 312 and estimate the second inner surface state.

[0077] (Variation 2) The first inner state and the second inner state may be inner states represented by the same indicator. For example, the first inner state and the second inner state may both be represented by indicators relating to joy, anger, sadness, or happiness, with the first inner state being a state of expressed joy, anger, sadness, or happiness, and the second inner state being a state of inner joy, anger, sadness, or happiness.

[0078] In this case, the first inner state estimation unit 16 regards the inner state to be estimated as an expressed first inner state that can be seen by others, and calculates a first inner state estimation value based on a first estimation model trained using objective assessment values ​​as correct data. Meanwhile, the second inner state estimation unit 17 regards the inner state to be estimated as a second inner state of the subject that cannot be accurately determined by others just by looking at it, and calculates a second inner state estimation value based on a second estimation model trained using subjective assessment values ​​as correct data. Furthermore, in the learning process, as in the above-described embodiment, the first learning unit 21 trains the first estimation model using correct data indicating the objective assessment values, and the second learning unit 22 trains the second estimation model using correct data indicating the subjective assessment values. This allows for the training of a second estimation model that estimates the second inner state with high accuracy.

[0079] According to this modification, the inner surface state estimating device 1 can also calculate the second inner surface state estimated value that represents the inner surface state to be estimated with high accuracy.

[0080] Second Embodiment Fig. 11 shows a schematic configuration of an inner state estimating system 100A according to the second embodiment. The inner state estimating system 100A according to the second embodiment is a server-client model system, in which an inner state estimating device 1A functioning as a server device performs the processing of the inner state estimating device 1 according to the first embodiment. Hereinafter, the same components as those in the first embodiment will be appropriately designated by the same reference numerals, and their description will be omitted.

[0081] The inner state estimation system 100A mainly includes an inner state estimation device 1A that functions as a server, a storage device 4 that stores data similar to that in the first embodiment, and a terminal device 8 that functions as a client. The inner state estimation device 1A and the terminal device 8 communicate data via a network 7.

[0082] The terminal device 8 is a terminal having input, display, and communication functions, and functions as the input device 2 and output device 3 shown in Fig. 1. The terminal device 8 may be, for example, a personal computer, a tablet terminal, a PDA (Personal Digital Assistant), etc. The terminal device 8 transmits, to the inner surface state estimation device 1A, biological signals output by sensors (not shown) or input signals based on user input.

[0083] The inner state estimation device 1A has, for example, the same hardware configuration and functional configuration as the inner state estimation device 1. The inner state estimation device 1A receives information, such as that acquired by the inner state estimation device 1 shown in FIG. 1 from the input device 2 and the sensor 5, from the terminal device 8 via the network 7, and estimates the inner state of the subject based on the received information. In addition, the inner state estimation device 1A transmits an output signal indicating information related to the above-mentioned estimation result to the terminal device 8 via the network 7, based on a request from the terminal device 8. That is, in this case, the terminal device 8 functions as the output device 3 in the first embodiment. In this way, the inner state estimation device 1A suitably presents information related to the estimation result of the inner state to the user of the terminal device 8.

[0084] 12 is a block diagram of an inner surface state estimating device 1X according to a third embodiment. The inner surface state estimating device 1X mainly includes an observation data acquiring unit 15X, a first inner surface state acquiring unit 16X, and a second inner surface state estimating unit 17X. Note that the inner surface state estimating device 1X may be configured by a plurality of units.

[0085] The observation data acquisition means 15X acquires observation data obtained by observing a subject. The observation data acquisition means may be, for example, the observation data acquisition unit 15 in the first embodiment (including modifications, the same applies below) or the second embodiment.

[0086] The first inner state acquisition means 16X acquires an estimation result of the first inner state of the subject. The first inner state acquisition means 16X may be, for example, the first inner state estimation unit 16 in the first embodiment (excluding Modification 1) or the second embodiment. Furthermore, the first inner state acquisition means 16X may be the second inner state estimation unit 17 in Modification 1 of the first embodiment that acquires the ground truth data 312 as an estimation result of the first inner state.

[0087] The second inner state estimation means 17X estimates a second inner state of the subject, which is an inner state correlated with the first inner state, based on the observation data and the estimation result of the first inner state. The second inner state estimation means 17X can be, for example, the second inner state estimation unit 17 in the first or second embodiment.

[0088] 13 is an example of a flowchart executed by the inner state estimating device 1X in the third embodiment. The observation data acquiring means 15X acquires observation data obtained by observing the subject (step S21). The first inner state acquiring means 16X acquires an estimation result of the subject's first inner state (step S22). The second inner state estimating means 17X estimates the subject's second inner state, which is an inner state correlated with the first inner state, based on the observation data and the estimation result of the first inner state (step S23).

[0089] According to the third embodiment, the inner state estimating device 1X can accurately estimate the second inner state of the subject based on the estimation result of the first inner state.

[0090] Other Embodiments In each embodiment, the administrator is, for example, a medical professional.

[0091] In each embodiment, the output control unit 18 may output to the administrator, as information regarding advice, information indicating how to deal with the subject based on the estimated result of the subject's inner state (first inner state and / or second inner state).

[0092] In a first example of this case, information indicating the correspondence between the estimation result of the inner state and the countermeasure is stored in the memory 12, the storage device 4, or a device other than the internal state estimation device 1 (not shown), and the output control unit 18 determines the countermeasure for the subject by referring to the information indicating the correspondence, and outputs information indicating the determined countermeasure via the output device 3. Here, the "countermeasure" refers to a method of treatment or other procedure for the subject in accordance with the estimated internal state (including actions taken by the subject himself / herself), and the "information indicating the countermeasure" refers to any information for identifying the countermeasure (such as a name, an identification number, a description of the contents, or a combination thereof).

[0093] In a second example, the output control unit 18 may determine a countermeasure based on a model generated by machine learning the correspondence between the estimation result of the inner state and the countermeasure and the estimation result of the subject's inner state, and output information indicating the determined countermeasure. In this case, trained parameters of the model are stored in the memory 12, the storage device 4, or a device other than the inner state estimation device 1 (not shown). When an estimation result of the inner state is obtained, the output control unit 18 inputs the estimation result of the inner state into a model to which the trained parameters have been applied, determines a countermeasure based on the information output by the model, and outputs information indicating the determined countermeasure via the output device 3.

[0094] The method for determining the countermeasure is not limited to the above-described method. According to the present embodiment, it is possible to provide information for optimizing the actions of the manager who manages the condition of the subject, thereby making it possible to improve the condition of the subject, for example.

[0095] Furthermore, in each embodiment, the output control unit 18 may output, via the output device 3, observation data (first observation data and / or second observation data) that influenced the estimation result of the internal state (the estimation result of the first internal state and / or the estimation result of the second internal state). In this case, for example, if the estimation model that outputs the estimation result of the internal state is configured by a neural network, an attention mechanism is added to the estimation model so that output data before full connection is input. Then, the output control unit 18 outputs, via the output device 3, observation data corresponding to feature quantities whose coefficients output by the attention mechanism are equal to or greater than a predetermined value, as observation data that influenced the estimation result of the internal state. This makes it possible to provide the administrator with information to optimize their actions. For example, if the administrator determines that the output estimation result is inappropriate, the administrator refers to the observation data that influenced the estimation result of the first internal state and / or the estimation result of the second internal state, output by the output device 3. This allows the administrator to take action to improve the measurement state of the biological signal from the sensor 5 related to the observation data, based on the referenced observation data.

[0096] In the above-described embodiments, the program can be stored using various types of non-transitory computer-readable media and supplied to a computer processor or the like. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program may also be supplied to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media can be supplied to a computer via wired communication paths such as electric wires and optical fibers, or via wireless communication paths.

[0097] In addition, part or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0098] [Supplementary Note 1] An inner state estimation device comprising: observation data acquisition means for acquiring observation data obtained by observing a subject; first inner state acquisition means for acquiring an estimation result of a first inner state of the subject; and second inner state estimation means for estimating a second inner state of the subject, the second inner state being an inner state correlated with the first inner state, based on the observation data and the estimation result. [Supplementary Note 2] The inner state estimation device of Supplementary Note 1, wherein the first inner state is an inner state in which a physiological phenomenon of the subject is observed. [Supplementary Note 3] The inner state estimation device of Supplementary Note 1, wherein the first inner state acquisition means calculates the estimation result based on first observation data, which is at least a part of the observation data, and a first estimation model, and the first estimation model outputs an estimation result of the first inner state when the first observation data is observed, when the first observation data or feature quantities of the first observation data are input. [Supplementary Note 4] The inner state estimation device of Supplementary Note 3, wherein the first estimation model is a model trained based on training data including ground truth data of the first inner state generated based on objective evaluation. [Supplementary Note 5] The inner state estimation device of Supplementary Note 4, wherein the estimation result is the ground truth data, and the second inner state estimation means estimates the second inner state based on the observation data and the ground truth data. [Supplementary Note 6] The inner state estimation device of Supplementary Note 1, wherein the second inner state is an inner state resulting from a confusion between a state in which a physiological phenomenon of the subject is observed and a state in which the physiological phenomenon is not observed. [Supplementary Note 7] The inner state estimation device of Supplementary Note 1, wherein the second inner state estimation means estimates the second inner state based on second observation data that is at least a part of the observation data, the estimation result, and a second estimation model, and when the second observation data or feature quantities of the second observation data and the estimation result are input, the second estimation model outputs an estimation result of the second inner state when the second observation data is observed. [Supplementary Note 8] The inner state estimation device of Supplementary Note 7, wherein the second estimation model is a model trained based on training data including ground truth data generated based on subjective evaluation.[Supplementary Note 9] The inner state estimating device of Supplementary Note 1, wherein the first inner state acquisition means calculates the estimation result based on first observation data that is at least a part of the observation data, and the second inner state estimating means estimates the second inner state based on second observation data that is at least a part of the observation data and the estimation result, and the second observation data includes the first observation data. [Supplementary Note 10] The inner state estimating device of Supplementary Note 1, wherein the second inner state estimating means estimates a concentration level as the second inner state. [Supplementary Note 11] The inner state estimating device of Supplementary Note 1, wherein the first inner state and the second inner state are inner states represented by the same index. [Supplementary Note 12] The inner state estimating device of Supplementary Note 1, further comprising output control means for outputting information regarding the second inner state. [Supplementary Note 13] The inner state estimating device according to Supplementary Note 12, wherein the output control means outputs at least one of a countermeasure based on the first inner state and a countermeasure based on the second inner state, for optimizing the behavior of at least one of the subject and the subject's manager. [Supplementary Note 14] An inner state estimating method, wherein a computer acquires observation data obtained by observing the subject, acquires an estimation result of the subject's first inner state, and estimates a second inner state of the subject, which is an inner state that is correlated with the first inner state, based on the observation data and the estimation result. [Supplementary Note 15] A storage medium storing a program that causes a computer to execute the processes of acquiring observation data obtained by observing the subject, acquiring an estimation result of the subject's first inner state, and estimating a second inner state of the subject, which is an inner state that is correlated with the first inner state, based on the observation data and the estimation result.

[0099] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications within the scope of the present invention that would be understood by those skilled in the art can be made to the configuration and details of the present invention. In other words, the present invention naturally includes various modifications and alterations that would be possible for those skilled in the art based on the entire disclosure, including the claims, and the technical ideas. Furthermore, the disclosures of the above-cited patent documents and other documents are incorporated herein by reference.

[0100] For example, it can be used for services related to management (including self-management) based on inner states.

[0101] 1, 1A, 1X Inner surface state estimation device 2 Input device 3 Output device 4 Storage device 5 Sensor 8 Terminal device 100, 100A Inner surface state estimation system

Claims

1. Observation data acquisition means for acquiring observation data obtained by observing a subject; a first inner state acquisition means for acquiring an estimation result of a first inner state of the subject; a second inner state estimating means for estimating a second inner state of the subject, the second inner state being an inner state correlated with the first inner state, based on the observed data and the estimation result; An inner surface state estimation device comprising:

2. The inner state estimating device according to claim 1 , wherein the first inner state is an inner state in which a manifestation of a physiological phenomenon of the subject is observed.

3. the first inner surface state acquisition means calculates the estimation result based on first observation data which is at least a part of the observation data and a first estimation model; 2. The inner surface state estimation device according to claim 1, wherein the first estimation model outputs an estimation result of the first inner surface state at the time when the first observation data was observed, when the first observation data or a feature amount of the first observation data is input.

4. The inner surface state estimating device according to claim 3 , wherein the first estimation model is a model trained based on training data including ground-truth data of the first inner surface state generated based on an objective evaluation.

5. The estimation result is the correct answer data, The inner surface state estimating device according to claim 4 , wherein said second inner surface state estimating means estimates said second inner surface state based on said observed data and said corrective data.

6. The inner state estimating device according to claim 1 , wherein the second inner state is an inner state in which a state in which a manifestation of a physiological phenomenon of the subject is observed is confused with a state in which the manifestation is not observed.

7. the second inner surface state estimation means estimates the second inner surface state based on second observation data which is at least a part of the observation data, the estimation result, and a second estimation model; 2. The inner surface state estimation device according to claim 1, wherein, when the second observation data or a feature of the second observation data and the estimation result are input, the second estimation model outputs an estimation result of the second inner surface state at the time when the second observation data was observed.

8. The inner surface state estimating device according to claim 7 , wherein the second estimation model is a model trained on the basis of training data including ground truth data generated on the basis of a subjective evaluation.

9. The first inner surface state acquisition means calculates the estimation result based on first observation data which is at least a part of the observation data; the second inner surface state estimation means estimates the second inner surface state based on second observation data which is at least a part of the observation data and the estimation result; The inner surface state estimating device according to claim 1 , wherein the second observation data includes the first observation data.

10. The inner state estimating device according to claim 1 , wherein said second inner state estimating means estimates a degree of concentration as said second inner state.

11. The inner state estimating device according to claim 1 , wherein the first inner state and the second inner state are inner states represented by a same index.

12. The inner surface state estimating device according to claim 1 , further comprising an output control means for outputting information relating to the second inner surface state.

13. The output control means outputs at least one of a countermeasure based on the first inner state and a countermeasure based on the second inner state for optimizing a behavior of at least one of the subject and a manager of the subject. The inner surface state estimating device according to claim 12.

14. The computer Obtaining observational data of the subject; obtaining an estimation result of a first inner state of the subject; estimating a second inner state of the subject, the second inner state being an inner state correlated with the first inner state, based on the observed data and the estimation result; Inner state estimation method.

15. Obtaining observational data of the subject; obtaining an estimation result of a first inner state of the subject; A program that causes a computer to execute a process of estimating a second inner state of the subject, the second inner state being an inner state that is correlated with the first inner state, based on the observed data and the estimation result.