Stress estimation device, stress estimation method, and program

Abstract

This stress estimation device 1X mainly comprises a division means 15X, a first feature amount calculation means 16X, a second feature amount calculation means 17X, and a stress estimation means 18X. The division means 15X divides observation data that show the states of a subject in chronological order. The first feature amount calculation means 16X calculates a first feature amount that is a feature amount of the divided observation data on the basis of a learned feature amount extraction model. The second feature amount calculation means 17X calculates a second feature amount on the basis of a plurality of the first feature amounts. The stress estimation means 18X estimates a stress of the subject on the basis of the second feature amount.

Description

[Technical Field]

[0001] The present disclosure relates to the technical fields of a stress estimation device, a stress estimation method, and a storage medium that perform processing related to estimation of a stress state. [Background technology]

[0002] There are known devices or systems that determine the stress state of a subject based on data measured from the subject. For example, Patent Document 1 discloses a stress estimation device that estimates a subject's stress value (stress level) based on the subject's biological data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication WO2019 / 159252 Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, the features extracted to estimate the stress level of a subject from the biometric data of the subject are manually designed. However, there is a problem in that such manually designed features cannot extract sufficiently characteristic information from the biometric data.

[0005] In view of the above-mentioned problems, one of the objects of the present disclosure is to provide a stress estimation device, a learning method, a stress estimation method, and a storage medium that can suitably estimate the stress of a subject. [Means for solving the problem]

[0006] One aspect of the stress estimation device is A division means for dividing observation data representing a time-series state of a subject; a first feature calculation means for calculating a first feature, which is a feature of the divided observation data, based on the trained feature extraction model; a second feature amount calculation means for calculating a second feature amount based on the plurality of first feature amounts; stress estimation means for estimating stress of the subject based on the second feature amount; With death, The first feature amount calculation means a conversion means for converting each of the divided observation data into model input data that is data that matches an input format of the feature extraction model; a model application means for inputting the model input data into the feature extraction model and acquiring, as the first feature, a feature output from the feature extraction model; having It is a stress estimation device. Another aspect of the stress estimation device is A division means for dividing observation data representing a time-series state of a subject; a first feature calculation means for calculating a first feature, which is a feature of the divided observation data, based on the trained feature extraction model; a second feature amount calculation means for calculating a second feature amount based on the plurality of first feature amounts; stress estimation means for estimating stress of the subject based on the second feature amount; and the first feature calculation means performs dimension reduction of the feature output by the feature extraction model based on a dimension reduction model that reduces the dimension of input data, and calculates the feature after the dimension reduction as the first feature; The dimension reduction model is trained based on observation data in a stressed state and observation data in a non-stressed state. It is a stress estimation device.

[0007] One aspect of the stress estimation method includes: The computer Divide the observation data that represents the subject's state over time, converting each of the divided observation data into model input data that is data that matches the input format of the trained feature extraction model; The model input data is input to the feature extraction model, and the feature output from the feature extraction model is The feature quantity of the divided observation data is Obtained as the first feature, calculating a second feature amount based on the plurality of first feature amounts; Estimating stress of the subject based on the second feature amount. The stress estimation method is a method for estimating stress. Note that the "computer" includes any electronic device (or a processor included in an electronic device), and may be configured by a plurality of electronic devices.

[0008] One aspect of the program is Divide the observation data that represents the subject's state over time, converting each of the divided observation data into model input data that is data that matches the input format of the trained feature extraction model; The model input data is input to the feature extraction model, and the feature output from the feature extraction model is The feature quantity of the divided observation data is Obtained as the first feature, calculating a second feature amount based on the plurality of first feature amounts; The program causes a computer to execute a process of estimating stress of the subject based on the second feature amount. [Effects of the Invention]

[0009] The subject's stress can be estimated with high accuracy. [Brief explanation of the drawings]

[0010] [Figure 1] 1 shows a schematic configuration of a stress estimation system according to a first embodiment. [Figure 2] 1 shows an example of a hardware configuration of a stress estimation device common to each embodiment. [Figure 3] 2 is an example of a functional block of the stress estimation device according to the first embodiment. [Figure 4] FIG. 2 is a detailed functional block diagram of a short-time feature calculation unit; [Figure 5] 10A and 10B are diagrams each showing a schematic example of processing by a short-time feature calculation unit; [Figure 6] 10 is an example of a flowchart illustrating a procedure of a stress estimation process executed by the stress estimation device. [Figure 7] FIG. 10 is a functional block diagram of a short-time feature calculation unit in Modification 1. [Figure 8] FIG. 11 is a functional block diagram of a short-time feature calculation unit in Modification 2. [Figure 9] FIG. 1 is a functional block diagram relating to the training of a dimensionality reduction model. [Figure 10] 10 shows a schematic configuration of a stress estimation system according to a second embodiment. [Figure 11] FIG. 10 is a block diagram of a stress estimation device according to a third embodiment. [Figure 12] 11 is an example of a flowchart executed by the stress estimation device in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of a stress estimation device, a stress estimation method, and a storage medium will be described with reference to the drawings.

[0012] First Embodiment (1) System Configuration FIG. 1 shows a schematic configuration of a stress estimation system 100 according to a first embodiment. The stress estimation system 100 is a system that estimates the stress of a subject based on data observed from the subject, and mainly comprises a stress estimation device 1, an input device 2, a display device 3, a storage device 4, and a sensor 5. The "subject" is a person who is the subject of stress estimation, and may be an athlete or employee whose stress state is managed by an organization, or an individual user. The stress estimated in this embodiment is assumed to be chronic stress, which is stress from a long-term (chronic) perspective spanning several days to weeks or months.

[0013] The stress estimation device 1 estimates the stress state of a subject (specifically, a stress value indicating the degree of stress). In this case, the stress estimation device 1 communicates data related to the stress estimation process with the input device 2, display device 3, and sensor 5 via a communication network or by direct wireless or wired communication. For example, the stress estimation device 1 receives an input signal "S1" from the input device 2. The stress estimation device 1 also receives observation data "S3" indicating the observation results of the sensor 5, which observes the subject. The stress estimation device 1 also generates a display signal "S2" based on the estimation result of the subject's stress value and supplies the generated display signal S2 to the display device 3.

[0014] The input device 2 is an interface that accepts user input (manual input) of information about each subject. The user who inputs information using the input device 2 may be the subject himself / herself, or a person who manages or supervises the subject's activities. The input device 2 may be, for example, various user input interfaces such as a touch panel, buttons, a keyboard, a mouse, or a voice input device. The input device 2 supplies an input signal S1 generated based on the user's input to the stress estimation device 1. The display device 3 displays predetermined information based on a display signal S2 supplied from the stress estimation device 1. The display device 3 is, for example, a display or a projector.

[0015] The sensor 5 generates observation data S3, which is data obtained by observing the subject, and supplies the generated observation data S3 to the stress estimation device 1. In this case, the observation data S3 is, for example, any biological signal (including vital information) of the subject, such as the subject's heart rate, brain waves, sweat rate, hormone secretion, cerebral blood flow, blood pressure, body temperature, electromyography, respiratory rate, pulse wave, and acceleration. In this case, the sensor 5 may be a sensor provided in a wearable device worn by the subject, a camera that captures images of the subject, a microphone that generates audio signals of the subject's speech, or a sensor provided in a device such as a personal computer or smartphone operated by the subject. For example, the wearable device described above may include a GNSS (global navigation satellite system) receiver, an acceleration sensor, or any other sensor that detects biological signals, and outputs the output signals of these sensors as the observation data S3. The sensor 5 may also supply information corresponding to the amount of operation of the personal computer, smartphone, or the like to the stress estimation device 1 as the observation data S3. The sensor 5 may also output observation data S3 representing biological data (including sleeping hours) from the subject while the subject is sleeping. Date and time information (timestamp) representing the date and time of observation is associated with the observation data S3 by the sensor 5 or the stress estimation device 1 that receives the observation data S3.

[0016] The storage device 4 is a memory that stores various information necessary for estimating a stress state, etc. The storage device 4 may be an external storage device such as a hard disk connected to or built into the stress estimation device 1, or may be a storage medium such as a flash memory. The storage device 4 may also be a server device that performs data communication with the stress estimation device 1. The storage device 4 may also be composed of multiple devices.

[0017] The storage device 4 functionally comprises a short-term feature storage unit 41, a feature extraction model storage unit 42, and a stress estimation model storage unit 43.

[0018] The short-term feature storage unit 41 stores short-term features, which are features of data obtained by dividing the observation data S3 into data observed per unit time (also referred to as "unit observation data"). Here, the above-mentioned unit time is set to an arbitrary length of time shorter than the observation period of the subject (also referred to as the "required observation period") required to estimate the subject's stress level (i.e., the degree of chronic stress). The required observation period is, in other words, a period that affects the stress to be estimated. For example, when one month's worth of observation data S3 is used to estimate the subject's stress level (i.e., the required observation period is one month immediately preceding the time of stress estimation), the above-mentioned unit time is set to several minutes (e.g., one minute). The stress estimation device 1 calculates short-term features for each unit observation data obtained by dividing the observation data S3 received from the sensor 5, and stores the calculated short-term features in the short-term feature storage unit 41. In this case, as will be described later, the stress estimation device 1 converts the unit observation data into a data format (tensor, which is an image in this embodiment) that can be input to a feature extraction model whose learned parameters are stored in the feature extraction model storage unit 42, and calculates the feature extracted by the feature extraction model from the converted data as a short-term feature. The short-term feature is an example of a "first feature."

[0019] The feature extraction model storage unit 42 stores parameters of a feature extraction model used to calculate short-time features (in other words, parameters necessary to configure the feature extraction model). In this embodiment, the feature extraction model is a model trained to output features (feature vectors) representing the features of an image when an image of a predetermined size is input. The feature extraction model may use the architecture of any general feature extraction model for image recognition. For example, various deep learning models such as VGG16, VGG19, and MobileNet exist as such feature extraction models. Note that various forms of feature extraction models (feature extractors) that receive images as input have been proposed, and any of these forms may be used. The feature extraction model storage unit 42 stores trained parameters of the feature extraction model. For example, if the feature extraction model is based on a neural network, the feature extraction model storage unit 42 stores information on various parameters, such as the layer structure, the neuron structure of each layer, the number and size of filters in each layer, and the weights of each element of each filter.

[0020] The feature extraction model used in this embodiment may be trained so as to extract features suitable for this embodiment. In this case, for example, the feature extraction model is trained using, as input data, images obtained by converting unit observation data prepared as training data, and parameters of the feature extraction model obtained by training are stored in advance (i.e., before stress estimation for the subject) in the feature extraction model storage unit 42.

[0021] The stress estimation model storage unit 43 stores parameters of the stress estimation model, which is a model for estimating the stress value of a subject (in other words, parameters necessary to configure the stress estimation model). Here, the features input to the stress estimation model are features representing statistics of short-term features generated from unit observation data showing observation results during the required observation period, and will hereinafter also be referred to as "stress features." Note that when the short-term features and stress features are in vector format, a vector representing statistics for each element of the short-term features (e.g., average value) is calculated as the stress feature. When the required observation period is the one month immediately preceding the time of stress estimation, the stress feature is calculated using short-term features generated from unit observation data showing observation results during that one month. The stress feature is an example of a "second feature."

[0022] The stress estimation model is a model that estimates the relationship between a subject's stress feature amount and the subject's stress level, and is trained in advance to output an estimate of the subject's stress level when the subject's stress feature amount is input. Here, the stress estimation model may be any machine learning model (including a statistical model) such as a neural network or a support vector machine. For example, if the stress estimation model is a model based on a neural network such as a convolutional neural network, the stress estimation model storage unit 43 stores information on various parameters such as the layer structure, the neuron structure of each layer, the number and filter size of filters in each layer, and the weight of each element of each filter.

[0023] The stress estimation model may be a model trained for each predetermined classification of subjects. In this case, parameters of each model for each classification are stored in the stress estimation model storage unit 43. In this case, the above-mentioned classification may be performed based on the attributes of the subjects and / or the type of observation data S3 used for stress estimation. Here, classification based on attributes is, for example, classification based on personality, gender, occupation, race, age, height, weight, muscle mass, stress tolerance, lifestyle habits, exercise habits, cognitive tendencies, or a combination thereof. When performing stress estimation, the stress estimation device 1 determines the stress estimation model to be used based on the attribute information of the subjects and / or type information of the observation data S3. In this way, a stress estimation model may be trained and used for each group with a bias in stress tendency.

[0024] The configuration of the stress estimation system 100 shown in FIG. 1 is an example, and various modifications may be made to the configuration. For example, the input device 2 and the display device 3 may be configured as an integrated device. In this case, the input device 2 and the display device 3 may be configured as a tablet terminal that is integrated with or separate from the stress estimation device 1. In this case, the stress estimation device 1, the input device 2, the display device 3, and the sensor 5 (and may include the storage device 4) may be configured as a single smartphone or wearable terminal used by the subject. The stress estimation device 1 may also be configured as a plurality of devices. In this case, the plurality of devices that configure the stress estimation device 1 exchange information required to execute pre-assigned processing between these plurality of devices.

[0025] (2) Hardware configuration of the stress estimation device 2 shows the hardware configuration of the stress estimation device 1. The stress estimation device 1 includes, as hardware, a processor 11, a memory 12, and an interface 13. The processor 11, the memory 12, and the interface 13 are connected via a data bus 90.

[0026] The processor 11 executes a program stored in the memory 12 to function as a controller (arithmetic unit) that performs overall control of the stress estimation device 1. The processor 11 is, for example, a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a TPU (Tensor Processing Unit). The processor 11 may be composed of multiple processors. The processor 11 is an example of a computer.

[0027] The memory 12 is configured by various types of volatile and non-volatile memories such as RAM (Random Access Memory), ROM (Read Only Memory), and flash memory. The memory 12 also stores programs for executing the processes performed by the stress estimation device 1. Note that part of the information stored in the memory 12 may be stored in one or more external storage devices capable of communicating with the stress estimation device 1, or may be stored in a storage medium that is detachable from the stress estimation device 1.

[0028] The interface 13 is an interface for electrically connecting the stress estimation device 1 to other devices. These interfaces may be wireless interfaces such as network adapters for wirelessly transmitting and receiving data to and from other devices, or may be hardware interfaces for connecting to other devices via cables or the like.

[0029] The hardware configuration of the stress estimation device 1 is not limited to the configuration shown in Fig. 2. For example, the stress estimation device 1 may include at least one of an input device 2 or a display device 3. Furthermore, the stress estimation device 1 may be connected to or built-in with a sound output device such as a speaker.

[0030] (3) Stress Estimation Overview In general, the stress estimation device 1 calculates short-term features by image-converting unit observation data obtained by dividing the observation data S3, and estimates the stress level of the subject based on the stress features calculated by statistically processing the multiple short-term features. In this case, the stress estimation device 1 calculates the short-term features using a trained feature extraction model that employs an architecture developed for the purpose of image recognition, etc., thereby acquiring short-term features useful for stress estimation and achieving highly accurate stress estimation.

[0031] Fig. 3 shows an example of functional blocks of the stress estimation device 1. Functionally, the processor 11 of the stress estimation device 1 has a division unit 15, a short-term feature calculation unit 16, a stress feature calculation unit 17, and a stress estimation unit 18. Note that in Fig. 3, blocks where data is exchanged are connected by solid lines, but the combination of blocks where data is exchanged is not limited to that shown. The same applies to other functional block diagrams described later.

[0032] The dividing unit 15 acquires the observation data S3 of the subject supplied from the sensor 5 via the interface 13, and generates unit observation data by dividing the observation data S3 into data for each unit time. In other words, the dividing unit 15 extracts unit observation data from the observation data S3 by applying a short-time window set to the length of the above-mentioned unit time to the observation data S3. Note that the observation data S3 generated by the sensor 5 may be temporarily stored in the storage device 4 or memory 12, and then acquired and divided by the dividing unit 15. The dividing unit 15 supplies the generated unit observation data to the short-term feature calculation unit 16.

[0033] The short-term feature calculation unit 16 converts each unit observation data supplied from the division unit 15 into a short-term feature. In this case, the short-term feature calculation unit 16 converts the unit observation data so that it conforms to the input format of the feature extraction model. Thereafter, the short-term feature calculation unit 16 constructs a trained feature extraction model by referring to the feature extraction model storage unit 42, and inputs data (images in this embodiment) converted from the unit observation data into the feature extraction model. The short-term feature calculation unit 16 then associates the short-term features output by the feature extraction model with date and time information included in the corresponding unit observation data and stores them in the short-term feature storage unit 41. The short-term feature calculation unit 16 also identifies missing data (i.e., data that does not properly represent the subject's condition) from the data converted to conform to the input format of the feature extraction model, and performs processing to exclude the missing data from the data to be input to the feature extraction model. Details of the processing by the short-term feature calculation unit 16 will be described later.

[0034] When estimating the stress of a subject, the stress feature calculation unit 17 acquires short-term features corresponding to dates and times within the required observation period from the short-term feature storage unit 41 and calculates stress features based on the acquired short-term features. The stress features are in vector format, and each element of the stress feature is a statistical quantity such as the average of the corresponding element of the acquired short-term features. In this case, the stress feature calculation unit 17 may calculate features (also referred to as "daily features") representing the statistical quantity of the short-term features for each observation day, and then calculate stress features representing the statistical quantity of the daily features corresponding to days belonging to the required observation period. In this way, the stress feature calculation unit 17 may calculate stress features by performing multi-stage statistical processing on the target short-term features. The stress feature calculation unit 17 then supplies the calculated stress features to the stress estimation unit 18.

[0035] The stress estimation unit 18 calculates an estimate of the subject's stress (also referred to as an "estimated stress value") based on the stress feature calculated by the stress feature calculation unit 17. In this case, the stress estimation unit 18 inputs the stress feature to a stress estimation model configured by referring to the stress estimation model storage unit 43, and acquires the resulting stress value output by the stress estimation model as the estimated stress value. Note that if the stress estimation model storage unit 43 stores parameters of multiple stress estimation models, the stress estimation unit 18 may select the stress estimation model to use based on attribute information of the subject and / or the type of observation data S3 used for stress estimation, etc.

[0036] The stress estimation unit 18 may also control the display of the display device 3 regarding the calculated stress estimate value. For example, the stress estimation unit 18 generates a display signal S2 for displaying information regarding the calculated stress estimate value and supplies the display signal S2 to the display device 3, thereby causing the display device 3 to display the information regarding the stress estimate value. In another example, instead of or in addition to controlling the display of the stress estimate value itself, the stress estimation unit 18 controls the display of information regarding the stress level determined based on a comparison between the stress estimate value and a predetermined threshold, and / or information regarding advice corresponding to the stress level. In this case, the viewer of the display device 3 may be, for example, the subject, or a person managing or supervising the subject's stress state. The stress estimation unit 18 may also output the information regarding the stress estimate value as audio using an audio output device (not shown).

[0037] The components of the division unit 15, short-term feature calculation unit 16, stress feature calculation unit 17, and stress estimation unit 18 described in FIG. 3 can be realized, for example, by the processor 11 executing a program. Alternatively, the necessary programs may be recorded in any nonvolatile storage medium and installed as needed to realize the components. At least some of these components may not necessarily be realized by software programs, but may be realized by any combination of hardware, firmware, and software. At least some of these components may be realized using a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcontroller. In this case, the integrated circuit may be used to realize a program consisting of the above components. At least some of the components may be configured by an ASSP (Application Specific Standard Produce), an ASIC (Application Specific Integrated Circuit), or a quantum processor (quantum computer control chip). In this way, the components may be realized by various types of hardware. The same applies to other embodiments described below. Furthermore, each of these components may be realized by the cooperation of multiple computers, for example, using cloud computing technology.

[0038] (4) Details of the short-term feature calculation section 4 is an example of a detailed functional block diagram of the short-term feature calculation unit 16. Functionally, the short-term feature calculation unit 16 has an image conversion unit 61, a missing data removal unit 62, and a model application unit 63.

[0039] The image converter 61 converts each piece of unit observation data generated by dividing the observation data S3 into an image of a predetermined size (also referred to as a "model input image") so as to match the input format of the feature extraction model. Here, the image converter 61 generates, as the model input image, a spectrogram image representing the frequency characteristics of the unit observation data. The spectrogram image is, for example, an image with time on the horizontal axis and frequency on the vertical axis, and with pixel values ​​representing amplitudes (intensities) corresponding to the corresponding times and frequencies. In this case, to calculate pixel values ​​for each vertical column, the image converter 61 sets a window based on the time corresponding to the column for which pixel values ​​are to be calculated, and performs a Fourier transform on the unit observation data within the window. The image converter 61 then slides the window according to the column for which pixel values ​​are to be calculated, and calculates pixel values ​​(frequency intensity) corresponding to each time (column). The window is set to a length shorter than the time length of the unit observation data, for example, half the time length of the unit observation data. The model input image is an example of "model input data."

[0040] In this way, the image conversion unit 61 can convert each unit observation data into data that is consistent with the input format of the feature extraction model. Note that instead of generating a spectrogram image, the image conversion unit 61 may generate, as a model input image, an image representing a graph with time on the horizontal axis and the magnitude of the observation value (index value) for each time on the vertical axis from the unit observation data. Furthermore, the image conversion unit 61 may generate data by converting the unit observation data into a tensor with any number of dimensions other than an image that is a two-dimensional tensor.

[0041] The missing data elimination unit 62 detects missing data, which is data that does not adequately reflect the subject's condition, from the model input image generated by the image conversion unit 61 and performs a process of excluding the identified missing data. Missing data is a model input image generated from unit observation data (e.g., data with constant observation values) that does not substantially observe the subject. Furthermore, if short-term features based on such missing data are included in stress estimation, the accuracy of stress estimation will decrease. In consideration of the above, the missing data elimination unit 62 determines that model input images that satisfy a predetermined condition are missing data and supplies model input images other than the missing data to the model application unit 63. The predetermined condition may be any condition that is deemed to be missing data, such as a condition based on the variance of pixel values ​​of the model input image (e.g., the variance is less than a predetermined value). The model application unit 63 may also detect missing data using a discriminative model trained to identify whether an input image is missing data. In this case, the trained parameters of the above-mentioned discrimination model are stored in the storage device 4 or the like, and the model application unit 63 inputs a spectrogram image to a discrimination model configured based on the trained parameters, and determines whether the input image is missing data based on the discrimination result output from the discrimination model. Note that the process of excluding missing data may, of course, be performed on unit observation data.

[0042] The model application unit 63 inputs each of the model input images other than the missing data supplied from the missing data removal unit 62 to a feature extraction model configured using the trained parameters stored in the feature extraction model storage unit 42. The model application unit 63 then associates the features output by the feature extraction model with date and time information included in the corresponding unit observation data, and stores them in the short-term feature storage unit 41 as short-term features.

[0043] 5 is a diagram schematically illustrating a specific example of processing by the short-term feature calculation unit 16. In this example, observation data S3, whose horizontal axis represents time, is divided into unit observation data including data "d1," "d2," "d3," ..., "dn," and the image conversion unit 61 converts each of these unit observation data into a model input image. Note that, as representative examples of model input images, model input image "Im2" corresponding to unit observation data d2, which is observation data with a length of x minutes (x is a positive number) from time "t1," and model input image "Imn" corresponding to unit observation data dn, which is observation data with a length of x minutes from time "t2," are shown.

[0044] The missing data excluding unit 62 then determines that the model input image Im2 does not satisfy the above-mentioned predetermined condition and is not missing data (i.e., is normal data), and the model application unit 63 calculates short-term features by inputting the model input image Im2 to the trained feature extraction model. On the other hand, the missing data excluding unit 62 determines that the model input image Imn satisfies the above-mentioned predetermined condition and is missing data, and deletes the model input image Imn without supplying it to the model application unit 63.

[0045] In this way, the short-term feature calculation unit 16 can convert unit observation data into a model input image that matches the input format of the trained feature extraction model, thereby suitably generating short-term features to which a feature extraction model used for image recognition, etc. Furthermore, the short-term feature calculation unit 16 can suppress the generation of short-term features based on unit observation data that corresponds to missing parts in the observation data S3 that do not appropriately represent the state of the subject.

[0046] (5) Processing Flow 6 is an example of a flowchart showing the procedure of the stress estimation process executed by the stress estimation device 1. The stress estimation device 1 repeatedly executes the process of the flowchart in FIG.

[0047] First, the stress estimation device 1 acquires observation data S3 from the sensor 5 (step S11). Then, the stress estimation device 1 divides the observation data S3 into unit observation data, which are data per unit time (step S12). Then, the stress estimation device 1 converts each unit observation data into a model input image (step S13). Then, the stress estimation device 1 calculates short-term features from the model input image using a feature extraction model based on learned parameters stored in the feature extraction model storage unit 42, and stores the calculated short-term features in association with date and time information in the short-term feature storage unit 41 (step S14). In this case, the stress estimation device 1 may determine whether each model input image generated in step S13 is missing data, and exclude model input images determined to be missing data from targets to be input to the feature extraction model.

[0048] The stress estimation device 1 then determines whether it is time to calculate an estimated stress value (step S15). For example, the stress estimation device 1 determines that it is time to calculate an estimated stress value when it detects a user input instructing calculation of an estimated stress value, or when it is time to calculate a predetermined estimated stress value. If it is not time to calculate an estimated stress value (step S15; No), the stress estimation device 1 returns to step S11, receives observation data S3, and processes the received observation data S3.

[0049] If it is time to calculate a stress estimate (step S15; Yes), the stress estimation device 1 calculates a stress feature based on the short-term feature corresponding to the observation data S3 generated during the required observation period (step S16). The stress estimation device 1 then calculates a stress estimate based on the stress feature calculated in step S16 and outputs information related to the stress estimate (step S17). In this case, the stress estimation device 1 inputs the stress feature into a stress estimation model based on parameters stored in the stress estimation model storage unit 43, and outputs the stress value output by the model as the stress estimate for the subject.

[0050] (6) Variations Preferred modifications of the above-described embodiment will be described below. The following modifications may be applied to the above-described embodiment in any combination.

[0051] (Variation 1) The stress estimation device 1 may estimate stress by using, as short-term features, predetermined types of features extracted from unit observation data, in addition to short-term features based on the trained feature extraction model.

[0052] Fig. 7 is a functional block diagram of the short-term feature calculation unit 16 in Modification 1. As shown in Fig. 7, the short-term feature calculation unit 16 according to the modification includes an image conversion unit 61, a missing data removal unit 62, a model application unit 63, and a predetermined feature extraction unit 64. The image conversion unit 61, the missing data removal unit 62, and the model application unit 63 generate short-term features by the same processing as in the above-described embodiment.

[0053] The predetermined feature extraction unit 64 calculates a predetermined type of feature from each of the generated unit observation data and stores the calculated feature as a short-time feature in the short-time feature storage unit 41. Here, the predetermined feature extraction unit 64 calculates, for example, the mean value, variance / standard deviation, maximum value, minimum value, quartile, or a combination thereof of the unit observation data as the feature. In this way, the feature calculated by the predetermined feature extraction unit 64 is a predetermined type (i.e., a predetermined type) of feature extracted without using a trained feature extraction model. Note that the predetermined feature extraction unit 64 may regard a feature that satisfies a predetermined condition among the calculated short-time features as a feature generated based on unit observation data corresponding to a missing portion and discard the feature without storing it in the short-time feature storage unit 41.

[0054] When the same unit observation data is used, the short-term features calculated by the model application unit 63 and the short-term features calculated by the specified feature extraction unit 64 are associated with the same date and time information and stored in the short-term feature storage unit 41.

[0055] Here, a supplementary explanation will be given of the stress estimation method using these short-term features. In a first example, the stress feature calculation unit 17 combines these short-term features associated with the same date and time information by increasing the number of dimensions of the vector, and calculates a stress feature using the combined short-term feature. In this case, the number of dimensions of the combined short-term feature is the sum of the number of dimensions of the short-term feature calculated by the model application unit 63 and the number of dimensions of the short-term feature calculated by the predetermined feature extraction unit 64. Thereafter, the stress estimation unit 18 calculates a stress estimation value by using the calculated stress feature as input data for the stress estimation model.

[0056] In a second example, the stress feature calculation unit 17 calculates stress features for each of the short-term features calculated by the model application unit 63 and the short-term features calculated by the predetermined feature extraction unit 64, and combines these calculated stress features. The stress estimation unit 18 then uses the combined stress feature as input data for a stress estimation model to calculate a stress estimated value. In this case, the number of dimensions of the combined stress feature is the sum of the number of dimensions of the stress feature based on the short-term features calculated by the model application unit 63 and the number of dimensions of the stress feature based on the short-term features calculated by the predetermined feature extraction unit 64. Note that the stress estimation unit 18 may calculate a stress value by applying a stress estimation model to each of the stress feature based on the short-term features calculated by the model application unit 63 and the stress feature based on the short-term features calculated by the predetermined feature extraction unit 64, and then calculate the stress estimated value as the average of the calculated stress values.

[0057] (Variation 2) The stress estimation device 1 may further perform a process of reducing the number of dimensions of the feature amounts output by the feature amount extraction model, and calculate the feature amounts after the dimension reduction as short-term feature amounts.

[0058] 8 is a functional block diagram of the short-term feature calculation unit 16 in Modification 2. The short-term feature calculation unit 16 in Modification 2 includes an image conversion unit 61, a missing data removal unit 62, a model application unit 63, and a dimension reduction unit 65. The image conversion unit 61, the missing data removal unit 62, and the model application unit 63 perform the same processes as those in the above-described embodiment. The storage device 4 also includes a dimension reduction model storage unit 44.

[0059] The dimension reduction unit 65 reduces the number of dimensions of the feature calculated by the feature extraction model supplied from the model application unit 63. In this case, parameters of the dimension reduction model are stored in advance in the dimension reduction model storage unit 44. The dimension reduction unit 65 then inputs the feature calculated by the feature extraction model to a dimension reduction model based on the parameters stored in the dimension reduction model storage unit 44, and stores the dimension-reduced feature output by the model in the short-time feature storage unit 41 as a short-time feature. In this case, the dimension reduction model whose parameters are stored in the dimension reduction model storage unit 44 may be any dimension reduction model. For example, the dimension reduction model storage unit 44 stores a transformation matrix whose number of columns and number of rows correspond to the number of dimensions of the feature before and after dimension reduction, respectively.

[0060] According to this configuration, the short-term feature calculation section 16 can generate short-term features with a reduced number of dimensions, and the processing load on the stress feature calculation section 17 and the stress estimation section 18 can be reduced.

[0061] Furthermore, the dimension reduction model whose parameters are stored in the dimension reduction model storage unit 44 may be trained in advance so as to perform dimension reduction suitable for stress estimation.

[0062] 9 is a functional block diagram related to learning of a dimension reduction model. Here, a learning device 7 having an image conversion unit 71, a model application unit 72, and a learning unit 73 learns the dimension reduction model, and the parameters of the dimension reduction model obtained by learning are stored in a dimension reduction model storage unit 44. The learning device 7 may be the stress estimation device 1, or may be a device other than the stress estimation device 1. In the former case, the processor 11 functions as the image conversion unit 71, the model application unit 72, and the learning unit 73. The image conversion unit 71 performs the same processing as the image conversion unit 61, and the model application unit 72 performs the same processing as the model application unit 63.

[0063] In addition, as training data, unit observation data (also referred to as "stressed data") obtained by observing a person (not necessarily the subject) in a state of acute stress, and unit observation data (also referred to as "non-stressed data") obtained by observing a person in a state of not being under acute stress are prepared. The number of samples of the stressed data and the non-stressed data is set, for example, to be approximately the same. Note that acute stress is stress that lasts for a relatively short period of time (several minutes to a day). For example, acute stress can be inflicted on a person by tasks such as solving math problems, giving a speech in public, or pedaling a cycling machine.

[0064] Then, the image conversion unit 71 converts each stress data and non-stress data into an image, and the model application unit 72 inputs the above-mentioned image into a feature extraction model obtained by referring to the feature extraction model memory unit 42, thereby calculating short-term features corresponding to each stress data and non-stress data.

[0065] The learning unit 73 learns the parameters of the dimension reduction model based on the short-term features calculated by the model application unit 72. For example, the learning unit 73 performs principal component analysis on all short-term features corresponding to the stressed data and non-stressed data, which are training data, and calculates a transformation matrix that transforms the short-term features into a predetermined number of dimensions with the highest variance. The parameters stored in the dimension reduction model storage unit 44 are then referenced by the dimension reduction unit 65 shown in FIG. 8 and used to reduce the dimensions of the short-term features in the stress estimation phase.

[0066] Thus, according to this modified example, by training a dimension reduction model using stress data and non-stress data, it is possible to train a dimension reduction model that is useful for estimating chronic stress, which is closely related to acute stress.

[0067] Second Embodiment 10 shows a schematic configuration of a stress estimation system 100A according to the second embodiment. The stress estimation system 100A according to the second embodiment includes a stress estimation device 1A that performs the same processing as the stress estimation device 1 according to the first embodiment, a storage device 4, and a terminal device 8 and sensor 5 used by the subject. Hereinafter, the same components as those in the first embodiment will be appropriately designated by the same reference numerals, and their description will be omitted.

[0068] In the second embodiment, the stress estimation device 1A functions as a server, and the terminal device 8 functions as a client. The stress estimation device 1A and the terminal device 8 perform data communication via a network 9.

[0069] The terminal device 8 is a terminal used by a user who is a subject. The terminal device 8 has input, display, and communication functions and functions as the input device 2 and display device 3 shown in FIG. 1 . The terminal device 8 may be, for example, a personal computer, a tablet terminal such as a smartphone, or a personal digital assistant (PDA). The terminal device 8 is electrically connected to a sensor 5, such as a wearable sensor worn by the user, and transmits the subject's biological signal output by the sensor 5 (i.e., information corresponding to the observation data S3 in FIG. 1 ) to the stress estimation device 1A via a network 9. The terminal device 8 also transmits information generated by user input (information corresponding to the input signal S1 in FIG. 1 ) to the stress estimation device 1A. The sensor 5 may be built into the terminal device 8. The sensor 5 may also have the functions of the terminal device 8 and perform data communication with the stress estimation device 1A.

[0070] The stress estimation device 1A has the same hardware configuration as the stress estimation device 1 shown in Fig. 2, and the processor 11 of the stress estimation device 1A has the functional blocks shown in Fig. 3. The stress estimation device 1A receives information corresponding to the input signal S1 and observation data S3 in Fig. 1 from the terminal device 8 via the network 9, and performs stress estimation processing for the subject by referring to various information stored in the storage device 4. In addition, the stress estimation device 1A transmits an output signal for outputting the stress estimation result to the terminal device 8 via the network 9, based on a display request from the terminal device 8.

[0071] In this way, the stress estimation device 1A in the second embodiment estimates the stress state of the subject based on the subject's biological signals, etc. received from the terminal used by the subject, and can conveniently present the estimation results to the subject on the terminal.

[0072] Third Embodiment 11 is a block diagram of a stress estimation device 1X according to the third embodiment. The stress estimation device 1X mainly includes a division means 15X, a first feature amount calculation means 16X, a second feature amount calculation means 17X, and a stress estimation means 18X. Note that the stress estimation device 1X may be configured by a plurality of devices.

[0073] The dividing means 15X divides the observation data that represents the state of the subject in time series. The dividing means 15X can be, for example, the dividing unit 15 in the first embodiment (including modified examples, the same applies below) or the second embodiment.

[0074] The first feature calculation means 16X calculates first features, which are features of the divided observation data, based on the trained feature extraction model. The "divided observation data" is, for example, the unit observation data in the first or second embodiment, and the "first features" is, for example, the short-time features in the first or second embodiment. The first feature calculation means 16X can be, for example, the short-time feature calculation unit 16 in the first or second embodiment.

[0075] The second feature amount calculation means 17X calculates a second feature amount based on a plurality of first feature amounts. The "second feature amount" is, for example, the "stress feature amount" in the first or second embodiment. The second feature amount calculation means 17X can be, for example, the stress feature amount calculation unit 17 in the first or second embodiment.

[0076] The stress estimation means 18X estimates the stress of the subject based on the second feature amount. The stress estimation means 18X can be, for example, the stress estimation unit 18 in the first or second embodiment.

[0077] FIG. 12 is an example of a flowchart executed by the stress estimation device 1X in the third embodiment. First, the division means 15X divides the observation data representing the time-series state of the subject (step S21). Next, the first feature calculation means 16X calculates first feature amounts, which are feature amounts of the divided observation data, based on the learned feature extraction model (step S22). Then, the second feature calculation means 17X calculates second feature amounts based on the multiple first feature amounts (step S23). The stress estimation means 18X estimates the stress of the subject based on the second feature amounts (step S24).

[0078] According to the third embodiment, the stress estimation device 1X calculates a first feature amount useful for stress estimation using a trained feature amount extraction model, and can estimate the stress of a subject with high accuracy.

[0079] In the above-described embodiments, the program can be stored using various types of non-transitory computer-readable media and supplied to a computer processor or the like. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program may also be supplied to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media can supply the program to a computer via a wired communication path such as an electric wire or optical fiber, or via a wireless communication path.

[0080] In addition, some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes.

[0081] [Appendix 1] A division means for dividing observation data representing a time-series state of a subject; a first feature calculation means for calculating a first feature, which is a feature of the divided observation data, based on the trained feature extraction model; a second feature amount calculation means for calculating a second feature amount based on the plurality of first feature amounts; stress estimation means for estimating stress of the subject based on the second feature amount; A stress estimation device having the above structure. [Appendix 2] The first feature amount calculation means a conversion means for converting each of the divided observation data into model input data that is data that matches an input format of the feature extraction model; a model application means for inputting the model input data into the feature extraction model and acquiring, as the first feature, a feature output from the feature extraction model; 2. The stress estimation device of claim 1, comprising: [Appendix 3] 3. The stress estimation device according to claim 2, wherein the conversion means generates a spectrogram image for each of the divided observation data as the model input data. [Appendix 4] The first feature amount calculation means 4. The stress estimation device according to claim 2, further comprising a missing data exclusion means for determining missing data that has been lost from the model input data and excluding the missing data from the data to be input to the feature extraction model. [Appendix 5] The stress estimation device according to any one of appendices 1 to 4, wherein the second feature calculation means calculates the second feature based on the first feature corresponding to the observation data generated during a period that affects the stress. [Appendix 6] 6. The stress estimation device according to claim 1, wherein the second feature amount calculation means calculates the second feature amount representing a statistic of the first feature amount. [Appendix 7] The stress estimation device according to any one of appendixes 1 to 6, wherein the first feature calculation means performs dimension reduction on the feature output by the feature extraction model and calculates the feature after the dimension reduction as the first feature. [Appendix 8] the first feature calculation means performs dimension reduction of the feature based on a dimension reduction model that performs dimension reduction of input data; The dimension reduction model is trained based on observation data in a stressed state and observation data in a non-stressed state. 8. The stress estimation device according to claim 7. [Appendix 9] 9. The stress estimation device according to claim 1, wherein the dividing means divides the observation data into unit observation data, which are the observation data per unit time. [Appendix 10] The computer Divide the observation data that represents the subject's state over time, calculating a first feature amount that is a feature amount of the divided observation data based on the trained feature amount extraction model; calculating a second feature amount based on the plurality of first feature amounts; Estimating stress of the subject based on the second feature amount. Stress estimation methods. [Appendix 11] Divide the observation data that represents the subject's state over time, calculating a first feature amount that is a feature amount of the divided observation data based on the trained feature amount extraction model; calculating a second feature amount based on the plurality of first feature amounts; A storage medium storing a program that causes a computer to execute a process of estimating the stress of the subject based on the second feature amount.

[0082] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications within the scope of the present invention that would be understood by those skilled in the art can be made to the configuration and details of the present invention. In other words, the present invention naturally includes various modifications and alterations that would be possible for those skilled in the art based on the entire disclosure, including the claims, and the technical ideas. Furthermore, the disclosures of the above-cited patent and non-patent documents are incorporated herein by reference. [Explanation of symbols]

[0083] 1, 1A, 1X stress estimator 2 Input devices 3 Display device 4 Storage device 5 sensors 8 Terminal Equipment 100, 100A Stress Estimation System

Claims

1. A division means for dividing observation data representing a time-series state of a subject; a first feature calculation means for calculating a first feature, which is a feature of the divided observation data, based on the trained feature extraction model; a second feature amount calculation means for calculating a second feature amount based on the plurality of first feature amounts; a stress estimation means for estimating stress of the subject based on the second feature amount; and The first feature amount calculation means a conversion means for converting each of the divided observation data into model input data that is data that matches an input format of the feature extraction model; a model application means for inputting the model input data into the feature extraction model and acquiring, as the first feature, a feature output from the feature extraction model; A stress estimation device comprising:

2. 2. The stress estimation device according to claim 1, wherein the conversion means generates a spectrogram image for each of the divided observation data as the model input data.

3. The first feature amount calculation means 3. The stress estimation device according to claim 1, further comprising a missing data exclusion means for determining missing data that has been lost from the model input data and excluding the missing data from the data to be input to the feature extraction model.

4. 4. The stress estimation device according to claim 1, wherein the second feature calculation means calculates the second feature based on the first feature corresponding to the observation data generated during a period that affects the stress.

5. 5. The stress estimation device according to claim 1, wherein the second feature amount calculation means calculates the second feature amount representing a statistic of the first feature amount.

6. A dividing means for dividing observation data representing a time-series state of a subject; a first feature calculation means for calculating a first feature, which is a feature of the divided observation data, based on the trained feature extraction model; a second feature amount calculation means for calculating a second feature amount based on the plurality of first feature amounts; a stress estimation means for estimating stress of the subject based on the second feature amount; and the first feature calculation means performs dimension reduction of the feature output by the feature extraction model based on a dimension reduction model that reduces the dimension of input data, and calculates the feature after the dimension reduction as the first feature; The dimension reduction model is trained based on observation data in a stressed state and observation data in a non-stressed state. Stress estimation device.

7. 7. The stress estimation device according to claim 1, wherein the dividing means divides the observation data into unit observation data, which are the observation data per unit time.

8. The computer Divide the observation data that represents the subject's state over time, converting each of the divided observation data into model input data that is data that matches the input format of the trained feature extraction model; inputting the model input data into the feature extraction model, and acquiring a feature output from the feature extraction model as a first feature that is a feature of the divided observation data; calculating a second feature amount based on the plurality of first feature amounts; estimating stress of the subject based on the second feature amount; Stress estimation methods.

9. Divide the observation data that represents the subject's state over time, converting each of the divided observation data into model input data that is data that matches the input format of the trained feature extraction model; inputting the model input data into the feature extraction model, and acquiring a feature output from the feature extraction model as a first feature that is a feature of the divided observation data; calculating a second feature amount based on the plurality of first feature amounts; A program that causes a computer to execute a process of estimating stress of the subject based on the second feature amount.

Images