Attention target estimation device, method and program

Abstract

Under an environment in which a plurality of types of external stimuli are generated, when estimating which of the plurality of types of external stimuli a user pays attention to, one aspect of this invention: acquires a measurement signal to which an electrodermal activity of the user is reflected in the environment; extracts a frequency feature amount of a phasic component from the acquired measurement signal; estimates the type of the external stimulus corresponding to the frequency feature amount on the basis of the extracted frequency feature amount; and outputs information that indicates the estimated type of the external stimulus as information that indicates the subject to which the user pays attention.

Description

[Technical Field]

[0001] One aspect of the present invention relates to an attention target estimation device, method, and program that estimates which external stimulus a user is paying attention to, for example, when the user is in an environment where the user is exposed to multiple types of external stimuli. [Background technology]

[0002] When people experience tension, stress, or anxiety, the sympathetic nervous system of the autonomic nervous system becomes active, causing sweating in the extremities of the hands and feet. This is called psychological sweating, and this sweating increases skin conductance. This change in the electrical conductivity and resistivity of the skin is called electrodermal activity (EDA). EDA also appears as a response to external stimuli (event-related skin conductance responses: eSCR), changing to peak 1 to 4 seconds after the stimulus. The component of EDA that responds to external stimuli in this way is called the phasic component.

[0003] Meanwhile, people generally live in environments where they are exposed to a wide variety of external stimuli, and understanding which external stimuli people are paying attention to is useful for supporting people's daily lives and various activities.

[0004] Therefore, in recent years, research has been conducted into methods for estimating a user's attention to external stimuli using EDA. For example, Non-Patent Document 1 introduces a method in which electrodes are attached to the fingers to measure EDA and the peak value of the phasic component is used. Specifically, it suggests that a user's reaction to vibration notifications and notification sounds from a smartphone can be estimated from EDA. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Pascal E. Fortin, Elisabeth Sulmont, and Jeremy Cooperstock. “Detecting perception of smartphone notifications using skin conductance responses.” Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 2019. Summary of the Invention [Problem to be solved by the invention]

[0006] However, the method using the peak value of the phasic component as described in Non-Patent Document 1 is difficult to apply to a person's normal living environment, because multiple types of external stimuli occur intermittently in a person's normal living environment, and it is difficult to identify which of the multiple types of external stimuli the peak value of the phasic component detected in such an environment is a reaction to.

[0007] This invention was made with the above-mentioned circumstances in mind, and aims to provide a technology that enables an estimation of the external stimulus to which a user is paying attention, even in an environment where multiple types of external stimuli are present. [Means for solving the problem]

[0008] In order to solve the above problem, one aspect of the attention object estimation device or estimation method according to the present invention is a device for estimating an attention object in an environment where a plurality of types of external stimuli are occurring, in which a user selects one of the plurality of types of external stimuli. Either wayWhen estimating whether the user is paying attention, a measurement signal reflecting the user's electrodermal activity in the environment is acquired, a frequency feature of the phasic component is extracted from the acquired measurement signal, the type of external stimulus corresponding to the frequency feature is estimated based on the extracted frequency feature, and information representing the estimated type of external stimulus is output as information representing the object to which the user is paying attention.

[0009] According to one aspect of the present invention, a frequency feature is extracted from the phasic component of the measurement signal, and the type of external stimulus to which the user is paying attention is estimated based on the extracted frequency feature. Therefore, even in an environment where multiple types of external stimuli are mixed, it is possible to accurately estimate the type of external stimulus to which the user is paying attention. [Effects of the Invention]

[0010] That is, according to one aspect of the present invention, it is possible to provide a technology that makes it possible to estimate the external stimulus to which the user is paying attention, even in an environment in which multiple types of external stimuli occur. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a system including an attention object estimation device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing an example of the hardware configuration of the attention object estimation device according to an embodiment of the present invention. [Figure 3] FIG. 3 is a block diagram showing an example of the software configuration of the attention object estimation device according to an embodiment of the present invention. [Figure 4] FIG. 4 is a flowchart showing an example of the processing procedure and processing content of the classification model generation processing executed by the control unit of the attention object estimation device shown in FIG. 3 in the learning phase. [Figure 5]FIG. 5 is a flowchart showing an example of the processing procedure and processing content of the classification model generation processing executed by the control unit of the attention object estimation device shown in FIG. 3 in the test phase. [Figure 6A] FIG. 6A is a diagram showing an example of an EDA measurement signal obtained in an environment where the only external stimulus is music. [Figure 6B] FIG. 6B is a diagram showing an example of an EDA measurement signal obtained in an environment where the only external stimulus is the task. [Figure 7A] FIG. 7A is a diagram showing an example of a phasic component detected from an EDA measurement signal in an environment where the only external stimulus is music. [Figure 7B] FIG. 7B is a diagram showing an example of a phasic component detected from an EDA measurement signal in an environment where the only external stimulus is the task. [Figure 8A] FIG. 8A is a diagram showing an example of an EDA measurement signal obtained in a state where a user is paying attention to music in an environment where music and a task are present as external stimuli. [Figure 8B] FIG. 8B is a diagram showing an example of an EDA measurement signal obtained in a state where a user is paying attention to a task in an environment where music and a task are present as external stimuli. [Figure 9A] FIG. 9A is a diagram showing an example of a phasic component detected from an EDA measurement signal in a state where a user is paying attention to music in an environment where music and a task are present as external stimuli. [Figure 9B] FIG. 9B is a diagram showing an example of a phasic component detected from an EDA measurement signal in a state where a user is paying attention to a task in an environment where music and a task are present as external stimuli. [Figure 10] FIG. 10 is a diagram showing an example of a calculation result of a matching rate when an attention object estimation result obtained by the attention object estimation device shown in FIG. 3 is compared with a correct answer label. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0013] [One embodiment] (Configuration example) (1) System FIG. 1 is a diagram showing an example of the configuration of a system including an attention object estimation device according to an embodiment of the present invention.

[0014] A system according to one embodiment includes a measurement terminal that measures a user's EDA, and an attention target estimation device AS that estimates, based on the measured EDA, which external stimulus the user is paying attention to.

[0015] The measurement terminal includes a sensor SS attached to a finger of the user's hand HD, and an EDA measurement terminal UT that measures the EDA based on the detection signal of the sensor SS.

[0016] The sensor SS has conductive terminals attached to, for example, the middle and index fingers of the hand HD, and outputs a detection signal that indicates the electrical resistance of the skin.

[0017] The EDA measurement terminal UT generates an EDA measurement signal representing a time-series change in electrodermal activity based on the detection signal, and transmits the generated EDA measurement signal to the attention object estimation device AS via the network NW.

[0018] The EDA measurement terminal UT may be a dedicated terminal for measuring only the EDA, or may be provided as one function in an information processing terminal such as a smartphone or a wearable terminal.

[0019] The network NW may be a wired or wireless LAN (Local Area Network), or may be a public data communication network including the Internet.

[0020] (2) Attention Object Estimation Device AS The attention object estimation device AS is configured with an information processing device such as a personal computer or a server computer, and is located, for example, on the cloud or the web. The attention object estimation device AS may be located in a local area such as an office or business so that it can be shared by multiple users, or may be located individually for each user. Furthermore, the attention object estimation device AS may be configured integrally with the EDA measurement terminal UT.

[0021] 2 and 3 are block diagrams showing an example of the hardware configuration and software configuration of the attention object estimation device AS, respectively.

[0022] The attention object estimation device AS includes a control unit 1 that uses a hardware processor such as a central processing unit (CPU). Connected to the control unit 1 via a bus 6 are a storage unit having a program storage unit 2 and a data storage unit 3, a communication interface (hereinafter, interface will be abbreviated as I / F) unit 4, and an input / output I / F unit 5. The control unit 1 may be configured using a programmable logic device (PLD), a field programmable gate array (FPGA), or the like.

[0023] The communication I / F unit 4 is used to receive EDA measurement signals transmitted from the EDA measurement terminal UT in accordance with a communication protocol defined by the network NW.

[0024] An input device 7 and an output device 8 are connected to the input / output I / F unit 5. The input device 7 is used by a system administrator or a user to specify an operation mode for the attention object estimation device AS and to input learning data, etc., required for generating a classification model. The output device 8 is used to display information representing the estimation result by the attention object estimation device AS on, for example, a display.

[0025] The input device 7 and the output device 8 may be devices attached to the attention object estimation device AS, or may be input / output devices provided in other information processing terminals such as a personal computer or a smartphone other than the attention object estimation device AS. As a means for connecting to the input / output I / F unit 5, in addition to a direct connection using a signal cable such as a USB cable, a wireless interface adopting a short-range wireless data communication standard such as Bluetooth (registered trademark) or WiFi (registered trademark), a wireless interface such as G4 or G5, or even a wired interface may be used.

[0026] The program storage unit 2 is configured by combining, for example, a non-volatile memory such as a solid-state drive (SSD) as a storage medium that can be written to and read from at any time, and a non-volatile memory such as a read-only memory (ROM), and stores middleware such as an operating system (OS), as well as application programs required to execute various control processes according to one embodiment. Hereinafter, the OS and each application program will be collectively referred to as the program.

[0027] The data storage unit 3 is, for example, a combination of a non-volatile memory such as an SSD that can be written to and read from at any time as a storage medium, and a volatile memory such as a RAM (Random Access Memory), and is provided with an EDA measurement data storage unit 31 and a classification model storage unit 32 as the main storage areas required to implement one embodiment.

[0028] The EDA measurement data storage unit 31 is used to store EDA measurement data obtained by sampling the EDA measurement signal sent from the EDA measurement terminal UT at a predetermined sampling rate.

[0029] The classification model storage unit 32 is used to store a classification model made up of a machine learning model for determining an attention target.

[0030] The control unit 1 includes, as processing functions necessary for carrying out one embodiment, an EDA measurement signal acquisition processing unit 11, a frequency feature calculation processing unit 12, a classification model generation processing unit 13, a cautionary object determination processing unit 14, and an estimated information output processing unit 15. All of the above processing units 11 to 15 are realized by causing a hardware processor of the control unit 1 to execute application programs stored in the program storage unit 2.

[0031] The EDA measurement signal acquisition processing unit 11 receives the EDA measurement signal sent from the EDA measurement terminal UT via the network NW via the communication I / F unit 4, samples this EDA measurement signal at a predetermined sampling rate, and performs processing to store the EDA measurement data generated thereby in the EDA measurement data storage unit 31.

[0032] The frequency feature calculation processing unit 12 reads the EDA measurement data stored in the EDA measurement data storage unit 31 for a fixed period at a time, and calculates the frequency feature of the phasic component for each fixed period. An example of this frequency feature calculation processing will be described in the operation example.

[0033] The classification model generation processing unit 13 operates in the learning phase. The classification model generation processing unit 13 uses EDA measurement data obtained from a user when each of multiple types of assumed external stimuli is individually applied to the user. Then, using the frequency feature values ​​of the phasic components calculated from the EDA measurement data and labels (correct labels) indicating the types of external stimuli as learning data, the classification model generation processing unit 13 generates a classification model using, for example, a linear discriminant analysis (LDA) algorithm, and stores the generated classification model in the classification model storage unit 32.

[0034] The generated classification model takes the frequency features of the phasic components calculated from the EDA measurement data as input and outputs estimated labels representing the types of external stimuli corresponding to the frequency features of the phasic components. The machine learning model uses, for example, a convolutional neural network, but other neural networks may also be used.

[0035] The attention object determination processor 14 operates in the test phase. It uses EDA measurement data obtained from a user when multiple types of external stimuli are mixed and presented to the user. The attention object determination processor 14 then inputs the frequency feature values ​​of the phasic components calculated from the EDA measurement data into the classification model, and obtains labels representing the types of external stimuli output from the classification model.

[0036] The estimated information output processing unit 15 generates information representing an estimation result of the target to which the user is paying attention, based on the label representing the type of the external stimulus. Then, the information representing the generated estimation result is output from the input / output I / F unit 5 to the output device 8, and is displayed on a display, for example. The estimated information output processing unit 15 may also have a function of transmitting the generated information representing the estimation result from the communication I / F unit 4 to, for example, a server device that provides a service to support user behavior.

[0037] (Example of operation) Next, an example of the operation of the attention object estimation device AS configured as above will be described.

[0038] (1) Learning Phase When the control unit 1 of the attention object estimation device AS receives a request to execute the learning phase from, for example, the input device 7, it executes the process of generating a classification model as follows.

[0039] FIG. 4 is a flowchart showing an example of the processing procedure and processing content of a series of processes related to the generation of a classification model, which are executed by the control unit 1 of the attention object estimation device AS.

[0040] When generating a classification model, external stimuli are given to the user individually. For example, if two types of external stimuli are assumed, "playing music" and "performing a specific task," each of these two types of external stimuli is given to the user individually. In this example, the "specific task" is a task such as "pressing a button the moment a mark appears on the screen of the user's device."

[0041] When a user is given one of the two tasks, the user will show a response (eSCR) to the task. This response is measured as a time series of changes in electrodermal activity (EDA) using a sensor SS attached to the user's finger and an EDT measurement terminal UT. The EDT measurement terminal UT transmits successive EDA measurement signals over time to an attention target estimation device AS via a network NW.

[0042] (1-1) Acquisition of EDA measurement signal In step S10, the control unit 1 of the attention object estimation device AS acquires the EDA measurement signal transmitted from the EDA measurement terminal UT under the control of the EDA measurement signal acquisition processing unit 11. That is, the EDA measurement signal acquisition processing unit 11 receives the EDA measurement signal via the communication I / F unit 4, samples the received EDA measurement signal at a predetermined sampling rate, and stores the EDA measurement data generated thereby in the EDA measurement data storage unit 31. The sampling rate is set to, for example, 100 Hz.

[0043] (1-2) Calculation of frequency features Next, under the control of the frequency feature amount calculation processing unit 12, the control unit 1 of the attention object estimation device AS calculates the frequency feature amount of the phasic component from the EDA measurement data as follows.

[0044] That is, in step S11, the frequency feature calculation processor 12 first passes the EDA measurement data read from the EDA measurement data storage unit 31 through a high-pass filter (HPF) to extract the phasic components of the EDA. Then, in step S12, the frequency feature calculation processor 12 standardizes the extracted phasic components using a z-score. That is, the values ​​of the phasic components are converted so that the average is "0" and the standard deviation is "1."

[0045] In step S13, the frequency feature calculation processor 12 divides the standardized phasic components of the EDA by a window of a fixed period, performs a fast Fourier transform (FFT) on the phasic components for the divided period, and calculates a power spectrum for the fixed period.The frequency feature calculation processor 12 then extracts the power of the low-frequency band from the calculated power spectrum as a feature vector.

[0046] For example, when the sampling rate is 100 Hz, the frequency feature calculation processor 12 performs FFT processing on the phasic components of the EDA measurement data for every 4096 samples, and extracts the power of the low frequency band from 0 to 0.25 Hz from the power spectrum obtained as a result as a feature vector for the period of the 4096 samples, thereby obtaining a 10-dimensional feature vector.

[0047] The frequency feature calculation processing unit 12 repeats the process of extracting the feature vector for each period while shifting the position of the window of the fixed period length by a fixed interval, for example, by 10 samples.

[0048] (1-3) Learning the classification model In step S14, under the control of the classification model generation processing unit 13, the control unit 1 of the attention object estimation device AS trains a classification model using the extracted feature vector and a correct label representing the type of the external stimulus given solely to the user at that time as training data.

[0049] For example, when the subject is listening to music alone, the classification model is trained using the feature vector obtained at this time and the correct label indicating that attention is being paid to the music as training data.Also, when the subject is performing a specific task alone, the classification model is trained using the feature vector obtained at this time and the correct label indicating that attention is being paid to the specific task as training data.

[0050] Thus, through the above series of learning processes, a classification model trained for each external stimulus is generated. Note that even when the same external stimulus is given, there are individual differences in the reaction of each user. For this reason, it is desirable to generate a classification model for each user. The generated classification model is stored in the classification model storage unit 32.

[0051] (2) Testing Phase When the generation of the classification model is completed and a request to execute the test mode is input from the input device 7, the control unit 1 of the attention object estimation device AS executes a series of processes to estimate the user's attention object as follows.

[0052] FIG. 5 is a flowchart showing an example of a processing procedure and processing content of a series of processes for estimating an attention object, which are executed by the control unit 1 of the attention object estimation device AS.

[0053] In the test phase, the user is given a mixture of the two types of tasks. For example, the user is asked to perform the specific task while listening to music. When the two types of tasks are given simultaneously, the user responds to the task that interests them most. As in the learning phase, this response is measured as a time-series change in EDA using the sensor SS attached to the user's finger and the EDT measurement terminal UT, and the obtained time-series EDA measurement signal is transmitted from the EDT measurement terminal UT to the attention object estimation device AS via the network NW.

[0054] (2-1) Acquisition of EDA measurement signal In step S20, the control unit 1 of the attention object estimation device AS receives the EDA measurement signal transmitted from the EDA measurement terminal UT under the control of the EDA measurement signal acquisition processing unit 11. Then, the received EDA measurement signal is sampled at a sampling rate of 100 Hz, as in the learning phase, and the resulting EDA measurement data is temporarily stored in the EDA measurement data storage unit 31.

[0055] (2-2) Calculation of frequency features Next, under the control of the frequency feature calculation processing unit 12, the control unit 1 of the attention object estimation device AS performs a process of calculating frequency feature values ​​of phasic components from the stored EDA measurement data, in the same manner as in the learning phase.

[0056] That is, in step S21, the frequency feature calculation processor 12 first reads out the EDA measurement data from the EDA measurement data storage unit 31 and passes it through a high-pass filter (HPA) to extract the phasic components of the EDA. Next, in step S22, the frequency feature calculation processor 12 converts the extracted phasic components using a z-score so that the average is "0" and the standard deviation is "1."

[0057] Next, in step S23, the frequency feature calculation processor 12 divides the phasic components of the standardized EDA into sections with a window set to a period of 4096 samples, and performs FFT processing on the phasic components of the divided sections to calculate a power spectrum for the sections.The frequency feature calculation processor 12 then extracts the power in the low-frequency band from 0 to 0.25 Hz from the calculated power spectrum as a feature vector.Therefore, even in this test mode, a 10-dimensional feature vector is obtained.

[0058] The frequency feature calculation processing unit 12 repeats the process of extracting the feature vector for each period while shifting the position of the window by 10 samples at a time.

[0059] (2-3) Identifying items to be careful of and outputting the results Once the above feature vector for a specified target period is obtained, the control unit 1 of the attention object estimation device AS then performs the process of determining the user's attention object using a classification model under the control of the attention object determination processing unit 14 as follows.

[0060] That is, in step S24, the attention object determination processing unit 14 inputs the extracted feature vector to the classification model stored in the classification model storage unit 32. As a result, a label indicating the type of external stimulus corresponding to the input feature vector is output from the classification model. In step S25, the attention object determination processing unit 14 acquires the label output from the classification model and passes the acquired label to the estimated information output processing unit 15.

[0061] The estimated information output processing unit 15 generates estimated information representing the result of the attention object determination based on the label, and outputs the generated estimated information from the input / output I / F unit 5 to the output device 8. As a result, the estimated information is displayed on the display of the output device 8. The estimated information may be transmitted from the communication I / F unit 4 to a server device that provides a user behavior support service or the like.

[0062] (Verification example) Finally, the accuracy of the estimation operation by the attention object estimation device AS according to the embodiment described above will be verified.

[0063] (3-1) Generation of classification model For example, external stimuli are given to the four test subjects under the following conditions:

[0064] (1) Measure each user's EDA for 110 seconds while listening to music. This stimulus condition is music only.

[0065] (2) Each user's EDA was measured for 110 seconds while performing a task to measure reaction time (a task in which the user presses a button the instant a mark appears on the screen). This stimulus condition was called task-only. The mark was presented once every 5 to 6 seconds.

[0066] That is, two types of external stimuli are given independently to four users.

[0067] Under the above stimulus conditions (1) and (2), the target period is set to be from 20 seconds after the start of the task to the end of the task. When the EDA of each user is measured during this period, EDA measurement signals showing time-series changes are obtained, as shown in Figures 6A and 6B. Furthermore, when each of the above EDA measurement signals is input into the attention object estimation device AS and the phasic components are extracted and standardized by the frequency feature calculation processing unit 12, the standardized phasic components shown in Figures 7A and 7B are calculated.

[0068] That is, as shown in FIGS. 7A and 7B, under each of the stimulation conditions (1) and (2), phasic components with different peak heights and intervals are obtained even for the same user.

[0069] Next, the standardized phasic components are subjected to FFT processing to calculate the power spectrum. Then, feature vectors in the low-frequency band are extracted from the calculated power spectrum. As a result, 400 feature vectors are obtained for each user under each of the above stimulus conditions (1) and (2).

[0070] Next, the classification model generation processing unit 13 generates a classification model using the feature vectors obtained under the above stimulus conditions (1) and (2) and the correct labels representing the correct values ​​for each of the above conditions, "paying attention to music" and "paying attention to the task," as training data. This classification model is generated individually for each of the four users because there are individual differences in their responses.

[0071] (3-2) Verification In the verification, external stimuli were given to each of the four users under the following conditions. (3) EDA was measured for 110 seconds while the user was performing a task to measure reaction time while listening to music. However, the user was asked to sing the song in their head while listening to the music to focus their attention on listening to the music. In the task to measure reaction time, a mark was presented once every 5 to 6 seconds. This stimulus condition was called "music focus."

[0072] (4) EDA was measured for 110 seconds while the user was performing a task to measure reaction time while listening to music. However, the user was asked to concentrate on the task to focus their attention on the task. During the task to measure reaction time, a mark was presented once every 5 to 6 seconds. This stimulus condition was called the task focus.

[0073] In other words, four users are each given two types of external stimuli in a mixed state, and are asked to pay attention to one of the external stimuli.

[0074] Under the above stimulus conditions (3) and (4), the target period is from 20 seconds after the start of the task to the end of the task. When EDA is measured during this period, EDA measurement signals showing time-series changes are obtained, as shown in Figures 8A and 8B. Furthermore, when each of the above EDA measurement signals is input into the attention object estimation device AS and the phasic components are extracted and standardized by the frequency feature calculation processing unit 12, the standardized phasic components shown in Figures 9A and 9B are calculated.

[0075] That is, even in this case, even for the same user, phasic components with different peak heights and intervals are obtained depending on the stimulation conditions (3) and (4).

[0076] Next, the standardized phasic components are subjected to FFT processing to calculate the power spectrum. Then, feature vectors in the low-frequency band are extracted from the calculated power spectrum. As a result, 400 feature vectors are obtained for each user under each of the stimulation conditions (3) and (4).

[0077] Next, for each user, the feature vector extracted by the frequency feature calculation process for each of the stimulus conditions (3) and (4) is input to a classification model corresponding to the user. As a result, a cautionary label, i.e., an estimated label, corresponding to the user's reaction to each of the stimulus conditions (3) and (4) is output from the classification model for each user.

[0078] Figure 10 shows the estimated label output results, which reflect the responses of four users for each stimulus condition (3) and (4), compared with the correct labels, i.e., the estimation accuracy. In this example, 1600 pieces of data (400 pieces for 4 users) are used as the parameter.

[0079] As shown in FIG. 10, it was confirmed that by using an attention object estimation device AS according to one embodiment, it is possible to estimate the external stimulus to which the user is paying attention with an accuracy of approximately 85% for both stimulus conditions (3) and (4).

[0080] (Actions and Effects) As described above, in one embodiment, in the learning phase, a user's EDA measurement signal is acquired while the user is receiving only an external stimulus. Then, phasic components are extracted from the acquired EDA measurement signal at regular intervals. The extracted phasic components are standardized and subjected to FFT processing to calculate a power spectrum. A feature vector is then extracted from the power spectrum. The extracted feature vector and a ground truth label representing the external stimulus are used as training data to generate a classification model using machine learning.

[0081] Next, in the test phase, while the user is being presented with a mixture of various external stimuli, the EDA measurement signal of the user is acquired, and similarly to the learning phase, the power spectrum of the phasic component is calculated at regular intervals based on the acquired EDA measurement signal to extract feature vectors.The extracted feature vectors are then input into the classification model, and the resulting labels output from the classification model are used to estimate the type of external stimulus to which the user is paying attention.

[0082] Therefore, for each interval of a certain period of time, a feature vector is extracted from the power spectrum of the phasic component of the EDA measurement signal of the user, and the type of external stimulus to which the user is paying attention is estimated based on the extracted feature vector. Therefore, even in an environment where multiple types of external stimuli coexist, it is possible to accurately estimate the type of external stimulus to which the user is paying attention.

[0083] Furthermore, a classification model created in advance for each user by machine learning is used to determine the type of external stimulus from the feature vector, which makes it possible to perform highly accurate estimation in a short time with a small configuration, without requiring extensive calculations or large-capacity data tables each time.

[0084] [Other embodiments] (1) In one embodiment, the estimation of the type of external stimulus has been described using a classification model based on machine learning. However, it is not necessary to use a machine learning model. For example, a data table may be used that stores in advance information representing the correspondence between information representing the type of expected external stimulus and the corresponding frequency feature vector extracted from the phasic component of EDA.

[0085] (2) In one embodiment, the attention object estimation device AS is described as a dedicated device having only that function. However, the present invention is not limited to this, and the attention object estimation device AS may be provided as one of the functions of an existing server computer, personal computer, or mobile terminal such as a smartphone. Furthermore, the multiple functions of the attention object estimation device AS may be distributed and arranged in multiple information processing devices.

[0086] (3) The functions of the attention object estimation device AS, its processing procedures and processing contents, the type of neural network that constructs the classification model and its generation method, the type and number of external stimuli, and the use of the attention object estimation information can be modified and implemented in various ways without departing from the spirit of this invention.

[0087] Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, when implementing the present invention, specific configurations according to the embodiments may be appropriately adopted.

[0088] In short, this invention is not limited to the above-described embodiments, and in the implementation stage, the components can be modified and embodied without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined. [Explanation of symbols]

[0089] AS: Attention target estimation device UT...EDA measurement terminal SS...EDA sensor RB...Robot arm HD…User's Hands NW...Network 1...Control unit 2...Program memory section 3...Data storage unit 4...Communication I / F section 5...Input / output interface 6...Bus 7. Input Device 8...Output device 11...EDA measurement signal acquisition processing section 12...Frequency feature calculation processing unit 13...Classification model generation processing unit 14...Caution object determination processing unit 15...Estimated information output processing unit 31...EDA measurement data storage section 32...Classification model memory section

Claims

1. An attention target estimation device that estimates to which of a plurality of types of external stimuli a user is paying attention in an environment in which the plurality of types of external stimuli are occurring, a first processing unit that acquires a measurement signal that reflects the user's electrodermal activity in the environment; a second processing unit that extracts a frequency feature of a phasic component from the acquired measurement signal; a third processing unit that estimates a type of the external stimulus corresponding to the extracted frequency feature based on the extracted frequency feature; a fourth processing unit that outputs information representing the estimated type of the external stimulus as information representing an object to which the user is paying attention; An attention object estimation device comprising:

2. 2. The attention object estimation device according to claim 1, wherein the second processing unit divides the measurement signal into fixed periods, and each time the position of the division is shifted by a predetermined amount, converts the phasic component of the measurement signal in the fixed period into a power signal on the frequency axis, and extracts power in a predetermined frequency band from the converted power signal as the frequency feature.

3. 2. The attention object estimation device according to claim 1, wherein the third processing unit estimates the type of the external stimulus corresponding to the extracted frequency feature for each of a plurality of types of the external stimulus, using information representing a correspondence relationship between the frequency feature extracted from the measurement signal corresponding to the external stimulus and information representing the type of the external stimulus.

4. The attention object estimation device according to claim 3 , wherein the third processing unit uses a machine learning model that receives the extracted frequency feature as an input and outputs information representing the type of the external stimulus.

5. 5. The attention object estimation device according to claim 4, further comprising a fifth processing unit that generates the machine learning model using, as learning data, the frequency features extracted by the second processing unit and a correct label representing the type of the external stimulus for each of the external stimuli prior to the process of estimating the type of the external stimulus.

6. An attention target estimation method executed by an information processing device that estimates to which of a plurality of types of external stimuli a user is paying attention in an environment in which the plurality of types of external stimuli are occurring, comprising: a first processing step of acquiring a measurement signal reflecting the electrodermal activity of the user in the environment; a second processing step of extracting a frequency feature of a phasic component from the acquired measurement signal; a third processing step of estimating, based on the extracted frequency feature, a type of the external stimulus corresponding to the frequency feature; a fourth processing step of outputting information representing the estimated type of the external stimulus as information representing an object to which the user is paying attention; An attention object estimation method comprising:

7. A program that causes a processor provided in the attention object estimation device to execute processing by at least one of the first processing unit to the fourth processing unit provided in the attention object estimation device described in any one of claims 1 to 4, or processing by the fifth processing unit provided in the attention object estimation device described in claim 5.

Images