Information processing apparatus, information processing method, and program

Abstract

To provide an information processing apparatus which outputs information for controlling a space.SOLUTION: An information processing apparatus includes a control unit configured to output information on the atmosphere in a space formed by a plurality of participants. The information processing apparatus stores, in a storage unit, an atmosphere estimation model generated based on information on a state of each of the participants, information on interactions between the participants, and a dataset including an evaluation value obtained by evaluating the atmosphere of the space. The atmosphere estimation model is a model for estimating information on the atmosphere in a space on the basis of state information and interaction information.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] There is known a technique for estimating the atmosphere of a place formed by multiple participants. For example, Patent Document 1 discloses an atmosphere estimation device that analyzes the behavior and emotions of multiple members from images of a place including multiple members and audio recorded in the place, and infers that the atmosphere of the place is bad when the discomfort level of other members increases after the behavior of one member or when the mutual relationship between the multiple members is weak. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-185117 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional technology does not take into consideration controlling the space in which the venue is formed according to the atmosphere of the venue.

[0005] The present disclosure provides a technique for outputting information for controlling the space in which a field is formed. [Means for solving the problem]

[0006] An information processing device according to a first aspect of the present disclosure is an information processing device having a control unit that outputs information regarding the atmosphere of a place formed by a plurality of participants, wherein the control unit stores in a memory unit an atmosphere estimation model generated based on a dataset including information regarding the individual states of the participants, information regarding the interactions between the participants, and an evaluation value that evaluates the atmosphere of the place, and the atmosphere estimation model is a model for estimating information regarding the atmosphere of the place based on the information regarding the states and the information regarding the interactions.

[0007] According to the first aspect of the present disclosure, it is possible to output information for controlling the space in which the field is formed.

[0008] A second aspect of the present disclosure is an information processing device relating to the first aspect, in which a control unit estimates an index indicating the atmosphere of the field based on information regarding the state and information regarding the interaction, and outputs a correction value related to the thermal sensation in the space in which the field is formed based on the index indicating the atmosphere of the field.

[0009] According to the second aspect of the present disclosure, it is possible to output a correction value related to a thermal sensation in a space in which a field is formed.

[0010] A third aspect of the present disclosure is an information processing device relating to the first aspect, wherein a control unit estimates an index indicating the atmosphere of the field based on information regarding the state and information regarding the interaction, and controls the space in which the field is formed based on the index indicating the atmosphere of the field.

[0011] According to the third aspect of the present disclosure, the space in which the venue is formed can be controlled according to the atmosphere of the venue.

[0012] A fourth aspect of the present disclosure is an information processing device relating to the first aspect, wherein a control unit estimates an index indicating the atmosphere of the field based on information regarding the state and information regarding the interaction, estimates a correction value related to a thermal sensation in the space in which the field is formed based on the index indicating the atmosphere of the field, and controls the space based on the correction value related to the thermal sensation.

[0013] According to the fourth aspect of the present disclosure, the space in which the venue is formed can be controlled according to the atmosphere of the venue.

[0014] A fifth aspect of the present disclosure is an information processing device according to any one of the first to fourth aspects, wherein the information relating to the state is biometric information of the participant.

[0015] A sixth aspect of the present disclosure is the information processing device according to the fifth aspect, wherein the bio-information is bio-information related to an emotion of a participant.

[0016] A seventh aspect of the present disclosure is an information processing device according to any one of the first to sixth aspects, wherein the information regarding the interaction is information regarding communication between participants.

[0017] An eighth aspect of the present disclosure is the information processing device according to the seventh aspect, wherein the information related to communication is information related to an action of communicating between participants.

[0018] A ninth aspect of the present disclosure is an information processing device according to any one of the first to eighth aspects, in which the control unit estimates the atmosphere of a place by supervised machine learning.

[0019] A tenth aspect of the present disclosure is an information processing device according to any one of the first to ninth aspects, wherein the evaluation value of the atmosphere of the place is a subjective report or an objective evaluation of the atmosphere of the place.

[0020] An eleventh aspect of the present disclosure is an information processing device according to the tenth aspect, wherein the subjective report of the atmosphere of the place is an index related to the atmosphere of the place experimentally obtained in advance using a questionnaire method.

[0021] A twelfth aspect of the present disclosure is an information processing device according to any one of the first to eleventh aspects, wherein the control unit estimates an index indicating the atmosphere of a place based on a trained atmosphere estimation model.

[0022] A thirteenth aspect of the present disclosure is an information processing device related to the twelfth aspect, wherein the atmosphere estimation model takes as input information related to a state and information related to an interaction, and outputs an estimate of an index indicating the atmosphere of the place using an evaluation value that evaluates the atmosphere of the place as correct answer data.

[0023] A fourteenth aspect of the present disclosure is the information processing device according to the thirteenth aspect, wherein the atmosphere estimation model further receives information on attributes of the participants as an input.

[0024] A fifteenth aspect of the present disclosure is an information processing device relating to any of the twelfth to fourteenth aspects, in which a control unit acquires feedback information regarding the atmosphere of the venue from participants, and updates the atmosphere estimation model based on the feedback information.

[0025] A sixteenth aspect of the present disclosure is the information processing device according to the fifteenth aspect, wherein the feedback information includes an evaluation value that evaluates the atmosphere of the place.

[0026] A seventeenth aspect of the present disclosure is the information processing device according to the second or fourth aspect, wherein the control unit estimates a correction value related to a thermal sensation based on the trained thermal index estimation model.

[0027] An 18th aspect of the present disclosure is an information processing device related to the 17th aspect, in which a thermal index estimation model takes as input an index indicating the atmosphere of a place estimated based on information regarding the state and information regarding interactions, and outputs an estimate of a correction value related to thermal sensation.

[0028] A nineteenth aspect of the present disclosure is the information processing device according to the eighteenth aspect, wherein the thermal index estimation model further receives as input information on attributes of participants.

[0029] A twentieth aspect of the present disclosure is an information processing device relating to any of the seventeenth to nineteenth aspects, wherein a control unit acquires feedback information regarding the thermal sensation of the space from participants, and updates a thermal index estimation model based on the feedback information.

[0030] A twenty-first aspect of the present disclosure is the information processing device according to the twentieth aspect, wherein the feedback information includes an evaluation value that evaluates the thermal sensation of the space.

[0031] An information processing method according to a 22nd aspect of the present disclosure includes an information processing device that outputs information regarding the atmosphere of a place formed by a plurality of participants, a control unit of which stores in a memory unit an atmosphere estimation model generated based on a dataset including information regarding the state of each participant, information regarding the interactions between the participants, and an evaluation value that evaluates the atmosphere of the place, and the atmosphere estimation model is a model for estimating information regarding the atmosphere of the place based on the information regarding the state and the information regarding the interactions.

[0032] A program according to a 23rd aspect of the present disclosure causes a control unit of an information processing device that outputs information regarding the atmosphere of a place formed by a plurality of participants to execute a process of storing in a memory unit an atmosphere estimation model generated based on a dataset including information regarding the individual states of the participants, information regarding the interactions between the participants, and an evaluation value that evaluates the atmosphere of the place, and the atmosphere estimation model is a model for estimating information regarding the atmosphere of the place based on information regarding the states and information regarding the interactions. [Brief description of the drawings]

[0033] [Figure 1] FIG. 13 is a diagram showing an example of the relationship between the emotions of individual participants and the communication states between the participants. [Diagram 2] 1 is a block diagram showing an example of an overall configuration of a spatial control system. [Diagram 3] FIG. 2 is a block diagram showing an example of a hardware configuration of the estimation device. [Figure 4] 1 is a flowchart illustrating an example of a spatial control method. [Diagram 5] FIG. 13 is a diagram illustrating an example of learning data. [Figure 6] FIG. 1 is a diagram illustrating an example of learning data of a support vector machine. [Figure 7]FIG. 1 is a diagram illustrating an example of training data for a convolutional neural network. [Figure 8] FIG. 13 is a diagram showing an example of the relationship between the atmosphere of a place and a thermal sensation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and redundant description will be omitted.

[0035] [Embodiment] One embodiment of the present disclosure is a space control system that controls a space in which a place is formed by multiple participants. The space control system estimates the atmosphere of the place based on information on the state of each participant forming the place and information on the interaction between the participants, using an evaluation value that evaluates the atmosphere of the place as correct answer data. The space control system also estimates a correction value for thermal sensation based on an index indicating the atmosphere of the place. Furthermore, the space control system controls the space based on the correction value for thermal sensation.

[0036] For example, in a place where multiple participants gather and talk, it is necessary to provide an atmosphere where each participant feels comfortable speaking up. For example, in a meeting aimed at generating ideas, even if a participant has an idea, if the atmosphere is such that it is difficult to speak up, the participant may hesitate to speak up, and ideas may not be shared smoothly among participants. Therefore, there is a demand for technology that can estimate the atmosphere of a place formed by multiple participants.

[0037] It is also believed that the temperature sensation felt by participants changes depending on the atmosphere of the room. For example, if the meeting is heated, participants may feel hotter than the actual room temperature. Also, if the meeting participants are tense, participants may feel colder than the actual room temperature. Therefore, there is a demand for technology that can estimate the temperature sensation taking into account the atmosphere of the room and control the space appropriately.

[0038] The atmosphere of a place can be estimated with high accuracy by detecting both the emotions of the individual participants and the communication between them. FIG. 1 is a diagram showing an example of the relationship between the emotions of the individual participants and the communication state between the participants. FIG. 1 shows the emotions of four participants A to D and the communication state between the participants. Note that FIG. 1 classifies the emotions of the participants into four emotions: joy, anger, sadness, and pleasure. Also, the thicker the arrows connecting the participants, the more active the communication is.

[0039] As shown in Figure 1, Participant B is expressing emotions of joy, and is actively communicating with Participant A and Participant D. In this case, it can be assumed that the atmosphere is one in which Participant B feels comfortable talking. Participant A is expressing emotions of sadness, but is actively communicating with Participant B and Participant D. In this case, it can be assumed that the atmosphere is one in which Participant A feels somewhat comfortable talking. Participant C is expressing emotions of joy, but is not actively communicating with the other Participants A, B, and D. In this case, it can be assumed that the atmosphere is one in which Participant C feels uncomfortable talking.

[0040] The present embodiment aims to output information for controlling a space formed by multiple participants. As an example, the present embodiment outputs an index indicating the atmosphere of the space formed by multiple participants. As another example, the present embodiment outputs a correction value related to a thermal sensation based on the index indicating the atmosphere of the space. As a further example, the present embodiment outputs a control signal for controlling the space based on the correction value related to a thermal sensation.

[0041] In one aspect, participants can select an action to appropriately control the space by referring to an indicator indicating the atmosphere of the place. For example, when the atmosphere of the place is active (or inactive), participants can select an action to lower the temperature setting of the air conditioner (or an action to raise the temperature setting). Also, for example, when the correction value related to thermal sensation is a positive value (or a negative value), participants can select an action to lower the temperature setting of the air conditioner (or an action to raise the temperature setting).

[0042] In another aspect, the space can be automatically controlled appropriately according to the atmosphere of the place. For example, when the atmosphere of the place becomes lively (or inactive), the set temperature of the air conditioner can be automatically lowered (or automatically raised). Also, for example, when the correction value related to the thermal sensation becomes a positive value (or a negative value), the set temperature of the air conditioner can be automatically lowered (or automatically raised).

[0043] <Overall composition> Fig. 2 is a block diagram showing the overall configuration of the space control system in this embodiment. As shown in Fig. 2, the space control system 1000 in this embodiment includes an estimation device 10 and a space control device 20. The space control system 1000 is a control system that controls equipment installed in a space 2 in which multiple participants 1 (1-1 to 1-3) exist. In the space 2, a sensor 3-1 that acquires personal state information and a sensor 3-2 that acquires interaction information are installed.

[0044] The sensor 3-1 is an electronic device that measures the state of each participant 1. The sensor 3-1 may include, for example, a wearable device attached to each participant 1, or a video camera that captures each participant 1. A plurality of sensors 3-1 may be installed depending on the type of measurable information.

[0045] The sensor 3-2 is an electronic device that measures information regarding interactions between multiple participants 1. The sensor 3-2 may include, for example, a video camera, a microphone, or a motion sensor. A plurality of sensors 3-2 may be installed depending on the type of measurable information.

[0046] The personal status information is information indicating the status of each participant 1. An example of the personal status information is biometric information that can be acquired by observing the status of each participant 1. The biometric information may be biometric information related to the emotion of each participant 1. The biometric information related to emotion means biometric information that can be used to estimate emotion.

[0047] Specifically, the personal condition information includes at least one of electrocardiogram information, respiration information, skin potential information, facial expression information, pupil diameter information, gaze information, and skin temperature information.

[0048] The electrocardiographic information includes, for example, heart rate, low frequency components (LF; Low Frequency) of the electrocardiogram waveform, high frequency components (HF; High Frequency) of the electrocardiogram waveform, a parasympathetic nerve index (CVI; Cardiac vagal index), a sympathetic nerve index (CSI; Cardiac Sympathetic Index), etc. The respiration information includes, for example, a respiration frequency, a respiration depth, etc. The facial expression information includes, for example, a percentage of a predetermined emotion (e.g., happy) in image-based facial expression recognition, etc. The gaze information includes, for example, a label indicating where the gaze position is on the face, body, background, etc. The gaze position can be acquired by, for example, eye tracking technology.

[0049] The interaction information is information indicating an interaction between multiple participants 1. An example of the interaction information is information related to communication between multiple participants 1. The information related to communication is information related to communicative behavior between multiple participants 1. The communicative behavior may be, for example, a conversation or body language.

[0050] Specifically, the interaction information includes at least one of information regarding posture, information regarding conversation, information regarding gaze, information regarding the relative position of gaze, synchronization of electrocardiogram-related features, synchronization of respiratory features, synchronization of skin potential, and synchronization of posture angle.

[0051] The information about posture includes, for example, the angle of body leaning forward. The information about conversation includes, for example, the ratio of speech volume, the order of conversation, keywords in the conversation, etc. The information about the relative position of gaze includes, for example, a label indicating which participant the gaze position is directed toward. The synchronization of feature amounts includes, for example, the synchronization rate of feature amounts between multiple participants 1. The synchronization can be calculated using, for example, dynamic time warping (DTW), Euclidean distance, cosine similarity, etc. The synchronization of posture angles includes, for example, the state of posture coordination between multiple participants 1.

[0052] The estimation device 10 is an information processing device such as a personal computer, a server, or a workstation that estimates an index indicating the atmosphere of a place formed in the space 2 and a correction value related to a thermal sensation in the space 2. The estimation device 10 acquires personal condition information acquired by a sensor 3-1 and interaction information acquired by a sensor 3-2, and outputs a correction value related to a thermal sensation to the space control device 20. The estimation device 10 may output the index indicating the atmosphere of the place or the correction value related to a thermal sensation to a display, a speaker, or the like installed in the space 2.

[0053] The index indicating the atmosphere of the place is an index indicating the ease of talking in the place. The atmosphere of the place includes, for example, a subjective report of the atmosphere of the place, an objective evaluation of the atmosphere of the place, a group flow degree, and a total amount of communication. The subjective report of the atmosphere of the place is an index related to the atmosphere of the place experimentally obtained in advance using a questionnaire method. The objective evaluation of the atmosphere of the place is a third-party evaluation of the atmosphere of the place, or a result of evaluating the atmosphere of the place based on objective data such as the amount of speech.

[0054] Survey methods include, for example, the Likert scale, the visual analogue scale (VAS), the numeric rating scale, the face scale, and the semantic differential method.

[0055] The Likert scale is a method of giving an evaluation value in a predetermined number of stages by questionnaire. For example, the evaluation value of talkability can be calculated by a participant or a third party other than the participant inputting an evaluation value at a predetermined time interval while playing back a video of the situation. The evaluation value of talkability is, for example, a 5-stage evaluation value with 1 being "difficult to talk to" and 5 being "easy to talk to."

[0056] Group flow is an index based on the flow theory in psychology. The closer a group is to a flow state, the higher the group flow is. A flow state is a state in which a group exerts a high level of concentration on a task.

[0057] The correction value for thermal sensation is, for example, a thermal index correction value. The thermal index correction value is a value for correcting the thermal index. The thermal index includes, for example, a predicted mean thermal sensation vote (PMV) and a standard new effective temperature (SET*). The thermal index is calculated based on six thermal environment elements consisting of temperature, humidity, radiation, airflow, activity level, and amount of clothing.

[0058] The thermal index correction value is a value indicating the difference between the thermal index when the atmosphere of the place is taken into consideration and the thermal index when the atmosphere of the place is not taken into consideration. For example, when the thermal index correction value is +1, it means that the thermal index when the atmosphere of the place is taken into consideration is 1 unit larger than the thermal index when the atmosphere of the place is not taken into consideration.

[0059] The estimation device 10 may acquire participant information related to attributes of each participant 1. The participant information includes, for example, at least one of information indicating the scene of the meeting, age, sex, physical build, and positional relationship of the participants.

[0060] The estimation device 10 includes a trained atmosphere estimation model 11 and a trained thermal index estimation model 12. The estimation device 10 may obtain feedback information from the space 2 and update the atmosphere estimation model 11 and the thermal index estimation model 12.

[0061] The atmosphere estimation model 11 is a model that estimates an index indicating the atmosphere of a place based on personal state information and interaction information, using an evaluation value that evaluates the atmosphere of the place as correct answer data. The atmosphere estimation model 11 may further estimate an index indicating the atmosphere of the place based on participant information.

[0062] The thermal index estimation model 12 is a model that estimates a correction value for a thermal sensation based on an index indicating the atmosphere of a place. The thermal index estimation model 12 may further estimate a correction value for a thermal sensation based on participant information.

[0063] The atmosphere estimation model 11 and the thermal index estimation model 12 may be, for example, a model based on machine learning or a model based on a rule base. Machine learning may be, for example, supervised machine learning. The model based on machine learning includes, for example, a classifier based on narrow machine learning or a classifier based on deep learning. The narrow machine learning means machine learning excluding deep learning.

[0064] Examples of classifiers based on narrow machine learning include support vector machines and random forests. Examples of classifiers based on deep learning include convolutional neural networks (CNNs) and long short-term memories (LSTMs). Rule-based models include statistical analysis methods, mathematical models, maps, tables, patterns, and the like.

[0065] The feedback information includes an evaluation value of the atmosphere of the place by participant 1 or a third party other than participant 1, or an evaluation value of the thermal sensation of the place by participant 1 after control is performed in space 2. The evaluation value of the atmosphere of the place includes a subjective report of the atmosphere of the place input by participant 1, or an objective evaluation of the atmosphere of the place input by a third party. The evaluation value of the thermal sensation of the place includes a subjective report of the thermal sensation input by participant 1. The feedback information may include an amount of communication observed by sensor 3-2 after control is performed in space 2.

[0066] The evaluation value of the atmosphere of the place and the amount of communication included in the feedback information are used to update the atmosphere estimation model 11. The evaluation value of the thermal sensation included in the feedback information is used to update the thermal index estimation model 12.

[0067] The space control device 20 is a facility device that controls the space 2 based on the correction value related to the thermal sensation output by the estimation device 10. The space control device 20 has at least one of the functions of air conditioning (room temperature or humidity, etc.), airflow control, lighting control, fragrance control, sound control, video control, etc. The space control device 20 may include multiple facility devices for each control target. The space control device 20 may control the space 2 according to the operation of the participant 1.

[0068] The overall configuration of the space control system 1000 shown in FIG. 2 is an example, and various system configuration examples are possible depending on the application and purpose. For example, one or more of the estimation device 10 and the space control device 20 may be included in the space control system 1000. For example, the estimation device 10 may be realized by a plurality of computers, or may be realized as a cloud computing service. In addition, for example, the space control system 1000 may be configured from one space control device 20 by implementing the functions to be provided by the estimation device 10 in a control unit provided in the space control device 20. The division of devices such as the estimation device 10 and the space control device 20 shown in FIG. 2 is an example.

[0069] <Hardware configuration> Fig. 3 is a block diagram showing an example of a hardware configuration of the estimation device in this embodiment. As shown in Fig. 3, the estimation device 10 includes a processor 101, a memory 102, an auxiliary storage device 103, an operation device 104, a display device 105, a communication device 106, and a drive device 107. Each piece of hardware of the estimation device 10 is connected to each other via a bus 108.

[0070] The processor 101 includes various arithmetic devices such as a CPU (Central Processing Unit), etc. The processor 101 reads out various programs installed in the auxiliary storage device 103 onto the memory 102 and executes them.

[0071] The memory 102 has a main storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The processor 101 and the memory 102 form a so-called computer (hereinafter also referred to as a "control unit"), and the processor 101 executes various programs read onto the memory 102, whereby the computer realizes various functions.

[0072] The auxiliary storage device 103 (hereinafter also referred to as a “storage unit”) stores various programs and various data used when the processor 101 executes the various programs.

[0073] The operation device 104 is an operation device that allows a user of the estimation device 10 to perform various operations. The display device 105 is a display device that displays the results of various processes executed by the estimation device 10.

[0074] The communication unit 106 is a communication device for communicating with external devices via a network (not shown).

[0075] The drive device 107 is a device for setting the storage medium 109. The storage medium 109 here includes media that store information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, a magneto-optical disk, etc. The storage medium 109 may also include semiconductor memory that stores information electrically, such as a ROM, a flash memory, etc.

[0076] The various programs to be installed in the auxiliary storage device 103 are installed, for example, by setting the distributed storage medium 109 in the drive device 107 and reading out the various programs stored in the storage medium 109 by the drive device 107. Alternatively, the various programs to be installed in the auxiliary storage device 103 may be installed by being downloaded from a network via the communication device 106.

[0077] <Flow of spatial control method> FIG. 4 is a flowchart showing an example of the flow of a space control method executed by the space control system 1000 in this embodiment.

[0078] In step S1, the control unit of the estimation device 10 generates an atmosphere estimation model 11. Specifically, the control unit first generates learning data for generating the atmosphere estimation model 11. Next, the control unit generates the atmosphere estimation model 11 based on the learning data. Then, the control unit stores the atmosphere estimation model 11 in the storage unit.

[0079] Fig. 5 is a diagram showing an example of training data. As shown in Fig. 5, the training data in this embodiment is a data set having individual state information, interaction information, participant information, and correct answer data as data items.

[0080] The personal condition information includes at least one of, for example, information on electrocardiogram, information on respiration, information on skin potential, information on facial expression, information on pupil diameter, information on gaze, and information on skin temperature.

[0081] The interaction information includes at least one of, for example, information regarding posture, information regarding conversation, information regarding gaze, information regarding the relative position of gaze, synchronization of electrocardiogram-related features, synchronization of respiratory features, synchronization of skin potential, synchronization of posture angle, etc.

[0082] The participant information includes, for example, at least one of information indicating the scene of the meeting, age, sex, physical build, positional relationship of the participants, etc. Note that the participant information does not have to be included in the learning data.

[0083] The correct answer data is an evaluation value obtained by evaluating the atmosphere of the place. The evaluation value obtained by evaluating the atmosphere of the place includes, for example, at least one of a subjective report of the atmosphere of the place, an objective evaluation of the atmosphere of the place, and the like.

[0084] When the mood estimation model 11 is a support vector machine, for example, the mood estimation model 11 can be generated as follows. First, learning data is collected through an experiment. In the experiment, for example, an ice-breaking test is conducted for 20 minutes in groups of four people, and biometric information and subjective declaration of ease of conversation of each participant are obtained.

[0085] Next, the acquired biometric information and subjective report are averaged at two-minute intervals. Next, the video of the ice-breaker is played, and each participant evaluates the ease of talking on a five-point scale at two-minute intervals. Then, the model parameters of the atmosphere estimation model 11 are learned according to a supervised machine learning algorithm, with the biometric information and subjective report as explanatory variables and the evaluation value of ease of talking as the objective variable.

[0086] Fig. 6 is a diagram showing an example of training data of a support vector machine. As shown in Fig. 6, the training data is a data set including, as data items, identification information indicating a subject, heart rate, respiratory rate, posture, gaze, ease of speaking, etc. The heart rate, respiratory rate, posture, gaze, etc. correspond to explanatory variables, and ease of speaking corresponds to a target variable. Each item of the training data shown in Fig. 6 is an average value at a 2-minute interval.

[0087] When the mood estimation model 11 is a convolutional neural network, for example, the mood estimation model 11 can be generated as follows. First, learning data is collected through an experiment. In the experiment, for example, an ice-breaking test is conducted for 20 minutes in groups of four people, and biometric information and subjective declaration of ease of conversation of each participant are obtained.

[0088] Next, two minutes of time-series image data is sampled for each participant. In addition, feature data is generated for each participant based on the biometric information and subjective report corresponding to each sample. The feature data is two-dimensional data with a feature axis and a time axis. Next, the video of the ice-breaker is played back, and each participant evaluates the ease of speaking on a five-point scale at two-minute intervals. Then, the model parameters of a convolutional neural network that inputs each participant's feature data and outputs an estimate of ease of speaking are trained.

[0089] The feature data input to the convolutional neural network is all combinations of two or more participants. When inputting a combination of two or three participants, the feature data corresponding to the other participants can be set to 0.

[0090] FIG. 7 is a diagram showing an example of training data of a convolutional neural network. As shown in FIG. 7, the training data is a data set including feature data and correct answer data. The feature data is two-dimensional data for each participant, and has a feature axis and a time axis. The feature axis corresponds to each explanatory variable. The time axis corresponds to each sample. The explanatory variables include heart rate, respiratory rate, tonic component of skin potential, phasic component of skin potential, posture, gaze, relative position of gaze, pupil, and facial expression. Ease of speaking is a value averaged over a 2-minute interval.

[0091] In step S2, the control unit of the estimation device 10 generates the thermal index estimation model 12. Specifically, the control unit first generates learning data for generating the thermal index estimation model 12. Next, the control unit generates the thermal index estimation model 12 based on the learning data. Then, the control unit stores the thermal index estimation model 12 in the storage unit.

[0092] The thermal index estimation model 12 can be generated, for example, as follows. First, learning data is collected through an experiment. In the experiment, for example, a 20-minute ice-breaking test is conducted in groups of four people, and the biometric information and subjective report of each participant is obtained. The subjective report covers thermal sensation and ease of conversation. The subjective report of thermal sensation may be, for example, a VAS that answers "hot" to "cold". The subjective report of ease of conversation may be rated on a five-point scale from "easy to talk to" to "difficult to talk to".

[0093] Next, a correlation analysis is performed on the average talkability score and the average thermal sensation score in each group of four people. Then, a thermal index estimation model 12 is generated, which takes the estimated talkability score as an input and outputs a corrected thermal index score.

[0094] FIG. 8 is a diagram showing an example of the relationship between the atmosphere of a place and warmth. As shown in FIG. 8, the ease of talking in the whole place (average of four people) increased before and after the ice-breaking test. In addition, the warmth of the whole place increased before and after the ice-breaking test. According to the example shown in FIG. 8, it can be seen that there is a relationship in which an increase in the ease of talking in a place increases the warmth of the place.

[0095] In step S3, the control unit of the estimation device 10 acquires observation data from the sensor 3-1. Next, the control unit generates personal state information for each participant 1 based on the observation data of the sensor 3-1.

[0096] In step S4, the control unit of the estimation device 10 acquires observation data from the sensor 3-2. Next, the control unit generates interaction information between the multiple participants 1 based on the observation data of the sensor 3-2.

[0097] In step S5, the control unit of the estimation device 10 estimates the atmosphere of the place based on the personal state information acquired in step S3 and the interaction information acquired in step S4. Specifically, the control unit first reads out the atmosphere estimation model 11 from the storage unit. Next, the control unit inputs the personal state information and the interaction information to the atmosphere estimation model 11. The atmosphere estimation model 11 estimates an index indicating the atmosphere of the place based on the input personal state information and interaction information, and outputs the estimated value. The control unit acquires the index indicating the atmosphere of the place output from the atmosphere estimation model 11.

[0098] The control unit of the estimation device 10 may output an index indicating the atmosphere of the place to the space 2. For example, the control unit may display the index indicating the atmosphere of the place on a display or the like installed in the space 2. The indicator indicating the atmosphere of the place may be displayed in any manner, and for example, a numerical value indicating the index indicating the atmosphere of the place as a percentage may be displayed. At this time, the color, font, thickness, etc. of the number may be changed according to the indicator indicating the atmosphere of the place.

[0099] Also, for example, an indicator showing the atmosphere of the place may be displayed with an indicator showing "difficult to talk" at one end and "easy to talk" at the other end. This indicator may be displayed, for example, as a bar extending from the "difficult to talk" end to the "easy to talk" end, with the tip showing the indicator showing the atmosphere of the place. In this case, the color or shape of the bar may be changed according to the indicator showing the atmosphere of the place.

[0100] In step S6, the control unit of the estimation device 10 estimates a thermal index correction value based on the index indicating the atmosphere of the place acquired in step S5. Specifically, the control unit first reads out the thermal index estimation model 12 from the storage unit. Next, the control unit inputs the index indicating the atmosphere of the place to the thermal index estimation model 12. The thermal index estimation model 12 estimates a thermal index correction value based on the input index indicating the atmosphere of the place, and outputs the estimated value. The control unit acquires the thermal index correction value output from the thermal index estimation model 12.

[0101] In step S7, the control unit of the estimation device 10 generates a control signal for controlling the space 2 based on the thermal index correction value acquired in step S6. Next, the control unit transmits the control signal to the space control device 20. The space control device 20 receives the control signal from the estimation device 10. Then, the space control device 20 controls the space 2 in accordance with the control signal.

[0102] The content of the control signal differs depending on the function of the space control device 20. For example, if the space control device 20 has a function of performing air conditioning based on the thermal index correction value, the control unit of the estimation device 10 transmits a control signal including the thermal index correction value to the space control device 20. The thermal index correction value included in the control signal is the thermal index correction value acquired in step S6.

[0103] Furthermore, for example, if the space control device 20 has a function of performing air conditioning based on a thermal index, the control unit of the estimation device 10 transmits a control signal including a thermal index to the space control device 20. The thermal index included in the control signal is a value obtained by correcting the thermal index calculated from the six thermal environment elements without considering the atmosphere of the place based on the thermal index correction value acquired in step S6.

[0104] Also, for example, if the space control device 20 has a function of performing air conditioning based on a target temperature, the control unit of the estimation device 10 transmits a control signal including the target temperature to the space control device 20. The target temperature included in the control signal is a value obtained by converting the thermal index correction value acquired in step S6 according to a predetermined rule. The correspondence between the thermal index correction value and the target temperature may be set in advance and stored in a storage unit or the like. For example, if the thermal index correction value is +1, a rule may be established such that the target temperature is lowered by 1°C.

[0105] In step S8, the control unit of the estimation device 10 acquires feedback information. The feedback information may include at least one of an evaluation value for evaluating the atmosphere of the place and an evaluation value for evaluating the thermal sensation of the place. After that, the control unit returns the process to step S1.

[0106] In step S1 from the second time onwards, the control unit of the estimation device 10 updates the atmosphere estimation model 11 based on the feedback information acquired in the immediately preceding step S8. In this case, the feedback information needs to include an evaluation value that evaluates the atmosphere of the place.

[0107] In step S2 from the second time onwards, the control unit of the estimation device 10 updates the thermal index estimation model 12 based on the feedback information acquired in the immediately preceding step S8. In this case, the feedback information needs to include an evaluation value that evaluates the thermal sensation of the place.

[0108] <Summary> As described above, according to each embodiment of the present disclosure, it is possible to output information for controlling the space in which the scene is formed. The estimation device 10 in this embodiment stores in a storage unit an atmosphere estimation model generated based on a data set including information on the state of each participant, information on the interaction between the participants, and an evaluation value that evaluates the atmosphere of the scene, and the atmosphere estimation model outputs information on the atmosphere of the scene based on the information on the state and the information on the interaction. Taking into account the information on the atmosphere of the scene, the participants can control the environment of the space in which the scene is formed. Therefore, according to this embodiment, it is possible to output information for controlling the space in which the scene is formed.

[0109] The estimation device 10 in this embodiment estimates an index indicating the atmosphere of a place based on information on the state of each participant and information on the interaction between the participants, and outputs a correction value for a thermal sensation in the space based on the index indicating the atmosphere of the place. For example, the estimation device 10 transmits a control signal including the thermal index correction value to the space control device 20, and the space control device 20 controls the environment of the space according to the control signal. Therefore, according to this embodiment, it is possible to output information for controlling the space in which a place is formed.

[0110] The estimation device 10 in this embodiment estimates an index indicating the atmosphere of the scene based on information on the state of each participant and information on the interaction between the participants, and controls the space in which the scene is formed based on the index indicating the atmosphere of the scene. For example, the estimation device 10 transmits a control signal for controlling the space to the space control device 20. Therefore, according to this embodiment, it is possible to output information for controlling the space in which the scene is formed.

[0111] The estimation device 10 in this embodiment estimates an index indicating the atmosphere of the place based on information on the state of each participant and information on the interaction between the participants, estimates a correction value for a thermal sensation in the space based on the index indicating the atmosphere of the place, and controls the space in which the place is formed based on the correction value for the thermal sensation. For example, the estimation device 10 transmits a control signal for controlling the space to the space control device 20. Therefore, according to this embodiment, it is possible to output information for controlling the space in which the place is formed.

[0112] The estimation device 10 in this embodiment may display an index indicating the atmosphere of the place or a correction value for thermal sensation in the space in which the place is formed. Participants who refer to the index indicating the atmosphere of the place or the correction value for thermal sensation can take actions according to the atmosphere or thermal sensation of the place. For example, participants can change the temperature setting of the air conditioner, eat or drink, open or close a window, start or stop exercising, etc. Therefore, according to this embodiment, participants can be encouraged to change their behavior to control the space.

[0113] [supplement] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit) implemented by an electronic circuit, and an ASIC (Application Specific Integrated Circuit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), conventional circuit module, and other devices designed to execute each function described above.

[0114] Although the embodiments have been described above, it will be understood that various changes in form and details are possible without departing from the spirit and scope of the claims. [Explanation of symbols]

[0115] 1 participant 2 space 3 Sensors 10 Estimation device 11 Atmosphere estimation model 12 Thermal index estimation model 20 Space Control Device 1000 Spatial Control System

Claims

1. An information processing device having a control unit that outputs information about an atmosphere of a place formed by a plurality of participants, The control unit is storing an atmosphere estimation model generated based on a data set including information on the state of each participant, information on the interaction between the participants, and an evaluation value that evaluates the atmosphere of the event in a storage unit; The atmosphere estimation model is a model for estimating information about the atmosphere of the place based on information about the state and information about the interaction. Information processing device.

2. The control unit is estimating an indicator of the atmosphere of the place based on the information about the state and the information about the interaction; outputting a correction value for a thermal sensation in a space where the scene is formed based on an index indicating the atmosphere of the scene; The information processing device according to claim 1 .

3. The control unit is estimating an indicator of the atmosphere of the place based on the information about the state and the information about the interaction; and controlling a space in which the scene is formed based on an index indicating the atmosphere of the scene. The information processing device according to claim 1 .

4. The control unit is estimating an indicator of the atmosphere of the place based on the information about the state and the information about the interaction; estimating a correction value for a thermal sensation in a space where the scene is formed based on an index indicating the atmosphere of the scene; and controlling the space based on the correction value related to the thermal sensation. The information processing device according to claim 1 .

5. The information regarding the status is biometric information of the participant.

5. The information processing device according to claim 1.

6. The biometric information is biometric information related to emotions of the participants. The information processing device according to claim 5 .

7. the information about the interaction is information about communication between the participants; 5. The information processing device according to claim 1.

8. The information regarding the communication is information regarding the communicative actions between the participants. The information processing device according to claim 7.

9. The control unit is Estimating the atmosphere of the place using supervised machine learning; 5. The information processing device according to claim 1.

10. The evaluation value of the atmosphere of the place is a subjective report or an objective evaluation of the atmosphere of the place.

5. The information processing device according to claim 1.

11. The subjective report of the atmosphere of the place is an index regarding the atmosphere of the place experimentally obtained in advance using a questionnaire method. The information processing device according to claim 10.

12. The control unit is Estimating an index indicating the atmosphere of the scene based on the trained atmosphere estimation model; 5. The information processing device according to claim 1.

13. the atmosphere estimation model receives information about the state and information about the interaction as input, and outputs an estimate of an index indicating the atmosphere of the place as correct answer data. The information processing device according to claim 12.

14. The atmosphere estimation model further inputs information regarding attributes of the participants. The information processing device according to claim 13.

15. The control unit is Obtaining feedback information from the participants regarding the atmosphere of the venue; updating the atmosphere estimation model based on the feedback information; The information processing device according to claim 12.

16. The feedback information includes an evaluation value that evaluates the atmosphere of the place. The information processing device according to claim 15.

17. The control unit is Estimating a correction value related to the thermal sensation based on a trained thermal index estimation model; 5. The information processing device according to claim 2 or 4.

18. The thermal index estimation model receives an index indicating the atmosphere of the place estimated based on the information on the state and the information on the interaction as an input, and outputs an estimate of the correction value related to the thermal sensation. The information processing device according to claim 17.

19. The thermal index estimation model further inputs information regarding the attributes of the participants. The information processing device according to claim 18.

20. The control unit is Obtaining feedback information regarding the thermal sensation of the place from the participants; updating the thermal index estimation model based on the feedback information; The information processing device according to claim 17.

21. The feedback information includes an evaluation value evaluating the thermal sensation of the place. The information processing device according to claim 20.

22. A control unit included in an information processing device that outputs information about an atmosphere of a place formed by a plurality of participants, storing an atmosphere estimation model generated based on a data set including information on the state of each participant, information on the interaction between the participants, and an evaluation value that evaluates the atmosphere of the event in a storage unit; The atmosphere estimation model is a model for estimating information about the atmosphere of the place based on information about the state and information about the interaction. Information processing methods.

23. A control unit of an information processing device that outputs information about an atmosphere of a place formed by a plurality of participants, a process of storing, in a storage unit, an atmosphere estimation model generated based on a data set including information on the states of the individual participants, information on the interactions between the participants, and an evaluation value that evaluates the atmosphere of the place; The atmosphere estimation model is a model for estimating information about the atmosphere of the place based on information about the state and information about the interaction. program.

Images