Information processing device, information processing method, and program

Abstract

To provide an information processing apparatus which outputs information for controlling a space.SOLUTION: An information processing apparatus includes a control unit configured to output information on the atmosphere in a space formed by a plurality of participants. The information processing apparatus stores, in a storage unit, an atmosphere estimation model generated based on information on a state of each of the participants, information on interactions between the participants, and a dataset including an evaluation value obtained by evaluating the atmosphere of the space. The atmosphere estimation model is a model for estimating information on the atmosphere in a space on the basis of state information and interaction information.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] There are known techniques for estimating the atmosphere of a place formed by multiple participants. For example, Patent Document 1 discloses an atmosphere estimation device that analyzes the behavior and emotions of multiple members from images of a place including multiple members and audio recorded from the place, and infers that the atmosphere of the place is bad if the discomfort level of other members increases after the behavior of one member, or if the mutual relationships between the multiple members are weak. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-185117 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the prior art does not take into consideration controlling the space in which the scene is formed according to the atmosphere of the scene.

[0005] The present disclosure provides a technology for outputting information for controlling the space in which a field is formed. [Means for solving the problem]

[0006] An information processing device according to a first aspect of the present disclosure is an information processing device having a control unit that outputs information regarding the atmosphere of a place formed by multiple participants, wherein the control unit stores in a memory unit an atmosphere estimation model generated based on a dataset including information regarding the state of each participant, information regarding interactions between the participants, and an evaluation value that evaluates the atmosphere of the place, and the atmosphere estimation model is a model for estimating information regarding the atmosphere of the place based on the information regarding the state and the information regarding the interactions.

[0007] According to the first aspect of the present disclosure, it is possible to output information for controlling the space in which the field is formed.

[0008] A second aspect of the present disclosure is an information processing device according to the first aspect, wherein a control unit estimates an index indicating the atmosphere of a place based on information regarding the state and information regarding interactions, and outputs a correction value regarding the thermal sensation in the space in which the place is formed based on the index indicating the atmosphere of the place.

[0009] According to the second aspect of the present disclosure, it is possible to output a correction value related to a thermal sensation in a space where a field is formed.

[0010] A third aspect of the present disclosure is an information processing device relating to the first aspect, wherein a control unit estimates an index indicating the atmosphere of the place based on information regarding the state and information regarding the interaction, and controls the space in which the place is formed based on the index indicating the atmosphere of the place.

[0011] According to the third aspect of the present disclosure, the space in which the venue is formed can be controlled according to the atmosphere of the venue.

[0012] A fourth aspect of the present disclosure is an information processing device according to the first aspect, wherein a control unit estimates an index indicating the atmosphere of a place based on information regarding the state and information regarding interactions, estimates a correction value for the thermal sensation in the space in which the place is formed based on the index indicating the atmosphere of the place, and controls the space based on the correction value for the thermal sensation.

[0013] According to the fourth aspect of the present disclosure, the space in which the venue is formed can be controlled according to the atmosphere of the venue.

[0014] A fifth aspect of the present disclosure is the information processing device according to any one of the first to fourth aspects, wherein the information relating to the status is biometric information of the participant.

[0015] A sixth aspect of the present disclosure is the information processing device according to the fifth aspect, wherein the biometric information is biometric information related to emotions of participants.

[0016] A seventh aspect of the present disclosure is the information processing device according to any one of the first to sixth aspects, wherein the information regarding the interaction is information regarding communication between participants.

[0017] An eighth aspect of the present disclosure is the information processing device according to the seventh aspect, wherein the information related to communication is information related to behavior of communicating between participants.

[0018] A ninth aspect of the present disclosure is the information processing device according to any one of the first to eighth aspects, wherein the control unit estimates the atmosphere of a place by supervised machine learning.

[0019] A tenth aspect of the present disclosure is the information processing device according to any one of the first to ninth aspects, wherein the evaluation value of the atmosphere of the place is a subjective declaration or an objective evaluation of the atmosphere of the place.

[0020] An eleventh aspect of the present disclosure is the information processing device according to the tenth aspect, wherein the subjective declaration of the atmosphere of the place is an index relating to the atmosphere of the place experimentally obtained in advance using a questionnaire method.

[0021] A twelfth aspect of the present disclosure is an information processing device according to any one of the first to eleventh aspects, wherein the control unit estimates an index indicating the atmosphere of a place based on a trained atmosphere estimation model.

[0022] A thirteenth aspect of the present disclosure is an information processing device according to the twelfth aspect, wherein the atmosphere estimation model takes information related to the state and information related to the interaction as input, uses an evaluation value that evaluates the atmosphere of the place as correct data, and outputs an estimated value of an index that indicates the atmosphere of the place.

[0023] A fourteenth aspect of the present disclosure is the information processing device according to the thirteenth aspect, wherein the atmosphere estimation model further receives information on attributes of participants as an input.

[0024] A fifteenth aspect of the present disclosure is an information processing device relating to any of the twelfth to fourteenth aspects, in which the control unit acquires feedback information regarding the atmosphere of the venue from the participants and updates the atmosphere estimation model based on the feedback information.

[0025] A sixteenth aspect of the present disclosure is the information processing device according to the fifteenth aspect, wherein the feedback information includes an evaluation value that evaluates the atmosphere of the place.

[0026] A seventeenth aspect of the present disclosure is the information processing device according to the second or fourth aspect, wherein the control unit estimates a correction value related to the thermal sensation based on the trained thermal index estimation model.

[0027] An 18th aspect of the present disclosure is an information processing device related to the 17th aspect, in which a thermal index estimation model takes as input an index indicating the atmosphere of a place estimated based on information regarding the state and information regarding interactions, and outputs an estimated value of a correction value related to thermal sensation.

[0028] A nineteenth aspect of the present disclosure is the information processing device according to the eighteenth aspect, wherein the thermal index estimation model further receives as input information on attributes of participants.

[0029] A 20th aspect of the present disclosure is an information processing device relating to any of the 17th to 19th aspects, wherein the control unit acquires feedback information regarding the thermal sensation of the venue from participants and updates the thermal index estimation model based on the feedback information.

[0030] A twenty-first aspect of the present disclosure is the information processing device according to the twentieth aspect, wherein the feedback information includes an evaluation value that evaluates the thermal sensation of the space.

[0031] An information processing method according to a 22nd aspect of the present disclosure includes an information processing device that outputs information about the atmosphere of a place formed by multiple participants, wherein a control unit of the information processing device stores in a memory unit an atmosphere estimation model generated based on a dataset including information about the state of each participant, information about interactions between the participants, and an evaluation value that evaluates the atmosphere of the place, and the atmosphere estimation model is a model for estimating information about the atmosphere of the place based on the information about the state and information about interactions.

[0032] A program according to a 23rd aspect of the present disclosure causes a control unit of an information processing device that outputs information regarding the atmosphere of a place formed by multiple participants to execute a process of storing in a memory unit an atmosphere estimation model generated based on a dataset including information regarding the state of each participant, information regarding interactions between the participants, and an evaluation value that evaluates the atmosphere of the place, and the atmosphere estimation model is a model for estimating information regarding the atmosphere of the place based on information regarding the state and information regarding interactions. [Brief explanation of the drawings]

[0033] [Figure 1] FIG. 10 is a diagram illustrating an example of the relationship between the emotions of individual participants and the communication states between the participants. [Figure 2] 1 is a block diagram showing an example of the overall configuration of a spatial control system. [Figure 3] FIG. 2 is a block diagram illustrating an example of a hardware configuration of the estimation device. [Figure 4] 1 is a flowchart illustrating an example of a spatial control method. [Figure 5] FIG. 10 is a diagram illustrating an example of learning data. [Figure 6] FIG. 10 is a diagram illustrating an example of training data for a support vector machine. [Figure 7]FIG. 10 is a diagram illustrating an example of training data for a convolutional neural network. [Figure 8] FIG. 10 is a diagram illustrating an example of the relationship between the atmosphere of a place and a thermal sensation. DETAILED DESCRIPTION OF THE INVENTION

[0034] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0035] [Embodiment] One embodiment of the present disclosure is a space control system that controls a space where a place is formed by multiple participants. The space control system uses an evaluation value that evaluates the atmosphere of the place as correct data and estimates the atmosphere of the place based on information about the state of each participant who forms the place and information about the interaction between the participants. The space control system also estimates a correction value for thermal sensation based on an index that indicates the atmosphere of the place. Furthermore, the space control system controls the space based on the correction value for thermal sensation.

[0036] For example, in a place where multiple participants gather to have a conversation, it is necessary to provide an atmosphere in which each participant feels comfortable speaking up. For example, in a meeting aimed at generating ideas, even if a participant has an idea, if the atmosphere in the room makes it difficult for them to speak up, they may hesitate to speak up, and ideas may not be shared smoothly among participants. Therefore, there is a need for technology that can estimate the atmosphere of a place formed by multiple participants.

[0037] It is also believed that participants' thermal sensations change depending on the atmosphere of the room. For example, if the meeting is heated, participants may feel hotter than the actual room temperature. On the other hand, if the meeting participants are tense, participants may feel colder than the actual room temperature. Therefore, there is a need for technology that can estimate thermal sensations taking the atmosphere of the room into account and control the space appropriately.

[0038] The atmosphere of a place can be estimated with high accuracy by detecting both the emotions of individual participants and the communication between them. Figure 1 is a diagram showing an example of the relationship between the emotions of individual participants and the communication state between them. Figure 1 shows the emotions of four participants A to D and the communication state between them. Note that Figure 1 classifies the emotions of participants into four emotions: joy, anger, sadness, and pleasure. Furthermore, the thicker the arrows connecting participants, the more active the communication.

[0039] As shown in Figure 1, Participant B is expressing emotions of joy and is actively communicating with Participant A and Participant D. In this case, it can be assumed that the atmosphere is one in which Participant B feels comfortable talking. Participant A is expressing emotions of sadness, but is actively communicating with Participant B and Participant D. In this case, it can be assumed that the atmosphere is one in which Participant A feels somewhat comfortable talking. Participant C is expressing emotions of joy, but is not actively communicating with the other Participants A, B, and D. In this case, it can be assumed that the atmosphere is one in which Participant C feels uncomfortable talking.

[0040] This embodiment aims to output information for controlling a space formed by multiple participants. As an example, this embodiment outputs an index indicating the atmosphere of the space formed by multiple participants. Also, as an example, this embodiment outputs a correction value for thermal sensation based on the index indicating the atmosphere of the space. Furthermore, as an example, this embodiment outputs a control signal for controlling the space based on the correction value for thermal sensation.

[0041] In one aspect, participants can refer to an indicator that indicates the atmosphere of a place and select an action to appropriately control the space. For example, when the atmosphere of a place is active (or inactive), participants can select an action to lower (or raise) the set temperature of an air conditioner. Also, for example, when the correction value for thermal sensation is a positive value (or a negative value), participants can select an action to lower (or raise) the set temperature of an air conditioner.

[0042] In another aspect, the space can be automatically controlled appropriately according to the atmosphere of the place. For example, when the atmosphere of the place becomes lively (or inactive), the set temperature of the air conditioner can be automatically lowered (or automatically raised). Also, for example, when the correction value for thermal sensation becomes a positive value (or a negative value), the set temperature of the air conditioner can be automatically lowered (or automatically raised).

[0043] <Overall structure> Fig. 2 is a block diagram showing the overall configuration of a space control system according to this embodiment. As shown in Fig. 2, the space control system 1000 according to this embodiment includes an estimation device 10 and a space control device 20. The space control system 1000 is a control system that controls equipment installed in a space 2 in which multiple participants 1 (1-1 to 1-3) exist. A sensor 3-1 that acquires personal state information and a sensor 3-2 that acquires interaction information are installed in the space 2.

[0044] The sensor 3-1 is an electronic device that measures the state of each participant 1. The sensor 3-1 may include, for example, a wearable device attached to each participant 1, or a video camera that captures images of each participant 1. A plurality of sensors 3-1 may be installed depending on the type of measurable information.

[0045] The sensor 3-2 is an electronic device that measures information about interactions between multiple participants 1. The sensor 3-2 may include, for example, a video camera, a microphone, or a motion sensor. Multiple sensors 3-2 may be installed depending on the type of information that can be measured.

[0046] The personal status information is information indicating the status of each participant 1. An example of the personal status information is biometric information that can be obtained by observing the status of each participant 1. The biometric information may be biometric information related to the emotions of each participant 1. The biometric information related to emotions means biometric information that can be used to estimate emotions.

[0047] Specifically, the personal condition information includes at least one of electrocardiographic information, respiration information, skin potential information, facial expression information, pupil diameter information, gaze information, and skin temperature information.

[0048] The electrocardiographic information includes, for example, heart rate, low frequency (LF) components of the electrocardiogram waveform, high frequency (HF) components of the electrocardiogram waveform, cardiac vagal index (CVI), and cardiac sympathetic index (CSI). The respiration information includes, for example, breathing frequency and breathing depth. The facial expression information includes, for example, the proportion of a predetermined emotion (e.g., happy) in image-based facial expression recognition. The gaze information includes, for example, a label indicating where the gaze position is on the face, body, background, etc. The gaze position can be acquired, for example, by eye tracking technology.

[0049] Interaction information is information that indicates interactions between multiple participants 1. An example of interaction information is information about communication between multiple participants 1. Information about communication is information about communicative behavior between multiple participants 1. Communicative behavior may be, for example, conversation or body language.

[0050] Specifically, the interaction information includes at least one of information regarding posture, information regarding conversation, information regarding gaze, information regarding the relative position of gaze, synchronization of electrocardiogram-related features, synchronization of respiration-related features, synchronization of skin potential, and synchronization of posture angle.

[0051] The information about posture includes, for example, the angle of forward body lean. The information about conversation includes, for example, the proportion of speech volume, the order of conversation, and keywords used in the conversation. The information about the relative position of gaze includes, for example, a label indicating which participant the gaze position is directed toward. The synchronization of feature amounts includes, for example, the synchronization rate of feature amounts between multiple participants 1. The synchronization can be calculated using, for example, dynamic time warping (DTW), Euclidean distance, cosine similarity, etc. The synchronization of posture angles includes, for example, the state of posture coordination between multiple participants 1.

[0052] The estimation device 10 is an information processing device such as a personal computer, a server, or a workstation that estimates an index indicating the atmosphere of a place formed in the space 2 and a correction value for a thermal sensation in the space 2. The estimation device 10 acquires personal condition information acquired by the sensor 3-1 and interaction information acquired by the sensor 3-2, and outputs a correction value for a thermal sensation to the space control device 20. The estimation device 10 may output the index indicating the atmosphere of the place or the correction value for a thermal sensation to a display, a speaker, or the like installed in the space 2.

[0053] The indicator of the atmosphere of the place is an indicator of the ease of talking in the place. The atmosphere of the place includes, for example, a subjective report of the atmosphere of the place, an objective evaluation of the atmosphere of the place, the degree of group flow, and the total amount of communication. The subjective report of the atmosphere of the place is an indicator related to the atmosphere of the place obtained experimentally in advance using a questionnaire method. The objective evaluation of the atmosphere of the place is a result of evaluating the atmosphere of the place based on a third-party evaluation of the atmosphere or objective data such as the amount of speech.

[0054] Survey methods include, for example, the Likert scale, the visual analogue scale (VAS), the numeric rating scale, the face scale, and the semantic differential method.

[0055] The Likert scale is a method of assigning evaluation values ​​in a predetermined number of stages in a questionnaire. For example, participants or a third party other than participants can calculate the evaluation value of ease of conversation by playing back a video of the situation and inputting evaluation values ​​at predetermined time intervals. The evaluation value of ease of conversation is, for example, a 5-stage evaluation value, with 1 being "difficult to talk to" and 5 being "easy to talk to."

[0056] Group flow is an index based on flow theory in psychology. The closer a group is to a flow state, the higher the group flow value. A flow state is a state in which a group is highly focused on a task.

[0057] The correction value for thermal sensation is, for example, a thermal index correction value. The thermal index correction value is a value for correcting the thermal index. The thermal index includes, for example, the predicted mean thermal vote (PMV) and the standard new effective temperature (SET*). The thermal index is calculated based on six thermal environment elements: temperature, humidity, radiation, airflow, activity level, and amount of clothing.

[0058] The thermal index correction value is a value that indicates the difference between the thermal index when the atmosphere of the place is taken into consideration and the thermal index when the atmosphere of the place is not taken into consideration. For example, if the thermal index correction value is +1, it means that the thermal index when the atmosphere of the place is taken into consideration is one unit larger than the thermal index when the atmosphere of the place is not taken into consideration.

[0059] The estimation device 10 may acquire participant information relating to the attributes of each participant 1. The participant information may include, for example, at least one of information indicating the scene of the meeting, age, gender, physique, and positional relationship of the participants.

[0060] The estimation device 10 includes a trained atmosphere estimation model 11 and a trained thermal index estimation model 12. The estimation device 10 may acquire feedback information from the space 2 and update the atmosphere estimation model 11 and the thermal index estimation model 12.

[0061] The atmosphere estimation model 11 is a model that estimates an index indicating the atmosphere of a place based on personal state information and interaction information, using an evaluation value that evaluates the atmosphere of the place as correct data. The atmosphere estimation model 11 may further estimate an index indicating the atmosphere of the place based on participant information.

[0062] The thermal index estimation model 12 is a model that estimates a correction value for the thermal sensation based on an index that indicates the atmosphere of the place. The thermal index estimation model 12 may further estimate a correction value for the thermal sensation based on participant information.

[0063] The atmosphere estimation model 11 and the thermal index estimation model 12 may be, for example, a model based on machine learning or a model based on a rule base. The machine learning may be, for example, supervised machine learning. The machine learning based model includes, for example, a classifier based on narrow machine learning or a classifier based on deep learning. The narrow machine learning means machine learning excluding deep learning.

[0064] Examples of classifiers based on narrow machine learning include support vector machines and random forests. Examples of classifiers based on deep learning include convolutional neural networks (CNNs) and long short-term memories (LSTMs). Rule-based models include statistical analysis methods, mathematical models, maps, tables, patterns, etc.

[0065] The feedback information includes an evaluation value of the atmosphere of the place by participant 1 or a third party other than participant 1, or an evaluation value of the thermal sensation of the place by participant 1 after control is performed in space 2. The evaluation value of the atmosphere of the place includes a subjective report of the atmosphere of the place input by participant 1, or an objective evaluation of the atmosphere of the place input by a third party. The evaluation value of the thermal sensation of the place includes a subjective report of the thermal sensation input by participant 1. The feedback information may include the amount of communication observed by sensor 3-2 after control is performed in space 2.

[0066] The evaluation value of the atmosphere of the place and the communication amount included in the feedback information are used to update the atmosphere estimation model 11. The evaluation value of the thermal sensation included in the feedback information is used to update the thermal index estimation model 12.

[0067] The space control device 20 is a facility device that controls the space 2 based on the correction value for thermal sensation output by the estimation device 10. The space control device 20 has at least one of the functions of air conditioning (room temperature or humidity, etc.), airflow control, lighting control, fragrance control, sound control, or video control. The space control device 20 may include multiple facility devices for each control target. The space control device 20 may control the space 2 according to the operation of the participant 1.

[0068] The overall configuration of the space control system 1000 shown in FIG. 2 is an example, and various system configuration examples are possible depending on the application and purpose. For example, the space control system 1000 may include multiple estimation devices 10 and one or more space control devices 20. For example, the estimation device 10 may be realized by multiple computers, or may be realized as a cloud computing service. Furthermore, for example, the space control system 1000 may be configured with one space control device 20 by implementing the functions that the estimation device 10 should have in a control unit that the space control device 20 has. The classification of devices such as the estimation device 10 and the space control device 20 shown in FIG. 2 is an example.

[0069] <Hardware configuration> Fig. 3 is a block diagram showing an example of the hardware configuration of the estimation device according to this embodiment. As shown in Fig. 3, the estimation device 10 includes a processor 101, a memory 102, an auxiliary storage device 103, an operation device 104, a display device 105, a communication device 106, and a drive device 107. The hardware components of the estimation device 10 are connected to each other via a bus 108.

[0070] The processor 101 has various arithmetic devices such as a CPU (Central Processing Unit), etc. The processor 101 reads out various programs installed in the auxiliary storage device 103 onto the memory 102 and executes them.

[0071] The memory 102 has a main storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The processor 101 and the memory 102 form a so-called computer (hereinafter also referred to as a "controller"), and the processor 101 executes various programs read onto the memory 102, thereby enabling the computer to realize various functions.

[0072] The auxiliary storage device 103 (hereinafter also referred to as a "storage unit") stores various programs and various data used when the processor 101 executes the various programs.

[0073] The operation device 104 is an operation device that allows a user of the estimation device 10 to perform various operations. The display device 105 is a display device that displays the results of various processes executed by the estimation device 10.

[0074] The communication device 106 is a communication device for communicating with external devices via a network (not shown).

[0075] Drive device 107 is a device for loading storage medium 109. The storage medium 109 here includes media that store information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks. Storage medium 109 may also include semiconductor memories that store information electrically, such as ROMs and flash memories.

[0076] The various programs to be installed in the auxiliary storage device 103 are installed, for example, by setting the distributed storage medium 109 in the drive device 107 and reading out the various programs stored in the storage medium 109 by the drive device 107. Alternatively, the various programs to be installed in the auxiliary storage device 103 may be installed by being downloaded from a network via the communication device 106.

[0077] <Flow of spatial control method> FIG. 4 is a flowchart showing an example of the flow of the space control method executed by the space control system 1000 in this embodiment.

[0078] In step S1, the control unit of the estimation device 10 generates an atmosphere estimation model 11. Specifically, the control unit first generates training data for generating the atmosphere estimation model 11. Next, the control unit generates the atmosphere estimation model 11 based on the training data. Then, the control unit stores the atmosphere estimation model 11 in the storage unit.

[0079] 5 is a diagram showing an example of training data. As shown in FIG. 5, the training data in this embodiment is a data set having individual state information, interaction information, participant information, and correct answer data as data items.

[0080] The personal condition information includes at least one of, for example, information on electrocardiogram, information on respiration, information on skin potential, information on facial expression, information on pupil diameter, information on gaze, and information on skin temperature.

[0081] The interaction information includes, for example, at least one of information regarding posture, information regarding conversation, information regarding gaze, information regarding the relative position of gaze, synchronization of electrocardiogram-related features, synchronization of breathing-related features, synchronization of skin potential, synchronization of posture angle, etc.

[0082] The participant information includes, for example, at least one of information indicating the scene of the meeting, age, sex, physique, positional relationship of the participants, etc. Note that the participant information does not have to be included in the learning data.

[0083] The correct answer data is an evaluation value that evaluates the atmosphere of the place. The evaluation value that evaluates the atmosphere of the place includes, for example, at least one of a subjective declaration of the atmosphere of the place, an objective evaluation of the atmosphere of the place, and the like.

[0084] When the mood estimation model 11 is a support vector machine, for example, the mood estimation model 11 can be generated as follows. First, learning data is collected through an experiment. In the experiment, for example, a 20-minute ice-breaking test is conducted in groups of four people, and biometric information and subjective assessment of ease of conversation are obtained from each participant.

[0085] Next, the acquired biometric information and subjective feedback are averaged at two-minute intervals. Subsequently, the video of the icebreaker is played back, and each participant evaluates the ease of speaking on a five-point scale at two-minute intervals. Then, using the biometric information and subjective feedback as explanatory variables and the evaluation value of ease of speaking as the objective variable, the model parameters of the atmosphere estimation model 11 are trained according to a supervised machine learning algorithm.

[0086] Fig. 6 is a diagram showing an example of training data for a support vector machine. As shown in Fig. 6, the training data is a data set including, as data items, identification information indicating the subject, heart rate, respiratory rate, posture, gaze, ease of speaking, etc. The heart rate, respiratory rate, posture, gaze, etc. correspond to explanatory variables, and ease of speaking corresponds to a target variable. Each item of the training data shown in Fig. 6 is an average value at two-minute intervals.

[0087] When the mood estimation model 11 is a convolutional neural network, for example, the mood estimation model 11 can be generated as follows. First, learning data is collected through an experiment. In the experiment, for example, a 20-minute ice-breaking test is conducted in groups of four people, and biometric information and subjective responses on ease of conversation are obtained from each participant.

[0088] Next, two minutes of time-series image data is sampled for each participant. Feature data is generated for each participant based on the biometric information and subjective assessment corresponding to each sample. The feature data is two-dimensional data with a feature axis and a time axis. Next, the recording of the icebreaker session is played back, and each participant rates the ease of speaking on a five-point scale at two-minute intervals. Finally, the feature data for each participant is used as input to train the model parameters of a convolutional neural network that outputs an estimated value for ease of speaking.

[0089] The feature data input to the convolutional neural network is all combinations of two or more participants. When inputting a combination of two or three participants, the feature data corresponding to the other participants can be set to 0.

[0090] FIG. 7 is a diagram showing an example of training data for a convolutional neural network. As shown in FIG. 7, the training data is a dataset including feature data and ground truth data. The feature data is two-dimensional data for each participant, and has a feature axis and a time axis. The feature axis corresponds to each explanatory variable. The time axis corresponds to each sample. The explanatory variables include heart rate, respiratory rate, tonic component of skin potential, phasic component of skin potential, posture, gaze direction, relative position of gaze direction, pupil, and facial expression. Ease of speaking is a value averaged over a two-minute interval.

[0091] In step S2, the control unit of the estimation device 10 generates a thermal index estimation model 12. Specifically, the control unit first generates learning data for generating the thermal index estimation model 12. Next, the control unit generates the thermal index estimation model 12 based on the learning data. Then, the control unit stores the thermal index estimation model 12 in the storage unit.

[0092] The thermal index estimation model 12 can be generated, for example, as follows. First, learning data is collected through an experiment. In the experiment, for example, a 20-minute ice-breaking test is conducted in groups of four people, and biometric information and subjective reports are obtained from each participant. The subjective reports are directed to thermal sensation and ease of conversation. The subjective report of thermal sensation may be, for example, a VAS in which participants answer "hot" to "cold." The subjective report of ease of conversation may be rated on a five-point scale from "easy to talk" to "difficult to talk."

[0093] Next, a correlation analysis is performed on the average values ​​of ease of conversation and thermal sensation in each group of four people, and a thermal index estimation model 12 is generated that inputs the estimated ease of conversation value and outputs a corrected thermal index value.

[0094] FIG. 8 is a diagram showing an example of the relationship between the atmosphere of a place and warmth. As shown in FIG. 8, the ease of talking in the entire place (average of four people) increased before and after the ice-breaking test. In addition, the warmth of the entire place increased before and after the ice-breaking test. The example shown in FIG. 8 shows that there is a relationship in which an increase in the ease of talking in a place increases the warmth of the place.

[0095] In step S3, the control unit of the estimation device 10 acquires observation data from the sensor 3-1. Next, the control unit generates personal state information for each participant 1 based on the observation data from the sensor 3-1.

[0096] In step S4, the control unit of the estimation device 10 acquires observation data from the sensor 3-2. Next, the control unit generates interaction information between the multiple participants 1 based on the observation data of the sensor 3-2.

[0097] In step S5, the control unit of the estimation device 10 estimates the atmosphere of the place based on the personal state information acquired in step S3 and the interaction information acquired in step S4. Specifically, the control unit first reads out the atmosphere estimation model 11 from the storage unit. Next, the control unit inputs the personal state information and the interaction information to the atmosphere estimation model 11. The atmosphere estimation model 11 estimates an index indicating the atmosphere of the place based on the input personal state information and interaction information, and outputs the estimated value. The control unit acquires the index indicating the atmosphere of the place output from the atmosphere estimation model 11.

[0098] The control unit of the estimation device 10 may output an index indicating the atmosphere of the place to the space 2. For example, the control unit may display the index indicating the atmosphere of the place on a display or the like installed in the space 2. The indicator indicating the atmosphere of the place may be displayed in any manner, for example, a numerical value indicating the index indicating the atmosphere of the place as a percentage may be displayed. In this case, the color, font, thickness, etc. of the number may be changed depending on the indicator indicating the atmosphere of the place.

[0099] Alternatively, an indicator showing the atmosphere of a place may be displayed, for example, with an indicator showing "difficult to talk" at one end and "easy to talk" at the other end. This indicator may be, for example, a bar extending from the "difficult to talk" end to the "easy to talk" end, with the tip of the bar showing the indicator showing the atmosphere of the place. In this case, the color or shape of the bar may be changed according to the indicator showing the atmosphere of the place.

[0100] In step S6, the control unit of the estimation device 10 estimates a thermal index correction value based on the index indicating the atmosphere of the place acquired in step S5. Specifically, the control unit first reads out the thermal index estimation model 12 from the storage unit. Next, the control unit inputs the index indicating the atmosphere of the place to the thermal index estimation model 12. The thermal index estimation model 12 estimates a thermal index correction value based on the input index indicating the atmosphere of the place and outputs the estimated value. The control unit acquires the thermal index correction value output from the thermal index estimation model 12.

[0101] In step S7, the control unit of the estimation device 10 generates a control signal for controlling the space 2 based on the thermal index correction value acquired in step S6. Next, the control unit transmits the control signal to the space control device 20. The space control device 20 receives the control signal from the estimation device 10. Then, the space control device 20 controls the space 2 in accordance with the control signal.

[0102] The content of the control signal differs depending on the function of the space control device 20. For example, if the space control device 20 has a function of performing air conditioning based on a thermal index correction value, the control unit of the estimation device 10 transmits a control signal including the thermal index correction value to the space control device 20. The thermal index correction value included in the control signal is the thermal index correction value acquired in step S6.

[0103] Furthermore, for example, if the space control device 20 has a function of performing air conditioning based on a thermal index, the control unit of the estimation device 10 transmits a control signal including a thermal index to the space control device 20. The thermal index included in the control signal is a value obtained by correcting the thermal index calculated from the six thermal environment elements without considering the atmosphere of the place based on the thermal index correction value acquired in step S6.

[0104] Furthermore, for example, if the space control device 20 has a function of performing air conditioning based on a target temperature, the control unit of the estimation device 10 transmits a control signal including the target temperature to the space control device 20. The target temperature included in the control signal is a value obtained by converting the thermal index correction value acquired in step S6 according to a predetermined rule. The correspondence between the thermal index correction value and the target temperature may be set in advance and stored in a memory unit or the like. For example, a rule may be established such that when the thermal index correction value is +1, the target temperature is lowered by 1°C.

[0105] In step S8, the control unit of the estimation device 10 acquires feedback information. The feedback information may include at least one of an evaluation value for the atmosphere of the place and an evaluation value for the thermal sensation of the place. Thereafter, the control unit returns the process to step S1.

[0106] In step S1 from the second time onwards, the control unit of the estimation device 10 updates the atmosphere estimation model 11 based on the feedback information acquired in the immediately preceding step S8. In this case, the feedback information needs to include an evaluation value that evaluates the atmosphere of the place.

[0107] In step S2 from the second time onwards, the control unit of the estimation device 10 updates the thermal index estimation model 12 based on the feedback information acquired in the immediately preceding step S8. In this case, the feedback information needs to include an evaluation value that evaluates the thermal sensation of the place.

[0108] <Summary> As described above, according to each embodiment of the present disclosure, it is possible to output information for controlling the space in which a scene is formed. The estimation device 10 in this embodiment stores in a storage unit an atmosphere estimation model generated based on a dataset including information on the state of each participant, information on interactions between participants, and an evaluation value that evaluates the atmosphere of the scene, and the atmosphere estimation model outputs information on the atmosphere of the scene based on the information on the state and information on the interactions. Participants can control the environment of the space in which a scene is formed by taking into account the information on the atmosphere of the scene. Therefore, according to this embodiment, it is possible to output information for controlling the space in which a scene is formed.

[0109] The estimation device 10 in this embodiment estimates an index indicating the atmosphere of a place based on information about the state of each participant and information about the interaction between the participants, and outputs a correction value for the thermal sensation in the space based on the index indicating the atmosphere of the place. For example, the estimation device 10 transmits a control signal including the thermal index correction value to the space control device 20, and the space control device 20 controls the environment of the space in accordance with the control signal. Therefore, according to this embodiment, it is possible to output information for controlling the space in which a place is formed.

[0110] The estimation device 10 in this embodiment estimates an index indicating the atmosphere of a scene based on information about the state of each participant and information about the interaction between the participants, and controls the space in which the scene is formed based on the index indicating the atmosphere of the scene. For example, the estimation device 10 transmits a control signal for controlling the space to the space control device 20. Therefore, according to this embodiment, it is possible to output information for controlling the space in which the scene is formed.

[0111] The estimation device 10 in this embodiment estimates an index indicating the atmosphere of a place based on information on the state of each participant and information on the interaction between the participants, estimates a correction value for a thermal sensation in the space based on the index indicating the atmosphere of the place, and controls the space in which the place is formed based on the correction value for the thermal sensation. For example, the estimation device 10 transmits a control signal for controlling the space to the space control device 20. Therefore, according to this embodiment, it is possible to output information for controlling the space in which the place is formed.

[0112] The estimation device 10 in this embodiment may display an index indicating the atmosphere of the place or a correction value for thermal sensation in the space where the place is formed. Participants can take actions according to the atmosphere or thermal sensation of the place by referring to the index indicating the atmosphere of the place or the correction value for thermal sensation. For example, participants can change the temperature setting of the air conditioning, eat or drink, open or close windows, start or stop exercising, etc. Therefore, this embodiment can encourage participants to change their behavior to control the space.

[0113] [supplement] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a central processing unit (CPU) or a graphics processing unit (GPU) implemented by an electronic circuit, as well as devices such as an application specific integrated circuit (ASIC), a digital signal processor (DSP), a field programmable gate array (FPGA), and conventional circuit modules designed to execute each of the above-described functions.

[0114] Although the embodiments have been described above, it will be understood that various changes in form and details can be made without departing from the spirit and scope of the claims. [Explanation of symbols]

[0115] 1 participant 2 space 3 sensors 10 Estimation device 11 Atmosphere estimation model 12 Thermal index estimation model 20 Space Control Device 1000 Spatial Control System

Claims

1. An information processing device having a control unit that outputs information about an atmosphere of a place formed by a plurality of participants, The control unit inputting information about the states of the individual participants, information about interactions between the participants, and information about attributes of the participants into a trained atmosphere estimation model to estimate an index indicating the atmosphere of the place; Controlling the space in which the scene is formed based on an index indicating the atmosphere of the scene; the atmosphere estimation model receives information about the state, information about the interaction, and information about attributes of the participants as input, and outputs an estimate of an index indicating the atmosphere of the event using an evaluation value that evaluates the atmosphere of the event as correct answer data; The information about the attributes of the participants includes at least one of information indicating a scene of the conference or a positional relationship of the participants. Information processing device.

2. An information processing device having a control unit that outputs information about an atmosphere of a place formed by a plurality of participants, The control unit inputting information about the states of the individual participants and information about interactions between the participants into a trained atmosphere estimation model to estimate an index indicating the atmosphere of the place; The index indicating the atmosphere of the place and information about the attributes of the participants are input into a trained thermal index estimation model to estimate a correction value for the thermal sensation in the space where the place is formed; controlling the space based on the correction value related to the thermal sensation; the atmosphere estimation model receives information about the state and information about the interaction as input, and outputs an estimate of an index indicating the atmosphere of the place using an evaluation value that evaluates the atmosphere of the place as correct answer data; the thermal index estimation model receives as input the index indicating the atmosphere of the venue and information on the attributes of the participants, and outputs an estimate of the correction value related to the thermal sensation; The information about the attributes of the participants includes at least one of information indicating a scene of the conference or a positional relationship of the participants. Information processing device.

3. the information about the status is biometric information of the participant; 3. The information processing device according to claim 1 or 2.

4. the biometric information is biometric information related to the emotion of the participant; The information processing device according to claim 3 .

5. the information about the interaction is information about communication between the participants; 3. The information processing device according to claim 1 or 2.

6. The information about the communication is information about the behavior of communicating between the participants. The information processing device according to claim 5 .

7. The evaluation value of the atmosphere of the place is a subjective report or an objective evaluation of the atmosphere of the place.

3. The information processing device according to claim 1 or 2.

8. The subjective report of the atmosphere of the place is an index regarding the atmosphere of the place experimentally obtained in advance using a questionnaire method. The information processing device according to claim 7 .

9. The control unit Obtaining feedback information from the participants regarding the atmosphere of the event; updating the atmosphere estimation model based on the feedback information; 3. The information processing device according to claim 1 or 2.

10. The feedback information includes an evaluation value that evaluates the atmosphere of the place. The information processing device according to claim 9 .

11. The control unit Obtaining feedback information regarding the thermal sensation of the space from the participants; updating the thermal index estimation model based on the feedback information; The information processing device according to claim 2 .

12. The feedback information includes an evaluation value that evaluates the thermal sensation of the space. The information processing device according to claim 11.

13. A control unit included in an information processing device that outputs information about an atmosphere of a place formed by a plurality of participants, inputting information about the states of the individual participants, information about interactions between the participants, and information about attributes of the participants into a trained atmosphere estimation model to estimate an index indicating the atmosphere of the place; Controlling the space in which the scene is formed based on an index indicating the atmosphere of the scene; the atmosphere estimation model receives information about the state, information about the interaction, and information about attributes of the participants as input, and outputs an estimate of an index indicating the atmosphere of the event using an evaluation value that evaluates the atmosphere of the event as correct answer data; The information about the attributes of the participants includes at least one of information indicating a scene of the conference or a positional relationship of the participants. Information processing methods.

14. A control unit included in an information processing device that outputs information about an atmosphere of a place formed by a plurality of participants, inputting information about the states of the individual participants and information about interactions between the participants into a trained atmosphere estimation model to estimate an index indicating the atmosphere of the place; The index indicating the atmosphere of the place and information about the attributes of the participants are input into a trained thermal index estimation model to estimate a correction value for the thermal sensation in the space where the place is formed; controlling the space based on the correction value related to the thermal sensation; the atmosphere estimation model receives information about the state and information about the interaction as input, and outputs an estimate of an index indicating the atmosphere of the place using an evaluation value that evaluates the atmosphere of the place as correct answer data; the thermal index estimation model receives as input the index indicating the atmosphere of the venue and information on the attributes of the participants, and outputs an estimate of the correction value related to the thermal sensation; The information about the attributes of the participants includes at least one of information indicating a scene of the conference or a positional relationship of the participants. Information processing methods.

15. A control unit included in an information processing device that outputs information about the atmosphere of a place created by a plurality of participants, inputting information about the states of the individual participants, information about the interactions between the participants, and information about the attributes of the participants into a trained atmosphere estimation model to estimate an index indicating the atmosphere of the place; controlling a space in which the scene is formed based on an index indicating the atmosphere of the scene; the atmosphere estimation model receives information about the state, information about the interaction, and information about attributes of the participants as input, and outputs an estimate of an index indicating the atmosphere of the event using an evaluation value that evaluates the atmosphere of the event as correct answer data; The information about the attributes of the participants includes at least one of information indicating a scene of the conference or a positional relationship of the participants. program.

16. A control unit included in an information processing device that outputs information about the atmosphere of a place created by a plurality of participants, inputting information about the state of each participant and information about the interaction between the participants into a trained atmosphere estimation model to estimate an index indicating the atmosphere of the place; The index indicating the atmosphere of the place and information about the attributes of the participants are input into a trained thermal index estimation model to estimate a correction value for the thermal sensation in the space where the place is formed; controlling the space based on the correction value related to the thermal sensation; the atmosphere estimation model receives information about the state and information about the interaction as input, and outputs an estimate of an index indicating the atmosphere of the place using an evaluation value that evaluates the atmosphere of the place as correct answer data; the thermal index estimation model receives as input the index indicating the atmosphere of the venue and information on the attributes of the participants, and outputs an estimate of the correction value related to the thermal sensation; The information about the attributes of the participants includes at least one of information indicating a scene of the conference or a positional relationship of the participants. program.

Images