Information processing method and information processing device

Abstract

An information processing device 16 acquires user behavior data outputted from sensors 12. The information processing device 16 uses the acquired user behavior data as a basis for determining a control parameter for changing the modality of target equipment 14. The information processing device 16 uses the determined control parameter to change the modality of the target equipment 14.

Description

Information Processing Method and Information Processing Apparatus

[0001] The present disclosure relates to data processing technology, and particularly to an information processing method and an information processing apparatus.

[0002] In leading a smooth social life, communication between people is important. Conversation is a typical form of communication between people. Patent Document 1 below describes a conversation promotion apparatus. The conversation promotion apparatus of Patent Document 1 selects a first subject and a second subject whose interests are similar, generates a topic along the interests of the first subject and the second subject that is related to the content of the conversation between the first subject and the second subject, and presents the topic to the first subject and the second subject.

[0003] Japanese Patent Application Laid-Open No. 2024-17074

[0004] On the other hand, in communities composed of multiple users such as families and offices, users with different hobbies and preferences often gather. Even if one wants to communicate with a third party other than oneself, there are cases where it is difficult to find an opportunity to start a conversation. Therefore, the inventor of the present invention came up with a technology to more effectively promote communication between people. One object of the present disclosure is to provide a technology that more effectively promotes communication between people by providing information that can serve as an opportunity for conversation between users.

[0005] In order to solve the above problems, an information processing method according to an aspect of the present disclosure includes: a computer acquiring user behavior data, determining a control parameter for changing a modality in a target device based on the user behavior data, and using the control parameter to change the modality in the target device.

[0006] Another aspect of the present disclosure is an information processing apparatus. The apparatus includes an acquisition unit that acquires user behavior data, a determination unit that determines a control parameter for changing a modality in a target device based on the user behavior data, and a device control unit that uses the control parameter to change the modality in the target device.

[0007] Another aspect of this disclosure is also an information processing device. This device comprises an acquisition unit that acquires user behavior data, a detection unit that detects changes in the user based on the user behavior data, a determination unit that determines control parameters for changing the modal in a target device based on the changes in the user, and a device control unit that changes the modal in the target device using the control parameters.

[0008] Another aspect of this disclosure is also an information processing device. This device comprises: an acquisition unit that acquires user behavior data generated in conjunction with the user's movements; a determination unit that determines control parameters for changing the modal in a target device installed in a space where a user other than the user is present, based on the user behavior data; and a device control unit that changes the modal in the target device using the control parameters.

[0009] Another aspect of this disclosure is also an information processing device. This device comprises: an acquisition unit that acquires user behavior data; an estimation unit that estimates events in which the user is involved and the user's feelings toward those events based on the user's behavior data; a determination unit that determines messages from other users toward the events in which the user is involved and the user's feelings toward those events; and a device control unit that notifies the user of messages from other users by changing the modal on the target device.

[0010] Another aspect of this disclosure is also an information processing device. This device includes: an acquisition unit that acquires communication data between a user and another user; a detection unit that detects, based on the communication data, whether the communication between the user and another user is in an active state or in a subdued state; a storage unit that stores at least a portion of the communication data in a predetermined storage area when it is detected that the communication between the user and another user is in an active state or in a subdued state; and a device control unit that changes the modal in a target device using at least a portion of the communication data stored in the storage area after a predetermined time has elapsed since the communication data was acquired.

[0011] Furthermore, any combination of the above components, or any conversion of the expressions of this disclosure between systems, computer programs, recording media containing computer programs, etc., are also valid forms of this disclosure.

[0012] The technology disclosed herein can effectively facilitate communication between people.

[0013] This is a diagram showing the configuration of the information processing system of the first embodiment. This is a diagram showing an example of the target device of the first embodiment. This is a block diagram showing the functional blocks of the information processing device of the first embodiment. This is a flowchart showing the operation of the information processing device of the first embodiment. This is a diagram showing an example of behavioral information of the first embodiment. This is a diagram showing an example of detecting whether or not there is a change in the target user. This is a diagram showing an example of determining control parameters. This is a diagram showing an example of control parameters spanning multiple layers. This is a diagram showing an example of the target device of the third embodiment. This is a block diagram showing the functional blocks of the information processing device of the third embodiment. This is a flowchart showing the operation of the information processing device of the third embodiment. This is a block diagram showing the functional blocks of the information processing device of the fourth embodiment. This is a flowchart showing the operation of the information processing device of the fourth embodiment. This is a diagram showing an example of behavioral information of the fourth embodiment. This is a diagram showing an example of estimating events and emotions related to the first user. This is a block diagram showing the functional blocks of the information processing device of the fifth embodiment. This is a flowchart showing the operation of the information processing device of the fifth embodiment. This is a diagram showing an example of communication information. This is a diagram showing an example of determining the level of excitement in communication.

[0014] The subject of the apparatus or method in this disclosure comprises a computer. The functions of the subject of the apparatus or method in this disclosure are realized by the computer executing a program. The computer comprises a processor as its main hardware component, which operates according to the program. The processor is of any type as long as it can realize its functions by executing the program. The processor consists of one or more electronic circuits, including an Integrated Circuit (IC) or a Large Scale Integration (LSI). Multiple electronic circuits may be integrated on one chip or provided on multiple chips. Multiple chips may be aggregated in one device or provided on multiple devices. The program may be recorded on a non-temporary recording medium such as a Read Only Memory (ROM), optical disc, or hard disk drive that is readable by the computer, or on a temporary storage medium such as a Random Access Memory (RAM) that is readable by the computer. The program may be pre-stored on the recording medium, or it may be supplied to the recording medium or storage medium via a wide-area communication network, including the Internet.

[0015] This disclosure proposes a technology that effectively promotes communication between a predetermined user and other users by changing the modal in a controlled device (hereinafter also referred to as the "target device") based on the user's behavioral data. The information processing device in each of the following embodiments (1) acquires the predetermined user's behavioral data, (2) determines control parameters for changing the modal in the target device based on that behavioral data, and (3) changes the modal in the target device using those control parameters. This makes it possible to promote communication between a predetermined user and other users based on the user's behavioral data.

[0016] The user behavior data in this disclosure may include at least one of the following items a through h: a) Data indicating the user's actions. This data includes, for example, how the user takes off their shoes at the entrance, various levels of the user's activity, and the number of steps taken by a pedometer. b) The user's vital data. This data includes, for example, heart rate and blood pressure values. c) Data indicating the content of the user's speech. d) Information exchanged between the user and other users. This information includes, for example, the content of telephone conversations and messages exchanged through SNS (Social Networking Service).

[0017] e) User's calendar data. This data includes, for example, the schedule of events the user is participating in or involved with. f) User attribute data. g) User's background noises. h) Images of the user together with other users. This includes, for example, still images (photographs, etc.) and videos. The data in items c through h above are also collectively referred to as "communication data".

[0018] In this disclosure, a modal is a user interface that stimulates at least one of the five human senses (sight, hearing, smell, taste, and touch). For example, a modal includes displaying images or text information on a display, and the content of such display. Also, for example, a modal includes causing a light-emitting element (in other words, a light source) to emit light in a predetermined manner, and the manner of such emission. Also, for example, a modal includes outputting sound from a speaker, and the content of the outputted sound.

[0019] The devices covered by this disclosure are devices whose modalities are controlled, and include a variety of electrical devices. For example, the devices covered may include electronic devices, information devices, home appliances, residential equipment, etc. Modal changes in the devices covered include changes in the appearance or function of the devices covered. For example, modal changes in the devices covered include the devices outputting sound, music, light, vibration, images (including still images and videos), text, etc.

[0020] <First Embodiment> The outline of the first embodiment will be described. The information processing system of the first embodiment detects changes in the user based on the user's behavioral data. Changes in the user refer to time-series changes related to predetermined things or the user's own biological characteristics. For example, changes in the user include changes in the user's voice or height, increases or decreases in test scores given at school, etc., and the situation or degree to which the user has become able to eat foods that they previously disliked, i.e., the degree to which they have overcome their dislikes. Based on the changes in the user, the information processing system of the first embodiment generates control parameters for the target device and uses the generated control parameters to change the modal of the target device.

[0021] Details of the first embodiment will now be described. Figure 1 shows the configuration of the information processing system 10 of the first embodiment. The information processing system 10 includes a plurality of sensors 12 (including sensors 12a, 12b, and 12c in Figure 1), a target device 14, and an information processing device 16. Each device constituting the information processing system 10 has a communication function and is connected to each other via a communication network 18. The communication network 18 may include a LAN, WAN, or the Internet.

[0022] Multiple sensors 12 detect and acquire behavioral data of at least one user (hereinafter also referred to as the "target user") among multiple users utilizing the information processing system 10, and transmit the target user's behavioral data to the information processing device 16. Multiple users are a group of users targeted by the information processing system 10 to facilitate communication. In the first embodiment, multiple users include a father, mother, and child who constitute one family. The target user is the user whose changes are detected. In the first embodiment, the target person is a child. Note that multiple users (for example, a mother and child, a father and child, or a father, mother, and child) may be set as target people.

[0023] The multiple sensors 12 may be installed in different spaces where the target user may be present. Furthermore, the multiple sensors 12 may be known sensors capable of acquiring different types of behavioral data. For example, the multiple sensors 12 may include voice recognition sensors, AI speakers, illuminance sensors, image sensors (cameras), touch sensors, and the like.

[0024] The target device 14 is an electrical device installed in a predetermined space. The information processing system 10 may include multiple target devices 14.

[0025] Figure 2 shows an example of the target device 14 of the first embodiment. The target device 14 of the first embodiment is a lighting fixture installed in the living rooms of multiple users' homes. The target device 14 is configured to change the color of light emitted according to control parameters. The target device 14 is also configured to change the distance from the ceiling (in other words, the height from the floor) according to control parameters. Hereinafter, increasing the distance from the ceiling (in other words, decreasing the height from the floor) is also referred to as increasing the size of the target device 14. Conversely, decreasing the distance from the ceiling (in other words, increasing the height from the floor) is also referred to as decreasing the size of the target device 14.

[0026] Returning to Figure 1, the information processing device 16 performs information processing to change the modal of the target device 14. The information processing device 16 may be a PC, tablet terminal, or smartphone operated by at least one user. Alternatively, the information processing device 16 may be a residential IoT (Internet of Things) infrastructure device installed in the homes of multiple users. Furthermore, the information processing device 16 may be a server device installed in the cloud.

[0027] Figure 3 is a block diagram showing the functional blocks of the information processing device 16 according to the first embodiment. Various functions shown in Figure 3 are realized by the computer mounted on the information processing device 16 executing a program. The computer includes, as hardware, a memory for loading programs, one or more processors (CPUs, etc.) for executing the loaded programs, auxiliary storage devices, and other LSIs. The processor is composed of multiple electronic circuits, including semiconductor integrated circuits and LSIs, and these multiple electronic circuits may be mounted on a single chip or on multiple chips. The functional blocks shown in Figure 3 are realized through the cooperation of hardware and software, and therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various ways using hardware alone, software alone, or a combination thereof.

[0028] The information processing device 16 comprises a processing unit 30, a storage unit 32, and a communication unit 34. The processing unit 30 performs various information processing. The storage unit 32 stores various data that are referenced or updated by the processing unit 30. The communication unit 34 communicates with external devices according to a predetermined communication protocol. The processing unit 30 sends and receives data with a plurality of sensors 12 and target devices 14 via the communication unit 34.

[0029] The storage unit 32 includes a sensor information storage unit 40 and an activity information storage unit 42. The sensor information storage unit 40 stores information (sensor information) about each of the multiple sensors 12. The sensor information includes the identification information and installation location of the sensor 12. The activity information storage unit 42 stores information (activity information) about the target user's activities. Details of the activity information will be described later.

[0030] The processing unit 30 includes an action data acquisition unit 50, a change detection unit 52, a control parameter determination unit 54, and an equipment control unit 56. The functions of the multiple functional blocks of the processing unit 30 may be implemented in a computer program, and this computer program may be installed in the storage of the information processing device 16. The processor mounted on the information processing device 16 may perform the functions of these multiple functional blocks by reading this computer program into main memory and executing it.

[0031] The behavior data acquisition unit 50 acquires behavior data of the target user transmitted from each of the multiple sensors 12. The change detection unit 52 detects changes in the target user based on the behavior data of the target user acquired by the behavior data acquisition unit 50.

[0032] The control parameter determination unit 54 determines control parameters for changing the modal in the target device 14 based on the changes in the target user detected by the change detection unit 52. In other words, the control parameter determination unit 54 determines control parameters for changing the modal in the target device 14 in a manner corresponding to the changes in the target user. The device control unit 56, as an output unit, changes the modal in the target device 14 using the control parameters determined by the control parameter determination unit 54.

[0033] Figure 4 is a flowchart showing the operation of the information processing device 16 in the first embodiment. The information processing device 16 repeatedly executes the process shown in Figure 4. The operation of the information processing system 10 in the first embodiment will be described below with reference to Figure 4.

[0034] Here, sensor 12a is a voice recognition sensor installed in the living room of the target user's home. Sensor 12b is an AI speaker installed in the target user's room (child's room at home). Sensor 12c is a voice recognition sensor installed in a facility (school, etc.) that the target user attends. Each of the multiple sensors 12 detects the target user's behavioral data and transmits it to the information processing device 16. The target user's behavioral data here includes voice data related to the target user's speech, or voice data related to conversations between the target user and other users.

[0035] When the behavior data acquisition unit 50 of the information processing device 16 receives behavior data of the target user transmitted from each of the multiple sensors 12 (Y in S10), it stores the target user's behavior information, including the target user's behavior data, in the behavior information storage unit 42 (S11). If no behavior data of the target user is received from the sensors 12 (N in S10), the process in S11 is skipped.

[0036] Figure 5 shows an example of behavioral information in the first embodiment. The behavioral information includes the type of behavioral data, the acquisition tool, the acquisition location, the acquisition date and time, and the behavioral data details. The type of behavioral data, the acquisition tool, and the acquisition date and time are provided by the sensor 12 along with the behavioral data. If the behavioral data is speech data of a conversation, the behavioral data acquisition unit 50 may estimate the speaker by known speech analysis processing and estimate the type of behavioral data (for example, who is talking to whom) based on the estimated speaker. The behavioral data acquisition unit 50 sets the item "acquisition location" to the installation location of the sensor 12 that is the source of the behavioral data, which is stored in the sensor information storage unit 40. The behavioral data acquisition unit 50 sets the item "behavioral data details" to the behavioral data transmitted from the sensor 12.

[0037] In the first embodiment, the target user's behavioral data includes data relating to communication between the target user and other users (communication data). The communication data includes conversation data between the target user and other users. The conversation data includes data indicating the content of what the target user said to other users (e.g., voice data or text data). The behavioral information in Figure 5 records multiple statements made by the target user about the food "bitter melon" over multiple dates and times. The location where the behavioral information in Figure 5 was acquired can be said to be the location where communication between the target user and other users took place.

[0038] An example of a method for acquiring communication data is shown below. The behavioral data acquisition unit 50 may acquire data indicating the content of what the target user directly spoke to the information processing device 16. Alternatively, the behavioral data acquisition unit 50 may use known biometric authentication technology or the like to extract the target user's speech data from data containing the speech content of multiple users. Furthermore, the behavioral data acquisition unit 50 may acquire the target user's speech data when the target user interacts with a generated AI or robot.

[0039] Returning to Figure 4, the change detection unit 52 of the information processing device 16 detects a change in the target user based on the target user's behavior data acquired in S10 (S12). In the first embodiment, the change detection unit 52 detects a change in the target user based on characteristic words included in the behavior data. Specifically, the change detection unit 52 detects a change in the target user based on the changes in characteristic words included in each of the multiple behavior data acquired at multiple points in time. Characteristic words can also be described as words or phrases that indicate characteristics related to the user's behavior (including communication between the user and other users).

[0040] Figure 6 shows an example of detecting whether or not there is a change in the target user. As shown in Figure 6, the change detection unit 52 performs morphological analysis to segment the behavioral data into multiple phrases (which may be clauses, words, or morphemes), and extracts a first feature word from among the segmented phrases. The first feature word is a keyword (including phrases, the same applies hereinafter) for task identification. The storage unit 32 may maintain a database (table, etc., the same applies hereinafter) showing predetermined candidates for the first feature word, and the change detection unit 52 may extract a phrase that matches the candidate from among multiple phrases in the behavioral data as the first feature word.

[0041] The change detection unit 52 further extracts second feature words related to the same or similar first feature word (in Figure 6, "bitter melon") from among multiple words segmented by morphological analysis. The second feature words are keywords for change determination and include, for example, words indicating whether an action is possible or not, or words indicating emotions. The memory unit 32 may maintain a database showing predetermined candidate second feature words for each first feature word. The change detection unit 52 may also extract words that match the candidates as second feature words from among multiple words in the behavior data. In Figure 6, words indicating whether or not bitter melon can be eaten, words indicating whether or not it is liked, or words indicating whether or not it is delicious are extracted as second feature words.

[0042] The change detection unit 52 detects changes in the target user by comparing multiple second feature words related to the same or similar first feature word in a time series of the date and time the behavioral data was acquired, and by tracking the changes in actions and emotions indicated by the multiple second feature words.

[0043] In the first embodiment, each of the plurality of second characteristic words is associated in advance with either a positive user state ("can" in FIG. 6) or a negative user state ("cannot" in FIG. 6). Basically, positive words and phrases (such as "can", "like", "tasty", etc.) are associated with positive user states, and negative words and phrases (such as "cannot", "dislike", "tasteless", etc.) are associated with negative user states.

[0044] In the first embodiment, when the user state indicated by the plurality of second characteristic words in the plurality of behavior data changes from negative to positive or from positive to negative along the time series, the change detection unit 52 detects that the target user has changed. In the example of FIG. 6, the user state regarding the first characteristic word "bitter melon" in the behavior data with data number 4 has changed from negative to positive. Therefore, the change detection unit 52 detects that the target user has changed in the behavior data with data number 4.

[0045] Another method for detecting a change in the user applicable to S12 in FIG. 4 will be described. Of course, a plurality of methods may be combined.

[0046] (Example 1) The change detection unit 52 may obtain the number of times regarding the first characteristic word from the clauses before and after the positive second characteristic word (such as "can...", etc.) in the behavior data. The first characteristic word here is a word or phrase in which the growth or change of the user is indicated by the number of times, and may be, for example, "handstand", etc. When the number of times increases in the time series of the acquisition date and time across a plurality of behavior data including the same first characteristic word, the change detection unit 52 may assign a growth flag to the target user. When the number of growth flags assigned to the target user is equal to or more than a predetermined threshold (for example, 5), the change detection unit 52 may detect a change in the target user.

[0047] (Example 2) The memory unit 32 may hold a first database (such as a table) in which first characteristic words that it is desirable to increase, such as height, are defined. Also, the memory unit 32 may hold a second database in which first characteristic words that it is desirable to decrease, such as the time in a competition (such as the 100-meter dash), are defined. When the value regarding the first characteristic word increases in the time series of the acquisition date and time for a plurality of action data including the same first characteristic word defined in the first database, the change detection unit 52 may assign a growth flag to the target user. Also, when the value regarding the first characteristic word decreases in the time series of the acquisition date and time for a plurality of action data including the same first characteristic word defined in the second database, the change detection unit 52 may assign a growth flag to the target user. When the number of growth flags assigned to the target user is equal to or greater than a predetermined threshold value (for example, three), the change detection unit 52 may detect a change in the target user.

[0048] Also, the memory unit 32 may hold a third database in which first characteristic words that it is desirable to maintain, such as weight (in middle age), are defined. When the value regarding the first characteristic word changes beyond a predetermined allowable range in the time series of the acquisition date and time for a plurality of action data including the same first characteristic word defined in the third database, the change detection unit 52 may assign an (undesirable) change flag to the target user. When the number of change flags assigned to the target user is equal to or greater than a predetermined threshold value (for example, three), the change detection unit 52 may detect an (undesirable) change in the target user.

[0049] (Example 3) When the change detection unit 52 detects a first verb phrase such as "not yet XX" or "unable to XX" in the first action data and detects a second verb phrase such as "XX-ed" or "able to XX" in the second action data acquired after the first action data, the change detection unit 52 may detect a change in the user. For example, the first verb phrase may be "My morning glory has not yet bloomed," and the second verb phrase may be "My morning glory has finally bloomed." The first action data and the second action data here may include the same first characteristic word.

[0050] (Example 4) The change detection unit 52 may increment a growth flag assigned to the target user each time it detects a specific keyword such as "I can now do ○○" in the behavioral data. Alternatively, the change detection unit 52 may detect a change in the user even if the specific keyword has only been detected once.

[0051] (Example 5) If the behavioral data is an image of the target user captured by the sensor 12 acting as a camera, the change detection unit 52 may detect a change in the target user if it detects through image analysis that the target user has performed a predetermined action (including gestures). The predetermined action may be an action that expresses joy, for example, a cheer or a fist pump. In order to reduce the possibility of false detection, the change in the user may be further detected by taking into account the volume of the space in which the target user is present (whether or not the volume is above a predetermined threshold) and the content of the target user's speech (whether or not it matches keywords during excitement) when the predetermined action of the target user is detected.

[0052] Returning to Figure 4, if the change detection unit 52 detects a change in the target user (Y in S12), the control parameter determination unit 54 of the information processing device 16 determines the control parameters for the target device 14 to change the modal in the target device 14 based on the change in the target user (S13).

[0053] Figure 7 shows an example of determining control parameters. In the first embodiment, the modal of the target device 14 (lighting fixture) to be controlled includes the color of light emitted by the target device 14 (i.e., the color of the lighting) and the size of the target device 14. Based on changes in the target user, the control parameter determination unit 54 determines one or both of the control parameters that control the color of light emitted by the target device 14 and the control parameters that control the size of the target device 14.

[0054] Specifically, the control parameter determination unit 54 determines the color of the target device 14 based on the first characteristic word and determines the control parameters for emitting light in the determined color. For example, the memory unit 32 may store a database in which tasks, objects, events, etc. indicated by the first characteristic word (candidates) are pre-associated with colors. For example, the vegetable "bitter melon" may be associated with the same green color as its appearance. The control parameter determination unit 54 may determine the color of the target device 14 to be a color pre-associated with the first characteristic word extracted from the behavioral data. As a modified example, the control parameter determination unit 54 may ask the generating AI for a color that can be recalled from the tasks, objects, events, etc. indicated by the first characteristic word, and determine the color indicated by the generating AI's answer to be the color of the target device 14.

[0055] In the example shown in Figure 7, a change in the user is detected when the behavior data for data number 4 is received. The control parameter determination unit 54 determines that the color of the target device 14 is green until the behavior data for data number 3 is received, when no change in the user has been detected, and determines the control parameters to make the target device 14 emit green light. When the behavior data for data number 4, in which a change in the user has been detected, is received, the control parameter determination unit 54 decides to change the color of the target device 14 to dark green and determines the control parameters to make the target device 14 emit dark green light.

[0056] Furthermore, in the example shown in Figure 7, the control parameter determination unit 54 determines the size of the target device 14 to be less than or equal to the initial setting value until it receives the behavior data for data number 3, in which no change in the user has been detected, and determines the control parameters to make the size of the target device 14 less than or equal to the initial setting value. When it receives the behavior data for data number 4, in which a change in the user has been detected, the control parameter determination unit 54 determines a value greater than the initial setting value for the size of the target device 14, and determines the control parameters to make the size of the target device 14 larger than the initial setting value.

[0057] Furthermore, in the first embodiment, the control parameter determination unit 54 counts the number of times identical behavioral data appear consecutively and adjusts the size of the target device 14 according to that number. Identical behavioral data may be behavioral data in which the first characteristic word is the same and the category of the second characteristic word (for example, positive or negative type) is the same. The control parameter determination unit 54 may increase the amount of change in the size of the target device 14 as the number of consecutive occurrences of identical behavioral data increases. Also, the control parameter determination unit 54 may make the target device 14 larger as the number of consecutive behavioral data containing positive second characteristic words increases, and may make the target device 14 smaller as the number of consecutive behavioral data containing negative second characteristic words increases.

[0058] In the example shown in Figure 7, since behavioral data containing a negative second characteristic word appeared three times in a row, the control parameter determination unit 54 determines the control parameters to gradually decrease the size of the target device 14 from the initial setting value. Subsequently, since behavioral data containing a positive second characteristic word appeared two times in a row, the control parameter determination unit 54 determines the control parameters to gradually increase the size of the target device 14 from the initial setting value.

[0059] As a variation, the memory unit 32 may store control parameters for adjusting the color and size of the target device 14 across multiple layers, for each of the first and second characteristic words. Figure 8 shows an example of control parameters across multiple layers. In Figure 8, the second characteristic word of the first layer is a phrase indicating whether or not it is operable. The second characteristic word of the second layer is a phrase indicating emotions, tastes, etc., that do not involve spontaneous actions. The second characteristic word of the third layer is a phrase indicating spontaneous actions or intentions. In the example in Figure 8, the complexity is defined to increase in the order of the first layer, second layer, and third layer.

[0060] As shown in Figure 8, in the case of a positive second characteristic word, the control parameter C of the first layer, the control parameter B of the second layer, and the control parameter A of the third layer may be set as price growth, in order to darken the color of the target device 14, or to increase the size of the target device 14. Conversely, in the case of a negative second characteristic word, the control parameter c of the first layer, the control parameter b of the second layer, and the control parameter a of the third layer may be set as negative growth, in order to lighten the color of the target device 14, or to decrease the size of the target device 14.

[0061] The control parameter determination unit 54 may identify the category and layer of the second feature word based on the first and second feature words. The control parameter determination unit 54 may determine the control parameters associated with the identified layer as the control parameters to be applied to the target device 14. Alternatively, the control parameter determination unit 54 may determine the control parameters of the target device 14 for applying the color and size associated with the identified layer to the target device 14.

[0062] Furthermore, the control parameter determination unit 54 may increase the luminance or saturation value related to the emission of light from the target device 14 by a predetermined value as the number of consecutive occurrences of behavioral data containing positive second characteristic words increases, or as the layer of positive second characteristic words increases across consecutive behavioral data (i.e., positive growth progresses). Conversely, the control parameter determination unit 54 may decrease the luminance or saturation value related to the emission of light from the target device 14 by a predetermined value as the number of consecutive occurrences of behavioral data containing negative second characteristic words increases, or as the layer of negative second characteristic words increases across consecutive behavioral data (i.e., negative growth progresses).

[0063] Other methods for determining control parameters applicable to S13 in Figure 4 are described below. Of course, multiple methods can be combined.

[0064] (Example 1) The control parameter determination unit 54 may generate image data relating to the behavior and communication of the target user as indicated by the behavioral data. In the example in Figure 6, it may generate image data indicating the first characteristic word "bitter melon". Alternatively, the control parameter determination unit 54 may have the generation AI generate the image data and acquire the image data generated by the generation AI. The control parameter determination unit 54 may acquire the mode of the RGB values ​​(pixel values) in the generated or acquired image data and determine the control parameters to set the color indicated by the mode of the RGB values ​​to the target device 14.

[0065] If multiple image data are generated or acquired, the control parameter determination unit 54 may determine the control parameters so that the target device 14 displays the color indicated by the mode RGB value in the multiple image data in sequence. If multiple image data are generated or acquired and the target device 14 has multiple light sources, the control parameter determination unit 54 may determine the control parameters so that the color indicated by the mode RGB value in the multiple image data (multiple colors) is output in parallel from those multiple light sources.

[0066] The target device 14 may be equipped with a low-resolution display (for example, a curved LED display). The low-resolution display may have a resolution of, for example, 64 x 64 pixels or 512 x 256 pixels. The control parameter determination unit 54 may generate control parameters for displaying a color on the target device 14 based on the pixel value on a plane in the generated or acquired image data.

[0067] Furthermore, the control parameter determination unit 54 may generate new image data with reduced resolution from the generated or acquired image data, regardless of the resolution of the target device 14's display, and display this new image data on the target device 14's display. Alternatively, the control parameter determination unit 54 may display only a portion of the generated or acquired image data on the target device 14's display, regardless of the resolution of the target device 14's display. This can stimulate the imagination of the user viewing the target device 14 (the user receiving the information) and provide an effective conversation starter.

[0068] (Example 2) The memory unit 32 may maintain a database of words (adjectives, etc.) that describe states, such as "big" and "fast." When the control parameter determination unit 54 detects a word defined in the database from the behavior data, it may dynamically change the display method of the display device on the target device 14 according to the detected word (word that describes a state). For example, if the word "big" is included in the behavior data, the control parameter determination unit 54 may determine the control parameters to gradually increase the area with high brightness in the light-emitting part of the target device 14 (lighting fixture). Also, if the word "fast" is included in the behavior data, the control parameter determination unit 54 may determine the control parameters to gradually increase the speed of change in the light emission of the target device 14 (lighting fixture).

[0069] Returning to Figure 4, the device control unit 56 of the information processing device 16 changes the modal in the target device 14 by controlling the target device 14 using the control parameters determined by the control parameter determination unit 54 (S14). If no change in the target user is detected (N in S12), the processes in S13 and S14 are skipped.

[0070] In the first embodiment, the equipment control unit 56 changes the color of the lighting by transmitting data including control parameters related to color determined by the control parameter determination unit 54 to the target equipment 14 (lighting fixture). The equipment control unit 56 also changes the size of the target equipment 14 (for example, the distance from the ceiling) by transmitting data including control parameters related to size determined by the control parameter determination unit 54 to the target equipment 14.

[0071] Examples of other operations of the device control unit 56 will be described. Of course, multiple operations can be combined.

[0072] (Example 1) The device control unit 56 may change the modal of the target device 14 using the control parameters determined by the control parameter determination unit 54 if the number of growth flags assigned to the target user by the control parameter determination unit 54 is equal to or greater than a predetermined threshold (for example, 5 flags).

[0073] (Example 2) The device control unit 56 may change the modal of the target device 14 if the number of growth flags assigned to the target user is equal to or greater than a predetermined threshold, and a user other than the target user is near the target device 14. Also, if there are multiple target devices 14 (notification target devices), the device control unit 56 may change the modal of the target device 14 if a user other than the target user is nearby.

[0074] (Example 3) If the target device 14 has multiple types of modals, the device control unit 56 may switch the type of modal to be changed over time, or gradually increase it over time. For example, the device control unit 56 may first change a modal that stimulates the visual sense (e.g., a light emission pattern), and if there is no response from the notified user (e.g., no conversation), it may further change a modal that stimulates the auditory sense (e.g., voice output).

[0075] According to the information processing system 10 of the first embodiment, the modal of the target device 14 is changed in response to changes in the target user (for example, daily growth, changes in emotions or health status, etc.). For example, the lighting status of the lighting fixture, which is the target device 14, is changed, such as the color temperature, intensity, or blinking of the light. As a result, through the modal changes of the target device 14, the time-series changes of the target user can be notified to other users as information that can serve as a trigger for conversations between users, and consequently, communication between the target user and other users can be promoted.

[0076] A modified example relating to the first embodiment will now be described. <Modification 1-1> As previously described, the behavior information stored in the behavior information storage unit 42 includes the acquisition location. The acquisition location includes the location where communication between the target user and other users took place. The control parameter determination unit 54 of the information processing device 16 may, in response to a change in the target user, select a target device 14 from among a plurality of target devices 14 that is installed at the location where communication between the target user and other users took place, and determine control parameters for changing the modal in the selected target device 14.

[0077] Furthermore, the device control unit 56 of the information processing device 16 may use the control parameters determined by the control parameter determination unit 54 to select a target device 14 from among a plurality of target devices 14 that is installed in a location where communication between the target user and other users took place, and change the modal in the selected target device 14.

[0078] This modified version changes the modal of the target device 14, which is linked to the location where communication took place between the target user (e.g., a child) and another user (e.g., the mother). This makes it easier for the notifying user (e.g., the father or mother) to feel personally aware of the changes in the target user (e.g., the child).

[0079] <Modification 1-2> The control parameter determination unit 54 of the information processing device 16 may determine the target device 14 to change the modal based on the relationship between the target user and other users who communicate with the target user.

[0080] For example, let's assume the relationship between the target user (here, a child) and the other users (here, a father and mother) is that of a family. Let's also assume there are target devices 14 installed in the child's room, target devices 14 installed in the father's room, and target devices 14 installed in the mother's room. In this case, the control parameter determination unit 54 may determine a target device 14 different from the target device 14 installed in the child's room as the target device 14 that changes the modal in accordance with the changes in the target user, the child. For example, the control parameter determination unit 54 may determine the target device 14 installed in the father's room and / or the target device 14 installed in the mother's room as the target device 14 that changes the modal. According to this modification, the target device 14 that changes the modal can be appropriately selected from the viewpoint of promoting communication.

[0081] <Modification 1-3> The control parameter determination unit 54 of the information processing device 16 may determine the timing for changing the modal of the target device 14 based on information indicated by the target user's behavioral data and information indicating the status of communication between the target user and other users. For example, even if a change in the target user is detected, the control parameter determination unit 54 may suppress changing the modal of the target device 14 installed at that location if it is detected that the target user and other users are in the same location.

[0082] As a concrete example, suppose a microphone that collects conversation data as the first sensor 12, a camera that detects people in the living room as the second sensor 12, and the target device 14 are installed in the living room of a house. In this case, even if the control parameter determination unit 54 detects a change (growth) in the child based on the child's behavioral data (e.g., conversational audio) as the target user, it does not need to change the modal of the target device 14 if both the child as the target user and the parent as another user are in the living room. This is because changing the modal of the target device 14 may actually cause annoyance to the target user. According to this modification, the modal of the target device 14 can be changed at an appropriate timing.

[0083] <Modification 1-4> If the change in the target user is growth, in addition to notifying other users of the target user's growth, a mechanism may be provided to please the target user. For example, the memory unit 32 may store information indicating the average growth rate (which may also be the growth rate of other users). If the growth of the target user is detected and the growth rate over a predetermined period is higher than the average growth rate (which may also be the growth rate of other users) that has been stored in advance, the control parameter determination unit 54 may determine control parameters for controlling the target device 14 in a special manner different from the normal as a surprise device control. For example, the control parameter determination unit 54 may change the color of light emitted by the lighting fixture, which is the target device 14, to a special color different from the normal color for the surprise device control.

[0084] <Second Embodiment> The second embodiment of this disclosure will be described focusing on the differences from the first embodiment. Components of the second embodiment that are the same as or equivalent to components of the first embodiment will be denoted by the same reference numerals as those of the first embodiment, and explanations that overlap with the description of the first embodiment will be omitted as appropriate.

[0085] The configuration of the information processing system 10 in the second embodiment is the same as the configuration of the information processing system 10 in the first embodiment shown in Figure 1. The target device 14 in the second embodiment is a digital display device, and in the following description, it will be a digital photo frame that displays digital photographs. As a modification, the target device 14 may be an electronic album (digital photo album).

[0086] The functional blocks of the information processing device 16 in the second embodiment are the same as those of the information processing device 16 in the first embodiment shown in Figure 3. The information processing device 16 in the second embodiment changes the display mode of content on the digital display device based on changes in communication between the target user and family members (other users).

[0087] The change detection unit 52 of the information processing device 16 derives the amount of interaction between the target user and their family (for example, the number of times they interacted within a predetermined period) based on multiple behavioral data of the target user. The change detection unit 52 also derives the number of events in which the target user and their family act together (for example, the number of times they ate together, the number of times they went out together, the number of common events, etc.) based on multiple behavioral data of the target user. Based on the amount of interaction between the target user and their family and the number of events in which the target user and their family act together, the change detection unit 52 detects a change in communication between the target user and their family. A change in communication between the target user and their family includes an increasing or decreasing trend in the number or frequency of communication.

[0088] The control parameter determination unit 54 of the information processing device 16 determines control parameters related to content display in the digital photo frame based on changes in communication between the target user and their family. The device control unit 56 of the information processing device 16 may change the display mode of digital photographs in the digital photo frame using the control parameters determined by the control parameter determination unit 54.

[0089] For example, if the amount of communication between the target user, the father, and other family members is decreasing, the control parameter determination unit 54 may determine the control parameters of the digital photo frame so that the father's image in the digital photograph (in this case, a family photograph) is displayed in a different manner from the images of other family members. Displaying in a different manner may mean that the father's image is displayed fainter than the images of other family members. Alternatively, displaying in a different manner may mean that the father's image is displayed with lower brightness, lightness, or saturation than the images of other family members.

[0090] Furthermore, if, after displaying the father's image in the digital photograph in a different manner from the images of other family members, the amount of communication between the father and other family members increases, the control parameter determination unit 54 may determine the control parameters of the digital photo frame to display the father's image in the same manner as the images of other family members (i.e., in the normal manner).

[0091] Furthermore, the control parameter determination unit 54 or the device control unit 56 may acquire photographic data of the location indicated by the target user's conversation data from a photo application and display it on the digital photo frame. Alternatively, the control parameter determination unit 54 or the device control unit 56 may perform an image search for the location indicated by the target user's conversation data, acquire photographic data of that location, and display it on the digital photo frame. Moreover, if the frequency of family communication is decreasing, the control parameter determination unit 54 or the device control unit 56 may change the display pattern of the background of the digital photograph displayed on the digital photo frame so that it gradually fades.

[0092] According to the information processing system 10 of the second embodiment, changes in communication between the target user and other users (for example, changes in the number and frequency of communications) are notified to the target user and other users through the content display mode on a digital display device. This provides a trigger for communication between the target user and other users, and also effectively promotes communication between the target user and other users.

[0093] <Modification 2-1> Modifications relating to the first and second embodiments will be described. The information processing device 16 may store memory information (such as information and photos of family trips) associated with the event logs of multiple users (here, a family of three). The information processing device 16 may use sensor data or positioning information from GPS (Global Positioning System) to detect the gathering state of users in a predetermined space. If a predetermined gathering condition is met, the information processing device 16 may output the memory information stored in association with the event logs of the family of three from the target device 14. The gathering condition may be, for example, that all three family members are present. The target device 14 may be an output device such as a digital photo frame.

[0094] If the predetermined gathering conditions are not met (for example, if only two out of three family members are present), the information processing device 16 may output memory information related to the missing member from a predetermined output device. This modified version can facilitate communication among the present members. Furthermore, this modified version can encourage conversations about the missing member among the present members, fostering relationships between the present members and the missing member.

[0095] <Third Embodiment> The third embodiment of this disclosure will be described focusing on the differences from the first embodiment. Components of the third embodiment that are the same as or equivalent to components of the first embodiment will be denoted by the same reference numerals as those of the first embodiment, and explanations that overlap with the description of the first embodiment will be omitted as appropriate.

[0096] The outline of the third embodiment will now be described. The information processing system of the third embodiment changes the modal (e.g., illumination state, color, vibration, shaking, etc.) of a target device installed in a space where a second user, who is different from the first user, is present, based on the behavioral data of the first user generated as a result of the first user's movements. For example, the first user is a grandmother living alone in the countryside, and the second user is the first user's child or grandchild living in the city.

[0097] The configuration of the information processing system 10 in the third embodiment is the same as the configuration of the information processing system 10 in the first embodiment shown in Figure 1. The sensor 12 is installed in the space where the first user is present and lives. On the other hand, the target device 14 is installed in a space where the first user is not present and where the second user is present and lives. For example, the sensor 12 is installed in the first user's house, while the target device 14 is installed in the second user's house, which is located far from the first user's house. The target device 14 changes its modal in a manner based on control parameters transmitted from the information processing device 16.

[0098] Figure 9 shows an example of the target device 14 of the third embodiment. The target device 14 of the third embodiment is a curtain (a light-up curtain, so to speak) with LED strip lights suspended from the ceiling to the floor as a light source. The target device 14 of the third embodiment makes the LED strip lights emit light in a manner based on control parameters transmitted from the information processing device 16, in other words, it switches the LED strip lights from an off state to an illuminated state. The target device 14 may be an electronic curtain, an electronic noren (Japanese curtain), a liquid crystal display, or an organic EL display.

[0099] As a variation, the target device 14 may be configured to change its physical shape as a modal change. For example, the target device 14 may be a device equipped with a fan or servo motor that causes a curtain (which may include LED strip lights) to sway, and may be a swaying curtain, so to speak.

[0100] Figure 10 is a block diagram showing the functional blocks of the information processing device 16 of the third embodiment. The information processing device 16 of the third embodiment includes an action data acquisition unit 50, an action information generation unit 53, a control parameter determination unit 54, and an equipment control unit 56.

[0101] The behavior data acquisition unit 50 acquires behavior data of the first user transmitted from multiple sensors 12. The behavior data of the first user is generated as a result of the first user's movements. The behavior information generation unit 53 generates behavior information indicating the behavior patterns of the first user based on the behavior data of the first user acquired by the behavior data acquisition unit 50.

[0102] The control parameter determination unit 54 determines control parameters for changing the modal in the target device 14 based on the behavior data of the first user acquired by the behavior data acquisition unit 50 (in the third embodiment, the behavior information of the first user generated based on the behavior data). The device control unit 56 changes the modal in the target device 14 using the control parameters determined by the control parameter determination unit 54.

[0103] Figure 11 is a flowchart showing the operation of the information processing device 16 of the third embodiment. The information processing device 16 repeatedly executes the process shown in Figure 11. The operation of the information processing system 10 of the third embodiment will be described below with reference to Figure 11.

[0104] Here, sensor 12a is a motion sensor installed in the first user's home (for example, in the kitchen). Sensor 12b is a positioning sensor that uses GPS or the like, built into a smartwatch worn by the first user. When sensor 12a detects the presence of the first user, it transmits behavioral data indicating this to the information processing device 16. Sensor 12b repeatedly transmits behavioral data indicating the first user's current location to the information processing device 16. This behavioral data may also be transmitted to the information processing device 16 via communication equipment in the first user's home.

[0105] The behavior data acquisition unit 50 of the information processing device 16 receives behavior data of the first user transmitted from each of the multiple sensors 12 (S20). When the behavior data of the first user is received (Y in S20), the behavior information generation unit 53 of the information processing device 16 generates behavior information of the first user based on the behavior data of the first user (S21). For example, based on multiple behavior data received within a predetermined period, the behavior information generation unit 53 calculates the first user's position, distance traveled, and the speed and acceleration of their movement (which can also be called motion) within the first user's residence, and generates behavior information that includes the values ​​of these multiple items. Note that the behavior data itself may include the first user's position, distance traveled, and the speed and acceleration of their movement (which can also be called motion).

[0106] The control parameter determination unit 54 of the information processing device 16 determines control parameters for changing the modal of the target device 14 based on the action information of the first user generated in S21 (S22). The device control unit 56 of the information processing device 16 transmits a control signal including the control parameters determined in S22 to the target device 14, thereby changing the modal of the target device 14 in the manner indicated by the control parameters (S23). If the action data of the first user is not received (N in S20), the processes in S21-S23 are skipped.

[0107] In the third embodiment, the storage unit 32 stores a predetermined correspondence between a set of values ​​(or ranges) of distance traveled, speed traveled, and acceleration that can be indicated by the behavioral information, and a control parameter value for the target device 14 (also called a "basic parameter value"). The control parameter determination unit 54 determines the basic parameter value corresponding to the set of values ​​of distance traveled, speed traveled, and acceleration indicated by the first user's behavioral information as the control parameter for the target device 14. For example, by setting the set of values ​​of distance traveled, speed traveled, and acceleration indicated by the first user's behavioral information and the basic parameter value to the same value, the movement of the first user can be reproduced in the movement of the target device 14. The control parameter determination unit 54 may also adjust the basic parameter value based on the magnitude relationship between the values ​​of distance traveled, speed traveled, and acceleration indicated by the first user's behavioral information and a predetermined reference value, and determine the adjusted value as the control parameter.

[0108] Furthermore, the control parameter determination unit 54 may determine control parameters for the target device 14 and change the modal of the target device 14 when the behavior information of the first user satisfies predetermined conditions. For example, the control parameter determination unit 54 may determine control parameters when one or more of the items among the distance traveled, speed traveled, and acceleration indicated by the behavior information of the first user satisfies predetermined conditions (for example, when the distance traveled or speed traveled is greater than or equal to a standard value). On the other hand, if these conditions are not met, the control parameter determination unit 54 may suppress the generation of control parameters and suppress the modal change of the target device 14.

[0109] As previously described, the target device 14 of the third embodiment is a light-up curtain including LED strip lights. The control parameters of the third embodiment are data that define the pattern of how the target device 14 lights up. The lighting pattern can also be described as a pattern of on and off, and also includes a pattern of the color of light emitted. The device control unit 56 causes the target device 14 to light up according to the lighting pattern defined by the control parameters.

[0110] As a variation, if the target device 14 is configured to change its physical shape, the control parameter determination unit 54 may determine control parameters for changing the physical shape of the target device 14. For example, the control parameter determination unit 54 may determine control parameters for instructing the drive of a fan or servo motor provided by the target device 14. The device control unit 56 may drive the fan or servo motor provided by the target device 14 by transmitting a control signal including the control parameters to the target device 14, thereby opening and closing or swinging the swaying curtain that is the target device 14.

[0111] The following describes an example of how to notify the first user's actions through modal changes in the target device 14. Of course, multiple methods can be combined.

[0112] (Example 1) The behavior data of the first user acquired by the behavior data acquisition unit 50 of the information processing device 16 may include information about the direction in which the first user is moving. The behavior information generation unit 53 may generate behavior information that includes information about the direction in which the first user is moving. Furthermore, if multiple motion sensors are installed at multiple locations in the first user's residence, the behavior information generation unit 53 may identify changes in the first user's position in the first user's residence based on the detection results of the multiple motion sensors within a predetermined period, and estimate the direction in which the first user has moved based on those changes. The control parameter determination unit 54 may determine control parameters that change the modal of the target device 14 in a manner corresponding to the direction in which the first user is moving. According to this embodiment, the direction in which the first user is moving can be communicated to the second user.

[0113] (Example 2) The behavior information generation unit 53 and the control parameter determination unit 54 may also include a notification function. The control parameter determination unit 54 may determine control parameters for the target device 14 and notify a third party of predetermined information when the speed or acceleration of the first user's movement exceeds a predetermined threshold indicating that the first user is in a normal state. As a modified example, the device control unit 56 may change the modal of the target device 14 using the control parameters determined by the control parameter determination unit 54 and notify a third party of predetermined information when the speed or acceleration of the first user's movement exceeds a predetermined threshold indicating that the first user is in a normal state.

[0114] The above threshold may be determined based on the normal movement speed or acceleration of the first user, which is measured by the sensor 12 when the first user is in a normal state. For example, the above threshold may be a value obtained by increasing the normal movement speed or acceleration of the first user by a predetermined percentage (e.g., 10%). The third party may be an individual or company different from both the first user and the second user. For example, the third party may be a security service company that provides monitoring or emergency response services, an insurance company, a medical service company, or a welfare service company.

[0115] The specified information described above may include information indicating that an abnormal situation has occurred or that the first user may be in danger, and may also include content prompting the person in charge to check on the safety of the first user. The control parameter determination unit 54 or the equipment control unit 56 may transmit the specified information described above to a third party's information device. According to this embodiment, by notifying a third party when it is presumed that the first user's actions are irregular and different from normal actions, it is possible to respond to the danger to the first user and enhance the safety of the first user.

[0116] (Example 3) The control parameter determination unit 54 of the information processing device 16 may detect the periodicity of the first user's movements when they change periodically, based on at least one of the behavior data acquired by the behavior data acquisition unit 50 and the behavior information generated by the behavior information generation unit 53. In other words, the control parameter determination unit 54 may detect the period in which the first user repeats the same movement. The control parameter determination unit 54 may determine control parameters for the target device 14 so that the target device 14 reproduces the detected period of the first user's movements. For example, the control parameter determination unit 54 may determine control parameters to cause the target device 14 to repeatedly turn on and off at the same period as or a corresponding period to the period of the first user's movements, or it may determine control parameters to shake the target device 14 at the above period.

[0117] According to this embodiment, by reflecting the movement cycle of the first user in the modal changes of the target device 14, the movements of the first user in the real world can be transmitted to the second user in a realistic manner.

[0118] (Example 4) The information processing device 16 may further include a weather information acquisition unit (not shown). The weather information acquisition unit acquires weather information for the location of a second user (for example, the address or residence of the second user) that has been registered in advance from a predetermined external device that provides weather information for various locations. The control parameter determination unit 54 of the information processing device 16 may adjust the control parameters for the target device 14 based on the current weather information at the location of the second user acquired by the weather information acquisition unit.

[0119] If the weather at the second user's location is severe, the second user may not easily notice the modal change of the target device 14. Therefore, the control parameter determination unit 54 may detect that the weather at the second user's location is severe if the current weather at the second user's location, as indicated by the weather information, meets predetermined severe weather criteria. The above severe weather criteria may be met if at least one of the following conditions is met: a typhoon is approaching, there is a thunderstorm or heavy rain, or the wind speed is above a predetermined threshold.

[0120] If the control parameter determination unit 54 detects that the weather at the second user's location is severe, it may increase the intensity of the modal change of the target device 14 or widen the range of the modal change of the target device 14. Here, "normal" refers to the case where the current weather at the second user's location does not meet the above severe weather criteria, or in other words, the case where severe weather at the second user's location has not been detected. According to this embodiment, it becomes easier for the second user to recognize the modal change of the target device 14, and it becomes easier for the second user to recognize the movements of the first user.

[0121] In another embodiment, the sensor 12 may include an illuminance sensor installed at a window in the second user's residence. The weather information acquisition unit of the information processing device 16 may acquire the measured value of the illuminance sensor as weather information for the second user's location. The control parameter determination unit 54 of the information processing device 16 may detect that the sunlight is strong (for example, the setting sun is dazzling) in the second user's environment if the measured value of the illuminance sensor (i.e., the illuminance at the window) is higher than a predetermined standard.

[0122] If the control parameter determination unit 54 detects that sunlight is strong in the environment of the second user, it may determine a control parameter to drive the servo motor to shake the target device 14, since changing the color of the target device 14 would not be easily noticed by the second user. Alternatively, if the control parameter determination unit 54 detects that sunlight is strong in the environment of the second user, it may determine a control parameter to change a modal that stimulates the second user's senses other than sight. For example, instead of determining a control parameter to change the color of the target device 14, the control parameter determination unit 54 may determine a control parameter to output a predetermined sound from the speaker of the target device 14.

[0123] (Example 5) The information processing device 16 may further include an activity intensity acquisition unit (not shown). At least some of the multiple sensors 12 may be installed in the second user's residence. The activity intensity acquisition unit of the information processing device 16 may acquire information about the second user's activity intensity transmitted from the sensors 12 installed in the second user's residence. The activity intensity acquisition unit may also calculate and acquire the second user's activity intensity based on the second user's behavior data transmitted from the sensors 12 installed in the second user's residence. The control parameter determination unit 54 of the information processing device 16 may adjust the control parameters for the target device 14 based on the second user's activity intensity acquired by the activity intensity acquisition unit.

[0124] The activity intensity of the second user may include, for example, the speed and acceleration of the second user's movements. The activity intensity acquisition unit may also calculate the speed and acceleration of the second user's movements by analyzing images (videos, etc.) of the second user transmitted from a sensor 12 (e.g., a camera) installed in the second user's home. The control parameter determination unit 54 may determine that the activity intensity of the second user is high if the speed and acceleration of the second user's movements meet predetermined criteria (for example, if they are above a predetermined threshold).

[0125] If the activity level of the second user is high, the second user may have difficulty noticing the modal changes of the target device 14. Therefore, if the control parameter determination unit 54 determines that the activity level of the second user is high, it may increase the intensity of the modal changes of the target device 14 above normal, or it may widen the range of modal changes of the target device 14 above normal. In addition, if the control parameter determination unit 54 determines that the activity level of the second user is high, it may change multiple modals in the target device 14. Here, "normal" refers to the case where the speed and acceleration of the second user's movement do not meet the above criteria, or in other words, the case where it is not determined that the activity level of the second user is high. According to this embodiment, it becomes easier for the second user to recognize the modal changes of the target device 14, and it becomes easier for the second user to recognize the movements of the first user.

[0126] (Example 6) The information processing device 16 may further include an event information acquisition unit (not shown). The event information acquisition unit of the information processing device 16 acquires event information relating to the second user from an event management device that stores event information relating to the second user. The event information acquisition unit may also acquire event information registered by the second user from a calendar application or schedule application installed on the second user's information terminal. The event information relating to the second user may include information indicating events that the second user is scheduled to participate in or has participated in, and may also include information indicating tasks that the second user is scheduled to perform or has performed. Furthermore, the event information relating to the second user may also include the second user's schedule information.

[0127] The control parameter determination unit 54 of the information processing device 16 may adjust the control parameters for the target device 14 based on event information relating to the second user obtained by the event information acquisition unit. In other words, the control parameter determination unit 54 may estimate the state of the second user based on the event information relating to the second user and generate control parameters that operate the target device 14 in a manner that does not interfere with the state of the second user. According to this embodiment, the modal of the target device 14 can be changed in a manner that is preferable to the second user.

[0128] For example, if the second user has just finished a night shift, opening the curtain (which is the target device 14) or illuminating it may disturb the second user's sleep. Therefore, the control parameter determination unit 54 normally determines the control parameter to open (or illuminate) the curtain (which is the target device 14), but if it detects from event information related to the second user that the second user has just finished a night shift, it may determine the control parameter to shake the curtain (which is the target device 14).

[0129] (Example 7) A product of the same series as the target device 14 installed in the second user's home (in this case, a curtain) may be installed in the first user's home. The curtain installed in the first user's home may be fitted with an IMU (Inertial Measurement Unit) as a sensor 12, which is capable of detecting three-dimensional inertial motion (for example, translational motion and rotational motion in the three orthogonal axes).

[0130] The behavior data acquisition unit 50 of the information processing device 16 may acquire the measured values ​​of the IMU sensor as behavior data of the first user. The behavior information generation unit 53 of the information processing device 16 may acquire the amplitude and period of the curtain's movement on the first user's side based on the measured values ​​of the IMU sensor. The control parameter determination unit 54 of the information processing device 16 may generate control parameters to make the target device 14 (LED strip light) in the second user's home move in the same manner as the amplitude and period of the curtain's movement on the first user's side.

[0131] (Example 8) If the behavior data acquisition unit 50 of the information processing device 16 acquires only trigger information from a human presence sensor (for example, information indicating presence or absence) as behavior information of the first user, the control parameter determination unit 54 of the information processing device 16 may acquire control parameters that indicate a pattern of light that causes the target device 14 (curtain) to sway, which are stored in the storage unit 32 in advance, and use those control parameters to change the modal of the target device 14.

[0132] According to the information processing system 10 of the third embodiment, the movements of the first user (for example, a grandmother living alone in the countryside) can be visualized through modal changes in the target device 14 on the second user's side (a child or grandchild living in the city). This allows the second user to be informed of the first user's situation even when the first and second users are in different locations, enabling the second user to remotely monitor the first user's lifestyle and health. It also provides an opportunity for communication between the first and second users.

[0133] A modification relating to the third embodiment will now be described. <Modification 3-1> A specific device related to the first user's daily routine (for example, a pet robot or a companion robot) may be installed on the first user's side. The sensor 12 on the first user's side may detect the first user's interaction actions with the specific device (for example, saying "good morning," stroking, picking up, etc.) and transmit behavioral data indicating the interaction actions to the information processing device 16. The control parameter determination unit 54 of the information processing device 16 may determine control parameters for the target device 14 based on the first user's interaction actions with the specific device. According to this modification, the loneliness of the first user can be reduced while the second user can feel the actions of the first user.

[0134] <Modification 3-2> Related to Modification 3-1 above, the control parameter determination unit 54 of the information processing device 16 may determine control parameters that change the modal of the target device 14 in a manner corresponding to the manner of the first user's actions (e.g., operation or interaction) toward the specific device. For example, if the first user gently strokes the specific device, the control parameter determination unit 54 may determine a control parameter to slowly sway the curtain representing the target device 14 (for example, swaying at the same speed as the stroking). Also, if the first user says "Good morning" to the specific device, the control parameter determination unit 54 may cause the lighting of the target device 14 to emit a pale yellowish-red color (i.e., the color of the morning sun). This makes it easier for the second user to imagine the actions and state of the first user.

[0135] <Modification 3-3> The control parameter determination unit 54 of the information processing device 16 may detect if a specific device on the first user's side shakes without being caused by the first user's actions. For example, if the motion sensor on the first user's side does not detect the first user, but the IMU of the specific device detects movement of the specific device (for example, shaking caused by wind), the control parameter determination unit 54 may detect that the specific device has moved without being caused by the first user's actions.

[0136] If the control parameter determination unit 54 detects that a specific device has moved without being caused by the actions of the first user, it may suppress the change in the modal of the target device 14 on the second user's side. Alternatively, the control parameter determination unit 54 may notify the second user that the specific device has moved without being caused by the actions of the first user by changing the modal of the target device 14 in a predetermined manner (for example, a predetermined sound, color, or duration of vibration) corresponding to the fact that the specific device has moved without being caused by the actions of the first user.

[0137] <Fourth Embodiment> The fourth embodiment of this disclosure will be described focusing on the differences from the first embodiment. Components of the fourth embodiment that are the same as or equivalent to those of the first embodiment will be denoted by the same reference numerals as those of the first embodiment, and explanations that overlap with the description of the first embodiment will be omitted as appropriate.

[0138] The outline of the fourth embodiment will now be described. The information processing system of the fourth embodiment estimates events in which the first user is involved and the first user's emotions regarding those events, based on the first user's behavioral data, and determines a message for the estimated events and emotions. The information processing system of the fourth embodiment provides the above message to the first user by changing the modal on the target device. In the fourth embodiment, the first user is a child, and the second user is the first user's father or mother.

[0139] The configuration of the information processing system 10 in the fourth embodiment is the same as the configuration of the information processing system 10 in the first embodiment shown in Figure 1. Sensor 12a in the fourth embodiment is a voice recognition sensor installed in the living rooms of the homes of the first and second users. Sensor 12b is an AI speaker installed in the first user's room (child's room in the home). Sensor 12c is a voice recognition sensor installed in the classroom of the school attended by the first user.

[0140] The target device 14 of the fourth embodiment includes an audio output device (speaker, etc.) and outputs sound from the audio output device as a modal change. Specifically, the target device 14 of the fourth embodiment includes an audio output device installed in the living rooms of the homes of the first user and the second user. The target device 14 of the fourth embodiment also includes an audio output device installed in the first user's room (child's room in the home).

[0141] Figure 12 is a block diagram showing the functional blocks of the information processing device 16 of the fourth embodiment. The information processing device 16 of the third embodiment includes a sensor information storage unit 40, an action information storage unit 42, a message storage unit 44, an action data acquisition unit 50, an estimation unit 58, a message determination unit 60, and an equipment control unit 56.

[0142] The message storage unit 44 stores the category of a message and the content of the message (also called "message data") in association. The categories of a message may include encouragement, expectation / joy, and empathy for sadness. For example, the category "encouragement" may be associated with the message data "do your best." For example, the category "expectation / joy" may be associated with the message data "it's finally happening" or "I'm sure it will work out." For example, the category "empathy for sadness" may be associated with the message data "that's sad, I understand."

[0143] Message categories may include events in which the first user is involved. For example, message categories may include travel events or funeral events. A travel event category may be associated with the message data "I'm looking forward to the trip," which shows positive empathy. A funeral event category may be associated with the message data "It's sad, but let's go and say a proper goodbye," which shows empathy for grief and the importance of participation.

[0144] The behavior data acquisition unit 50 acquires behavior data of the first user transmitted from multiple sensors 12. The estimation unit 58 estimates the events in which the first user is involved and the first user's emotions towards those events, based on the behavior data of the first user acquired by the behavior data acquisition unit 50.

[0145] The message determination unit 60 corresponds to the control parameter determination unit 54 of the first embodiment. The message determination unit 60 determines a message to the first user in relation to the events in which the first user is involved and the emotions of the first user, which are estimated by the estimation unit 58, as control parameters for the target device 14. The device control unit 56 notifies the first user of the message determined by the message determination unit 60 by changing the modal in the target device 14.

[0146] Figure 13 is a flowchart showing the operation of the information processing device 16 of the fourth embodiment. The information processing device 16 repeatedly executes the processes shown in Figure 13. The operation of the information processing system 10 of the fourth embodiment will be described below with reference to Figure 13.

[0147] Each of the multiple sensors 12 detects the behavior data of the first user and transmits it to the information processing device 16. The behavior data of the first user includes data relating to communication between the first user and others (for example, the second user or other people). The communication data includes text data and voice data (hereinafter also referred to as "conversation data") that indicate the content of the conversation. The conversation data includes, for example, conversation data between parents and children, conversation data between teachers and students, and conversation data between children and AI speakers.

[0148] The behavior data acquisition unit 50 of the information processing device 16 receives behavior data of the first user transmitted from each of the multiple sensors 12 and stores the behavior information of the first user, including the behavior data of the first user, in the behavior information storage unit 42 (S30). Figure 14 shows an example of behavior information in the fourth embodiment. The behavior information of the first user in the fourth embodiment includes the same items as the behavior information of the target user in the first embodiment.

[0149] Returning to Figure 13, the estimation unit 58 of the information processing device 16 estimates the events in which the first user is involved and the first user's emotions regarding those events based on the first user's behavioral data acquired in S30 (S31). In the fourth embodiment, the estimation unit 58 extracts pre-registered characteristic words related to events and emotions from the first user's behavioral data, and estimates the events in which the first user is involved and the first user's emotions regarding those events based on the extracted characteristic words. Specifically, the estimation unit 58 estimates events and emotions related to the first user based on the characteristic words contained in each of the multiple behavioral data acquired at multiple points in time.

[0150] Figure 15 shows an example of estimating events and emotions related to a first user. As shown in Figure 15, the estimation unit 58 performs morphological analysis to segment the behavioral data into multiple phrases and extracts a first characteristic word from among the segmented phrases. The first characteristic word is a keyword used to identify events and tasks in which the first user is involved. The memory unit 32 may maintain a database showing predetermined candidates for the first characteristic word (e.g., test, baseball game, trip, etc.). The estimation unit 58 may also extract a phrase that matches the candidate from among the multiple phrases in the behavioral data as the first characteristic word.

[0151] The estimation unit 58 further extracts second feature words related to the same or similar first feature word (in Figure 15, "test") from among the multiple words segmented by morphological analysis. The second feature words are keywords used to identify the time or date when an event or task involving the first user is carried out. If there are words indicating a time or date in the words before or after the first feature word, the estimation unit 58 may extract those words as second feature words.

[0152] The estimation unit 58 further extracts third feature words related to the same or similar first feature word (in Figure 15, "test") from among the multiple words segmented by morphological analysis. The third feature word is a keyword that indicates the first user's emotion towards an event in which the first user is involved. The memory unit 32 may maintain a database that associates predetermined candidates for the third feature word with emotions for each first feature word. In addition, multiple types of emotions may be used, for example, the 24 emotions shown in Plutchik's Wheel of Emotions.

[0153] The estimation unit 58 extracts words that match the candidate from among multiple words in the behavioral data as third feature words, and identifies the emotion associated with that third feature word as the first user's emotion in response to the event. In the example in Figure 15, the emotion "anxiety" is identified based on the third feature word "I wonder if I can do it well." Also, the emotion "expectation" is identified based on the third feature word "I'll do my best."

[0154] Returning to Figure 13, the message determination unit 60 of the information processing device 16 refers to the message storage unit 44 and determines a message for the events and emotions related to the first user estimated in S31 (S32). In the fourth embodiment, the message storage unit 44 determines a message to the first user based on the events and emotions related to the first user.

[0155] Example of message determination when an event concerning the first user belongs to the first category: The first category is an event in which the first user actively participates. Events belonging to the first category include, for example, tests and club activity competitions. If the first user's feelings towards the event are negative, such as anxiety or sadness, the message determination unit 60 may select a message data associated with the support category from among the multiple message data stored in the message storage unit 44 as the message to the first user.

[0156] On the other hand, if the first user's feelings towards the event are positive, such as anticipation or joy, the message determination unit 60 may select a message from among the multiple message data stored in the message storage unit 44 that corresponds to the anticipation / joy category or the positive category as the message to the first user.

[0157] Example of message determination when an event concerning the first user belongs to the second category: The second category is an event in which multiple people, including the first user, actively participate. Events belonging to the second category include, for example, travel, outings, funerals, etc. If multiple different types of emotional information are identified for multiple people, the message determination unit 60 may, instead of selecting message data associated with a specific emotion, select message data associated with the event as the message to the first user. For example, if the family is going on a trip tomorrow, the message determination unit 60 may select the general message data "I'm looking forward to tomorrow's trip" associated with the travel event. Also, if the family is attending a relative's funeral, the message determination unit 60 may select the general message data "It's sad, but let's go and say a proper goodbye" associated with the funeral event.

[0158] The device control unit 56 of the information processing device 16 notifies the first user of the message determined in S32 by changing the modal of the target device 14 (S33). In the fourth embodiment, the device control unit 56 determines the timing of the message notification based on the second characteristic word of the first user's behavior information. For example, if the event is to be held today, the device control unit 56 may determine the notification timing to be this morning (the day of the event). If the event is to be held tomorrow, the device control unit 56 may determine the notification timing to be this evening (the day before the event). At the determined notification timing, the device control unit 56 transmits audio data indicating the message determined in S32 to the target device 14, and causes the audio of the message determined in S32 to be output from the audio output device of the target device 14.

[0159] In the example shown in Figure 15, the device control unit 56 may determine that the test will be conducted tomorrow based on the second characteristic word of the behavior information of data number 5, and may decide that the notification timing for the message will be tomorrow morning (the day of the test). The message determination unit 60 may select the message "It will go well" which is associated with the emotion "expectation" based on the emotion judgment result "expectation" regarding the behavior information of data number 5. The device control unit 56 may output the voice of the message "It will go well" from the target device 14 installed in the first user's room tomorrow morning (the day of the test).

[0160] An example of the timing of message determination is described below. The message determination unit 60 may postpone the determination of a message for a specific event involving the first user until the date of the event is determined based on the second characteristic word. Once the date of the event is determined based on the second characteristic word, the message determination unit 60 may determine a message for the first user based on the type of event and the first user's feelings towards the event (typically, the feelings associated with the third characteristic word of the behavioral data that determined the date of the event), and store it in the message storage unit 44. The device control unit 56 may output an audio message from the target device 14 on the morning of the event or the night before the event, indicating the message stored in the message storage unit 44.

[0161] According to the information processing system 10 of the fourth embodiment, by notifying the first user of events in which the first user is involved and the first user's feelings towards those events, it is possible to support the first user and provide an opportunity for communication between the first user and other users (including the second user). Furthermore, according to the information processing device 16 of the fourth embodiment, by using communication data (conversation data, etc.) exchanged between the first user and the second user, it is possible to accurately estimate events in which the first user is involved and the first user's feelings towards those events.

[0162] A modified example relating to the fourth embodiment will now be described. <Modified Example 4-1> The message determination unit 60 (or device control unit 56) of the information processing device 16 may determine the target device 14 to change the modal based on the relationship between the first user and the second user who communicates with the first user.

[0163] For example, let's assume that the relationship between the first user (let's call them a child) and the second user (let's call them the father and mother) is that of a family. Let's also assume that there is a target device 14 installed in the child's room and another target device 14 installed in the parents' room. In this case, the message determination unit 60 (or the device control unit 56) may select the target device 14 installed in the parents' room as the output destination for messages related to events and emotions concerning the first user. For example, if the event concerning the first user is a test, and the first user is feeling anxious about the test, the message determination unit 60 may select "Please cheer me on during the test" as the message to be output from the target device 14 installed in the parents' room. According to this modification, the target device 14 whose modal changes can be appropriately selected from the perspective of promoting communication between multiple users.

[0164] <Modification 4-2> The message determination unit 60 (or device control unit 56) of the information processing device 16 may determine the timing for changing the modal of the target device 14 based on information indicated by the behavior data of the first user and information indicating the status of communication between the first user and the second user.

[0165] For example, the information processing device 16 may further include a user location detection unit (not shown). The user location detection unit may detect the locations of the first user and the second user based on analysis of images captured by a camera or positioning processing based on sensor data. The message determination unit 60 (or device control unit 56) may suppress the change of the modal of the target device 14 installed at that location, even if it is the timing for notifying the first user of a message (the timing when the modal of the target device 14 should be changed), if it is detected that the first user and the second user are in the same location. For example, the message determination unit 60 (or device control unit 56) may suppress the output of message audio from the target device 14 installed at that location. This is because outputting message audio from the target device 14 may cause annoyance to either the first or second user. According to this modification, the modal of the target device 14 can be changed at an appropriate timing.

[0166] <Modification 4-3> As a modification related to Modification 4-2, the message determination unit 60 (or device control unit 56) of the information processing device 16 may notify the first user of a message (for example, a message of encouragement) by changing the modal on the target device 14 when the first user is in a different location from the second user.

[0167] For example, the message determination unit 60 (or device control unit 56) may detect that it is time to notify a message when it is detected that the first user and the second user are in different locations, and may output an audio message of encouragement from the target device 14 installed at the location of the first user. According to this modification, the modal on the target device 14 is changed when the first user is in a different location from the second user, making it less likely for the first user to feel annoyed. In other words, according to this modification, the modal on the target device 14 can be changed at an appropriate time, and a message can be provided to the first user at an appropriate time.

[0168] <Modification 4-4> The message determination unit 60 (which can also be called the "message acquisition unit") of the information processing device 16 may present information to the second user by transmitting information indicating an event in which the first user is involved and the first user's feelings about that event to the second user's information terminal. The message determination unit 60 may acquire the second user's message regarding the event and feelings about the first user, transmitted from the second user's information terminal, and determine that the second user's message is to be provided to the first user. According to this modification, the first user can be notified of the message entered by the second user, and communication between the first user and the second user can be promoted.

[0169] Furthermore, if the message determination unit 60 does not receive a message from the second user regarding events and emotions related to the first user transmitted from the second user's information terminal, it may determine a message to provide to the first user by referring to the message storage unit 44, as described in the fourth embodiment.

[0170] <Modification 4-5> The information processing device 16 may further include an inquiry unit (not shown). There are cases where the date of an event (such as a test) in which the first user is involved cannot be determined from the first user's behavior data. For example, this may occur if a second characteristic word is not detected from the behavior data, or if the second characteristic word is "next time" or "next week". In this case, the inquiry unit may send information to the information terminal of a user different from the first user (for example, a second user such as a father or mother) inquiring about the date of the event in which the first user is involved, as indicated by the first characteristic word. The inquiry unit may determine the date of the event in which the first user is involved by recording the information on the date of the event in which the first user is involved, which is sent from the information terminal of the inquired user, in the first user's behavior information.

[0171] Furthermore, even if the date of an event involving the first user can be determined, there may be cases where the first user's feelings towards that event cannot be estimated. For example, this may occur if the third characteristic word is not detected from the first user's behavioral data. In this case, the inquiry unit may send information to the first user's information terminal inquiring about the first user's feelings towards the event in which the first user is involved. The inquiry unit may determine the first user's feelings towards the event by recording the information indicating the first user's feelings towards the event, which is sent from the first user's information terminal, in the first user's behavioral information.

[0172] <Modification 4-6> In the above embodiment, the modal change of the target device 14 was to output a message audio from the target device 14, but the modal change of the target device 14 is not limited to this. For example, if the first user's feelings towards the event in which the first user is involved are positive, the information processing device 16 may control the target device 14 (lighting fixture) to emit light in a first color (for example, green) on the morning of the event. Alternatively, the information processing device 16 may output uplifting music from the target device 14 (sound equipment).

[0173] On the other hand, if the first user's emotions towards the event they are involved in are anxious or negative, the information processing device 16 may control the target device 14 (lighting fixture) to emit light in a second color (for example, orange) on the morning of the event. Alternatively, the information processing device 16 may play relaxing music from the target device 14 (sound equipment). This provides the first user with an environment that helps them to calm down.

[0174] <Fifth Embodiment> The fifth embodiment of this disclosure will be described focusing on the differences from the first embodiment. Components of the fifth embodiment that are the same as or equivalent to those of the first embodiment will be denoted by the same reference numerals as those of the first embodiment, and explanations that overlap with the description of the first embodiment will be omitted as appropriate.

[0175] The outline of the fifth embodiment will now be described. The information processing system of the fifth embodiment detects the status of a gathering (such as a family gathering) of multiple users and, at a later point in time, changes the modal of the target device to reproduce the gathering status. The multiple users in the fifth embodiment are the father, mother, and children who make up the family. If the first user is the child, then the second user, who is different from the first user, corresponds to the father and mother.

[0176] The configuration of the information processing system 10 in the fifth embodiment is the same as the configuration of the information processing system 10 in the first embodiment shown in Figure 1. The multiple sensors 12 in the fifth embodiment are installed in the homes of multiple users (hereinafter referred to as "user homes"). For example, sensor 12a is a voice recognition sensor installed in the living room of a user's home.

[0177] The target device 14 of the fifth embodiment includes an audio output device (such as a speaker), and outputs audio from the audio output device as a modal change. Specifically, the target device 14 of the fifth embodiment includes an audio output device installed in the living room of the user's home.

[0178] Figure 16 is a block diagram showing the functional blocks of the information processing device 16 of the fifth embodiment. The information processing device 16 of the fifth embodiment includes a sensor information storage unit 40, a communication information storage unit 46, a control parameter storage unit 48, a communication data acquisition unit 62, a communication state detection unit 64, a control parameter determination unit 54, and an equipment control unit 56.

[0179] The communication information storage unit 46 stores communication information corresponding to the behavioral information of the first embodiment. Details of the communication information will be described later. The control parameter storage unit 48 stores control parameters for the target device 14.

[0180] The communication data acquisition unit 62 acquires communication data between multiple users. The communication state detection unit 64 detects, based on the communication data acquired by the communication data acquisition unit 62, that communication between multiple users is in an active state. Furthermore, the communication state detection unit 64 detects, based on the above communication data, that communication between multiple users is in a subdued state.

[0181] The control parameter determination unit 54 determines control parameters for the target device 14 when it detects that communication among multiple users is active or when it detects that communication among multiple users is subsiding. The control parameters for the target device 14 include at least some of the data from the communication data acquired by the communication data acquisition unit 62. The control parameter determination unit 54 also acts as a storage unit, storing the determined control parameters in the control parameter storage unit 48.

[0182] The device control unit 56, acting as an output unit, changes the modal in the target device 14 using control parameters stored in the control parameter storage unit 48 after a predetermined time has elapsed since the communication data was acquired by the communication data acquisition unit 62. For example, the device control unit 56 outputs at least a portion of the audio of a conversation between multiple users as data from the audio output device of the target device 14.

[0183] Figure 17 is a flowchart showing the operation of the information processing device 16 of the fifth embodiment. The information processing device 16 repeatedly executes the process shown in Figure 17. The operation of the information processing system 10 of the fifth embodiment will be described below with reference to Figure 17.

[0184] Each of the multiple sensors 12 transmits communication data between multiple users in the user's home to the information processing device 16. The communication data in the fifth embodiment includes conversation data (for example, voice data of the conversation). The communication data acquisition unit 62 of the information processing device 16 receives the communication data transmitted from each of the multiple sensors 12 (S40). When the communication data acquisition unit 62 receives the communication data (Y in S40), it stores the communication information including the communication data in the communication information storage unit 46 (S41).

[0185] Figure 18 shows an example of communication information. Multiple items of the communication information correspond to multiple items of the behavior information in the first embodiment. The type of communication data, acquisition tool, and acquisition date and time are provided by the sensor 12 along with the communication data. The communication data acquisition unit 62 sets the item "acquisition location" to the installation location of the sensor 12 that transmitted the communication data, which is stored in the sensor information storage unit 40. The communication data acquisition unit 62 sets the item "communication data details" to the communication data transmitted from the sensor 12. Note that the communication information in Figure 18 records family conversation data from the living room of the home regarding the favorite sports team "Impulse".

[0186] Returning to Figure 17, the communication state detection unit 64 of the information processing device 16 determines the level of engagement in communication between multiple users based on characteristic words (characteristic words) included in the communication data acquired in S40 (S42).

[0187] Figure 19 shows an example of determining the level of engagement in communication. As shown in Figure 19, the communication state detection unit 64 performs morphological analysis to segment the communication data into multiple phrases and extracts a first characteristic word from among the segmented phrases. The first characteristic word is a keyword for identifying events or tasks related to communication, and may be, for example, a topic of conversation or a phrase indicating the user's interests or concerns. The storage unit 32 may maintain a database showing predetermined candidates for the first characteristic word. Candidates for the first characteristic word may include various sports team names, various event names, and various place names. The communication state detection unit 64 may also extract a phrase that matches a candidate from among multiple phrases in the communication data as the first characteristic word.

[0188] The communication state detection unit 64 further extracts second feature words related to the same or similar first feature word (in Figure 15, the sports team "Impulse") from among multiple phrases segmented by morphological analysis. The second feature words are phrases that indicate the user's actions or emotions, and are used to estimate the level of engagement in the communication. The memory unit 32 may maintain a database showing predetermined candidates for second feature words for each first feature word (or category of the first feature word). The communication state detection unit 64 may also extract as second feature words from among multiple phrases in the communication data that match candidates associated with the first feature word in the database.

[0189] In the fifth embodiment, in the database, each of the multiple candidate second feature words is associated with the emotions of multiple users communicating (typically the speaker's emotion). Any of the 24 emotions represented by Plutchik's Wheel of Emotions may be used as the user's emotion. In the example in Figure 19, the three second feature words "That's strong!", "I want to watch it!", and "Let's go watch it!" are all associated with the emotion "joy". Each of the multiple candidate second feature words may also be associated with either the category "positive emotion" or the category "negative emotion". The communication state detection unit 64 identifies the emotion associated with the second feature word extracted from the communication data as the user's emotion.

[0190] The communication state detection unit 64 determines the level of engagement in communication by using multiple communication data sets, which are acquired at the same location (or at the same location and using the same acquisition tool) within a predetermined time (within one hour in the fifth embodiment) and which contain a common first characteristic word, as the target for determination. The communication state detection unit 64 repeatedly performs at least one of the following (both in the fifth embodiment): determining whether the communication between multiple users is in an engaged state, and determining whether the communication between multiple users is in a subdued state, until the conversation in which the same first characteristic word appears ends (up to one hour in the fifth embodiment).

[0191] In the fifth embodiment, the emotions of multiple users (typically the speaker's emotions) are pre-associated in the database with either the category "positive emotion" or the category "negative emotion". For example, the emotion "joy" is classified as a positive emotion. The communication state detection unit 64 determines whether the user's emotion based on the second characteristic word is positive or negative for each of the multiple communication data to be judged acquired within a predetermined time. A score of +1 is assigned to positive emotions, and a score of -1 is assigned to negative emotions. The communication state detection unit 64 aggregates the scores based on the user's emotions across the multiple communication data to be judged.

[0192] The communication state detection unit 64 determines that communication is thriving if the score aggregated across multiple communication data to be judged is positive. On the other hand, if the score aggregated across multiple communication data to be judged is negative, it determines that communication is declining. As a modification, the communication state detection unit 64 may also determine that communication is thriving if the score aggregated across multiple communication data to be judged is above a predetermined first threshold (for example, +2), and may determine that communication is declining if the score is below a predetermined second threshold (for example, -3).

[0193] An example of processing related to determining the level of engagement in communication will be explained. (Example 1) The database of second characteristic words may be determined independently of the first characteristic words. Even if the communication state detection unit 64 cannot detect a common first characteristic word from multiple communication data acquired at the same location (or at the same location and with the same acquisition tool) within a predetermined time (for example, within one hour), it may refer to the database and detect the second characteristic word from the multiple communication data.

[0194] If the communication state detection unit 64 has not detected a common first characteristic word across multiple communication data, and has determined that the communication is lively based on a second characteristic word (e.g., laughter), it may acquire audio other than the user's voice and extract the first characteristic word from that audio. For example, the communication state detection unit 64 may extract a phrase from audio output from a television or a joke phrase spoken by a comedian as the first characteristic word.

[0195] (Example 2) Communication data may include audio data of the communication or images of the users communicating. The communication state detection unit 64 may determine the level of excitement of the communication based on a second characteristic word, or it may determine it based on a second characteristic word and also based on the audio data of the communication. For example, the communication state detection unit 64 may determine that the communication is exciting when the volume of the conversation is above a predetermined threshold (e.g., 60 decibels).

[0196] Furthermore, the communication state detection unit 64 may determine the level of excitement in the communication based on the second characteristic word, or it may determine it based on the second characteristic word as well as the user's body posture and movements (including gestures). For example, the communication state detection unit 64 may analyze an image of the user and determine that the communication is exciting if it detects that the user is raising their hands in a victory pose or making a fist pump.

[0197] (Example 3) Communication data may include audio data of communication or images of users communicating. The communication state detection unit 64 may determine that communication is dying down if, despite multiple users being in the same place (for example, the living room of a user's home), no audio of conversation is detected for a predetermined period of time or longer. Alternatively, the communication state detection unit 64 may determine that communication is dying down if, despite multiple users being in the same place (for example, the living room of a user's home), no common feature word (first feature word) is included in the utterances of the multiple users.

[0198] Returning to Figure 17, if the communication state satisfies the notification condition (Y in S43), the control parameter determination unit 54 of the information processing device 16 determines the control parameters for the target device 14 and stores those control parameters in the control parameter storage unit 48 (S44). In the fifth embodiment, the notification condition is satisfied when it is determined that the communication state is high, and also satisfied when it is determined that the communication state is low. If the communication state does not satisfy the notification condition (N in S43), the process in S44 is skipped. If the communication data transmitted from the sensor 12 is not accepted (N in S40), the processes in S41-S44 are skipped.

[0199] In S44, the control parameter determination unit 54 recognizes the voices of conversations between multiple users included in the communication data, and detects characteristic words that appear in excess of a predetermined criterion within a predetermined time in the voice recognition results of the conversations. Based on the detected characteristic words, the control parameter determination unit 54 determines the control parameters to be stored in the control parameter determination unit 54.

[0200] In the fifth embodiment, the control parameter determination unit 54 stores a pair of audio data of a first characteristic word and audio data of a second characteristic word, extracted from communication data in which communication is determined to be active, as control parameters in the control parameter storage unit 48. In addition, the control parameter determination unit 54 stores a pair of audio data of a first characteristic word and audio data of a second characteristic word, extracted from communication data in which communication is determined to be inactive, as control parameters in the control parameter storage unit 48.

[0201] The set of audio data for the first feature word and the audio data for the second feature word will also be referred to as "family chat audio" below. In the example in Figure 19, the control parameter determination unit 54 may store in the control parameter storage unit 48 a control parameter that includes the audio data (family chat audio) of "Impulse, let's go see it," which is a combination of the first feature word "Impulse" and the second feature word "Let's go see it."

[0202] The control parameter determination unit 54 also stores data indicating the timing of playback of the conversational audio as a control parameter in the control parameter storage unit 48. The timing of playback of the conversational audio is a predetermined waiting time (for example, 1 hour) after it is determined that the communication state is at its peak or at its decline. The value of the waiting time may be determined based on the developer's knowledge or by conducting experiments using the information processing system 10.

[0203] The device control unit 56 of the information processing device 16 waits until the playback timing indicated by the control parameters stored in the control parameter storage unit 48 is reached. When the playback timing indicated by the control parameters is reached (Y in S45), the device control unit 56 changes the modal of the target device 14 in the manner indicated by the control parameters (S46). In the fifth embodiment, in S46, the device control unit 56 transmits a control signal to the target device 14 that includes audio data of the family conversation included in the control parameters, and reproduces the family conversation from a predetermined time ago by outputting the family conversation audio from the audio output device of the target device 14. If the playback timing indicated by the control parameters is not reached (N in S45), the process in S46 is skipped.

[0204] According to the information processing system 10 of the fifth embodiment, the modal of the target device 14 is changed to play back some of the communication data from when communication between multiple users became more or less engaging. In this way, by recreating past communication between multiple users, it is possible to provide a trigger for new communication between multiple users and promote new communication. Furthermore, by making only a portion of the communication data the target of playback, it is possible to prevent users from losing interest when they feel the modal change of the target device 14, and to reduce the size of the audio data stored in the control parameter storage unit 48.

[0205] Furthermore, in the fifth embodiment, by using conversation data as communication data, the level of engagement in communication can be detected with high accuracy, and communication between multiple users can be accurately reproduced by playing back the conversation audio.

[0206] A modified example relating to the fifth embodiment will now be described. <Modified Example 5-1> As previously described, the communication information stored in the communication information storage unit 46 includes the acquisition location. The acquisition location includes the location where communication between multiple users took place. The control parameter determination unit 54 of the information processing device 16 may select a target device 14 installed at the location where the communication took place from among multiple target devices 14 according to the level of engagement of the communication between multiple users, and determine control parameters for changing the modal in the selected target device 14.

[0207] Furthermore, the device control unit 56 of the information processing device 16 may use the control parameters determined by the control parameter determination unit 54 to select a target device 14 from among a plurality of target devices 14 that is installed in a location where communication between multiple users has taken place, and change the modal in the selected target device 14.

[0208] According to this modification, the modal of the target device 14, which is associated with the location where communication took place between the first user (e.g., a child) and the second user (e.g., the father), is changed. This makes it easier for the first and second users to recall past communication, thereby further promoting communication between them.

[0209] <Modification 5-2> The control parameter determination unit 54 (or device control unit 56) of the information processing device 16 may determine the target device 14 whose modal is changed based on the relationship between multiple users communicating (i.e., the relationship between the first user and the second user).

[0210] For example, let's assume that the relationship between the first user (let's call them a child) and the second user (let's call them a father and mother) is that of a family. Let's also assume that there is a target device 14 installed in the child's room and another target device 14 installed in the parents' room. In this case, the control parameter determination unit 54 (or the device control unit 56) may select the target device 14 installed in the parents' room as the output destination for the family conversation voice. According to this modification, by taking into account the relationships between multiple users, it is possible to appropriately select the target device 14 that will change the modal from among the multiple target devices 14.

[0211] <Modification 5-3> The control parameter determination unit 54 (or device control unit 56) of the information processing device 16 may determine the timing for changing the modal of the target device 14 based on information indicating the status of communication between multiple users. According to this modification, it becomes easier to change the modal of the target device 14 at an appropriate timing, and communication between multiple users can be supported.

[0212] For example, the control parameter determination unit 54 may determine the playback frequency and number of times the family gathering audio is played according to the excitement level (the aggregated score mentioned above, for example, in 10 stages from +1 to +10), and may also store data indicating the playback frequency and number of times the family gathering audio is played as a control parameter in the control parameter storage unit 48. The control parameter determination unit 54 may increase the playback frequency and number of times as the excitement level increases (as it approaches +10). For example, if the excitement level is +10, the number of times it is played may be set to 5, and the timer may be set to play it after 1 week, 1 month, 6 months, 1 year, and 5 years. On the other hand, if the excitement level is +1, the number of times it is played may be set to 1, and the timer may be set to play it only after 1 week.

[0213] Furthermore, the control parameter determination unit 54 may determine the playback frequency and number of times the family gathering audio is played according to the level of excitement decline (the aggregated score mentioned above, for example, in 10 stages from -1 to -10), and may also store data indicating the playback frequency and number of times the family gathering audio is played as a control parameter in the control parameter storage unit 48. The control parameter determination unit 54 may decrease the playback frequency and number of times as the level of excitement decline increases (as it approaches -10). For example, if the level of excitement decline is -1, the number of times it is played may be set to 3, and the timer may be set to play it after 1 week, 1 month, and 6 months. On the other hand, if the level of excitement is -10, the number of times it is played may be set to 1, and the timer may be set to play it only after 1 week.

[0214] <Modification 5-4> As a modification related to Modification 5-3, the message determination unit 60 (or device control unit 56) of the information processing device 16 may change the modal in the target device 14 using at least some of the data of the communication data (e.g., conversational voice) when the first user is in a different location from the second user.

[0215] For example, the message determination unit 60 (or the device control unit 56) may, when it detects that the first user and the second user are in different locations, output a conversational voice message from the target device 14 installed at the location of the first user or the second user. According to this modification, since the modal on the target device 14 is changed when the first user is in a different location from the second user, it is possible to suppress the sense of obligation that arises in the first or second user to communicate with the other, and to prevent the first or second user from feeling bothered.

[0216] <Modification 5-5> The information processing device 16 may further include a search unit (not shown). The search unit searches for event information that should be recommended to the user communicating (e.g., the first user) based on the characteristic words detected by the communication state detection unit 64. For example, the search unit may send a search request to an external information retrieval system that includes the first characteristic word as a search keyword and obtain search results based on the first characteristic word. The event information may include, for example, information on a game in which the sports team indicated by the first characteristic word is participating, or a travel plan to watch that game.

[0217] The control parameter determination unit 54 of the information processing device 16 may store control parameters, including event information based on the first characteristic word retrieved by the search unit, in the control parameter storage unit 48. The device control unit 56 of the information processing device 16 notifies the user of event information based on the first characteristic word when changing the modal in the target device 14 using at least some of the data of the communication data (e.g., the first characteristic word and the second characteristic word). For example, the device control unit 56 may output audio indicating the event information from the audio output device of the target device 14. Alternatively, the device control unit 56 may display the event information on the display device of the target device 14. According to this modification, event information corresponding to the characteristic word can be provided to the user, further promoting communication among multiple users.

[0218] <Modification 5-6> In the fifth embodiment, the target device 14 is equipped with an audio output device, but the target device 14 is not limited to this configuration. For example, the target device 14 may be equipped with an illumination device instead of, or together with, the audio output device. The control parameter determination unit 54 of the information processing device 16 may determine control parameters that indicate the illumination mode of the target device 14 along with the audio data and store them in the control parameter storage unit 48. The device control unit 56 of the information processing device 16 may cause the illumination device of the target device 14 to emit light in the manner indicated by the control parameters. For example, the device control unit 56 may cause the target device 14 to output the audio "strong impulse" and also cause the target device 14 to emit light in the team color of the impulse (for example, blue). According to this modification, it is possible to help the user vividly recall memories of past communications.

[0219] <Modification 5-7> The information processing system 10 may further include an interactive AI agent (not shown), and the communication data may include the user's speech data as well as the interactive AI agent's speech data. For example, when a user says, "Impulse is strong," the interactive AI agent may say, "Impulse had several injured players, but they all recovered and made it this far," or it may light up the team's color blue from an attached LED. This can enhance the user's desire to support the team and help to liven up conversations with family members. Furthermore, by playing back the audio including the content of the interactive AI agent's speech at a later date, the well-being time of multiple users can be increased.

[0220] <Modification 5-8> When the device control unit 56 of the information processing device 16 outputs at least a portion of the audio of the conversation data from the target device 14, it may output the audio (converted audio) after converting the original human voice (i.e., conversational voice) using known technology. Alternatively, the device control unit 56 may convert the original human voice in a manner that corresponds to the type of first characteristic word. For example, if the first characteristic word is related to an animal, the original human voice may be converted to reflect the sound of that animal.

[0221] <Modification 5-9> The sensor 12 may include a sensor that acquires noise in the spatial environment in which the target device 14 (sound output device) is installed. If the noise exceeds a predetermined threshold, the device control unit 56 of the information processing device 16 may cause the target device 14 to output special sounds such as animal sounds in order to attract the attention of users present in the space in which the target device 14 is installed. After outputting the special sounds, the device control unit 56 may cause the target device 14 to output conversational sounds extracted from communication data, or it may cause the target device 14 to output converted sounds obtained by converting the conversational sounds. If the noise is below the predetermined threshold, the device control unit 56 may output conversational sounds (or converted sounds) as usual without outputting special sounds.

[0222] The present disclosure has been described above based on the first to fifth embodiments. Each embodiment is illustrative, and it will be understood by those skilled in the art that various modifications are possible in the combination of components or processing processes of each embodiment, and that such modifications are also within the scope of the present disclosure.

[0223] Any combination of the embodiments and modifications described above is also useful as an embodiment of the present disclosure. The new embodiments resulting from such combinations will possess the combined effects of the respective embodiments and modifications. Furthermore, it will be understood by those skilled in the art that the functions to be performed by each constituent element described in the claims can be achieved by each component shown in the embodiments and modifications individually or in combination thereof. In addition, where terms such as "first," "second," etc. are used in this specification or in the claims, unless otherwise specified, they do not indicate any order or importance, but are used to distinguish one configuration from another.

[0224] <Note> Based on the above description of embodiments and modifications, the following technologies are disclosed. [Technology 1-1] An information processing method in which a computer performs the following actions: acquiring user behavior data, determining control parameters for changing the modal in a target device based on the user behavior data, and changing the modal in the target device using the control parameters. This information processing method can facilitate communication between one user and other users based on the behavior data of one user. [Technology 1-2] An information processing device comprising: an acquisition unit for acquiring user behavior data; a determination unit for determining control parameters for changing the modal in a target device based on the user behavior data; and an output unit for changing the modal in the target device using the control parameters. This information processing device can facilitate communication between one user and other users based on the behavior data of one user.

[0225] [Technology 2-1] An information processing method according to Technology 1-1, wherein the method detects changes in the user based on the user's behavior data and determines the control parameters based on the user's changes. This information processing method facilitates communication between a user and other users by changing the modal of the device in response to changes in a user (for example, daily growth, changes in emotions or health status, etc.). [Technology 2-2] An information processing method according to Technology 2-1, wherein the user's behavior data includes data relating to communication between the user and other users. This information processing method efficiently detects changes in a user that should be used as a trigger for changing the modal of the device based on communication between the user and other users. [Technology 2-3] An information processing method according to Technology 2-2, wherein the method further acquires data relating to the location where communication between the user and other users took place and changes the modal of a target device installed at the location where communication between the user and other users took place. This information processing method makes it easier for the notified user to feel the changes of the user more personally by changing the modal of the device in conjunction with the location where communication between a user and other users took place. [Technology 2-4] An information processing method according to Technology 2-1, which determines the target device whose modal is to be changed based on the relationship between the user and other users with whom the user communicates. This information processing method allows for the appropriate selection of the target device whose modal is to be changed. [Technology 2-5] An information processing method according to Technology 2-1, which determines the timing for changing the modal of the target device based on the communication status between the user and other users indicated by the user's behavioral data. This information processing method allows for the modal of the device to be changed at an appropriate timing.[Technical 2-6] An information processing device comprising: an acquisition unit for acquiring user behavior data; a detection unit for detecting changes in the user based on the user behavior data; a determination unit for determining control parameters for changing the modal in a target device based on the changes in the user; and an output unit for changing the modal in the target device using the control parameters. According to this information processing device, communication between one user and other users can be promoted by changing the modal of a device in response to changes in a certain user (for example, daily growth, changes in emotions or health status, etc.).

[0226] [Technical 3-1] The information processing method according to Technical 1-1, wherein the user's behavior data is generated in accordance with the user's movements, and control parameters for changing the modal of a target device installed in a space where a user other than the user is present are determined based on the user's behavior data. According to this information processing method, the movements of the first user (the user) can be visualized through modal changes in the device on the side of the second user (a user other than the user). This makes it possible to convey the situation of the first user to the second user even when the first user and the second user are in different locations, and also provides an opportunity for communication between the first user and the second user. [Technical 3-2] The information processing method according to Technical 3-1, wherein control parameters for changing the physical shape of the target device are determined as modals in the target device. According to this information processing method, the movements of the first user can be clearly presented to the second user through changes in the physical shape of the target device. [Technical 3-3] The information processing method according to claim 3-1, wherein the user's behavior data includes information regarding the direction in which the user is moving. According to this information processing method, the direction in which the first user is moving can be conveyed to the second user. [Technical 3-4] The information processing method according to claim 3-1, wherein the user's behavior data includes information regarding the speed or acceleration of the user's movement. This information processing method allows the speed or acceleration of the first user's movement to be transmitted to the second user. [Technical 3-5] The information processing method according to claim 3-4, wherein, when the speed or acceleration of the user's movement exceeds a predetermined threshold, the control parameters are determined and predetermined information is notified to a third party who is neither the user nor the other user. This information processing method allows for responding to danger to the first user and enhancing the safety of the first user by notifying a third party when it is estimated that the first user's movement is an irregular movement different from normal movement. [Technical 3-6] The information processing method according to claim 3-1, wherein, when the user's movement changes periodically, the control parameters are determined to reproduce that period.According to this information processing method, by reflecting the periodicity of the first user's movements in the modal changes of the device, the movements of the first user in the real world can be conveyed to the second user in a realistic manner. [Technical 3-7] The information processing method according to claim 3-1, wherein the control parameters are adjusted based on weather information of the location where the different user is present. According to this information processing method, the modal changes of the device are made more recognizable to the second user, and the movements of the first user are made more recognizable to the second user. [Technical 3-8] The information processing method according to claim 3-1, wherein the activity intensity of the different user is acquired, and the control parameters are adjusted based on the activity intensity of the different user. According to this information processing method, the modal changes of the device are made more recognizable to the second user, and the movements of the first user are made more recognizable to the second user. [Technical 3-9] The information processing method according to claim 3-1, wherein event information relating to the different user is acquired, and the control parameters are adjusted based on the event information relating to the different user. According to this information processing method, the modal of the device can be changed in a manner suitable for the second user. [Technical 3-10] An information processing device comprising: an acquisition unit that acquires user behavior data generated in accordance with the user's movements; a determination unit that determines control parameters for changing the modal of a target device installed in a space where a user other than the first user is present, based on the user's behavior data; and an output unit that changes the modal of the target device using the control parameters. According to this information processing device, the movements of the first user can be visualized through modal changes of the device on the second user's side. This makes it possible to convey the situation of the first user to the second user even when the first user and the second user are in different locations, and also provides an opportunity for communication between the first user and the second user.

[0227] [Technology 4-1] An information processing method according to Technology 1, comprising: estimating an event in which the user is involved and the user's feelings toward that event based on the user's behavioral data; determining a message for the event in which the user is involved and the user's feelings toward that event as the control parameter; and notifying the user of the message by changing the modal in the target device. According to this information processing method, by notifying the first user of a message for an event in which the first user is involved and the first user's feelings toward that event, it is possible to support the first user and provide an opportunity for communication between the first user and other users (hereinafter referred to as "second users"). [Technology 4-2] An information processing method according to Technology 4-1, wherein the user's behavioral data includes data relating to communication between the user and other users. According to this information processing method, an event in which the first user is involved and the first user's feelings toward that event can be accurately estimated using data relating to communication between the first user and the second user. [Technical 4-3] The information processing method according to Technical 4-2, wherein the data relating to communication between the user and the other user includes conversation data between the user and the other user. According to this information processing method, using data relating to a conversation between a first user and a second user, it is possible to accurately estimate events in which the first user is involved and the first user's feelings towards those events. [Technical 4-4] The information processing method according to Technical 4-1, wherein the target device whose modal is to be changed is determined based on the relationship between the user and the other user with whom the user communicates. According to this information processing method, the target device whose modal is to be changed can be appropriately selected. [Technical 4-5] The information processing method according to Technical 4-1, wherein the timing for changing the modal of the target device is determined based on the communication situation between the user and the other user indicated by the user's behavior data. According to this information processing method, the modal of the device can be changed at an appropriate timing.[Technology 4-6] An information processing method according to Technology 4-1, wherein characteristic words related to events and emotions that have been registered in advance are extracted from the user's behavior data, and based on the extracted characteristic words, the event in which the user is involved and the user's emotion toward that event are estimated. According to this information processing method, the event in which the first user is involved and the first user's emotion toward that event can be estimated with high accuracy. [Technology 4-7] An information processing method according to Technology 4-1, wherein information indicating the estimated event in which the user is involved and the user's emotion toward that event is presented to other users, and messages regarding the event in which the user is involved and the user's emotion are obtained from other users. According to this information processing method, the event in which the first user is involved and the first user's emotion toward that event can be presented to the first user, and messages entered by the second user can be notified to the first user. [Technology 4-8] An information processing method according to Technology 4-1, wherein the message is notified to the user by changing the modal on the target device when the user is in a different location from other users. According to this information processing method, the modal of the device can be changed at the appropriate time, and a message can be provided to the first user at the appropriate time. [Technical 4-9] Information processing device comprising: an acquisition unit that acquires user behavior data; an estimation unit that estimates an event in which the user is involved and the user's feelings toward that event based on the user's behavior data; a determination unit that determines a message for the event in which the user is involved and the user's feelings toward that event; and an output unit that notifies the user of the message by changing the modal of the target device. According to this information processing device, by notifying the first user of a message for an event in which the first user is involved and the first user's feelings toward that event, it is possible to support the first user and provide an opportunity for communication between the first user and the second user.

[0228] [Technical 5-1] An information processing method according to Technical 1, wherein communication data between the user and other users is acquired as user behavior data; based on the communication data, it is detected whether the communication between the user and other users is in an excited state or a depressed state; if it is detected that the communication between the user and other users is in an excited state or a depressed state, at least a portion of the communication data is stored in a predetermined storage area as the control parameters; and after a predetermined time has elapsed since the acquisition of the communication data, the modal of the target device is changed using at least a portion of the communication data stored in the storage area. According to this information processing method, the modal of the target device is changed using at least a portion of the communication data when the communication between the first user and the second user is excited or depressed. This provides an opportunity for communication between the first user and the second user, and also facilitates communication between the first user and the second user. [Technical 5-2] The information processing method according to Technical 5-1, wherein the communication data between the user and the other user includes conversation data between the user and the other user. According to this information processing method, it is possible to accurately detect, using conversation data between the first user and the second user, whether the communication between the first user and the second user is in a state of high engagement or low engagement. [Technical 5-3] The information processing method according to Technical 5-1, further comprising acquiring data regarding the location where the communication between the user and the other user took place, and changing the modal on a target device installed at the location where the communication between the user and the other user took place.According to this information processing method, by changing the modal of a device in association with the location where communication between a certain user and another user took place, it becomes easier for the notified user to feel the change of a certain user more personally. [Technology 5-4] The information processing method according to Technology 5-1, wherein the target device whose modal is to be changed is determined based on the relationship between the user and the other user. According to this information processing method, the target device whose modal is to be changed can be appropriately selected. [Technology 5-5] The information processing method according to Technology 5-1, wherein the timing of changing the modal of the target device is determined based on the state of communication between the user and the other user. According to this information processing method, the modal of the device can be changed at an appropriate timing. [Technology 5-6] The information processing method according to Technology 5-1, wherein at least one of the following is repeatedly performed: determining whether the communication between the user and the other user is in an exciting state, and determining whether the communication between the user and the other user is in a quiet state, until the conversation in which the same characteristic word appears ends. According to this information processing method, the level of engagement in communication between the first user and the second user can be detected with high accuracy, and communication between the first user and the second user can be further promoted. [Technical 5-7] The information processing method according to Technical 5-1, wherein the audio of a conversation between the user and the other user is recognized, characteristic words that appear in excess of a predetermined standard within a predetermined time in the recognition result of the audio of the conversation are detected, and data to be stored in the memory area is determined based on the detected characteristic words. According to this information processing method, communication between the first user and the second user can be further promoted by using characteristic words in the conversation between the first user and the second user. [Technical 5-8] The information processing method according to Technical 5-7, wherein information of an event recommended to the user is searched based on the detected characteristic words, and when the modal in the target device is changed using at least a portion of the communication data, the information of the event is notified to the user. According to this information processing method, information of an event that matches the characteristic words can be provided to the first user, and communication between the first user and the second user can be further promoted.[Technical 5-9] An information processing method according to Technical 5-1, wherein the modal on the target device is changed using at least a portion of the communication data when the user is in a different location from the other user. This information processing method allows the modal of the device to be changed at an appropriate time. [Technical 5-10] An information processing method according to Technical 5-1, wherein data relating to the location where communication between the user and the other user took place is further acquired, and the modal on the target device installed at the location where communication between the user and the other user took place is changed. This information processing method makes it easier for the user who senses the modal change on the target device to recall the previous communication, thereby further promoting communication between users. [Technical 5-11] An information processing device comprising: an acquisition unit that acquires communication data between a user and another user; a detection unit that detects, based on the communication data, whether the communication between the user and the other user is in an excited state or a depressed state; a storage unit that, when it is detected that the communication between the user and the other user is in an excited state or a depressed state, stores at least a portion of the communication data in a predetermined storage area; and an output unit that, after a predetermined time has elapsed since the acquisition of the communication data, changes the modal of a target device using at least a portion of the communication data stored in the storage area. According to this information processing device, the modal of a target device is changed using at least a portion of the communication data when the communication between a first user and a second user is excited or depressed. This provides an opportunity for communication between the first user and the second user, and also facilitates communication between the first user and the second user.

[0229] The technology disclosed herein can be applied to information processing devices and information processing systems.

[0230] 10 Information processing system, 12 Sensor, 14 Target device, 16 Information processing device, 50 Behavioral data acquisition unit, 52 Change detection unit, 53 Behavioral information generation unit, 54 Control parameter determination unit, 56 Device control unit, 58 Estimation unit, 60 Message determination unit, 62 Communication data acquisition unit, 64 Communication state detection unit.

Claims

1. An information processing method in which a computer performs the following actions: acquires user behavior data, determines control parameters for changing the modal in a target device based on the user behavior data, and changes the modal in the target device using the control parameters.

2. The information processing method according to claim 1, comprising detecting a change in the user based on the user's behavior data, and determining the control parameters based on the change in the user.

3. The information processing method according to claim 1, wherein the user behavior data is generated in accordance with the user's movements, and control parameters for changing the modal in a target device installed in a space where a user other than the user exists are determined based on the user behavior data.

4. An information processing method according to claim 1, comprising: estimating an event in which the user is involved and the user's emotions regarding that event based on the user's behavioral data; determining a message for the event in which the user is involved and the user's emotions as the control parameters; and notifying the user of the message by changing the modal in the target device.

5. The information processing method according to claim 1, comprising: acquiring communication data between the user and other users as user behavior data; detecting, based on the communication data, whether the communication between the user and other users is in an active state or a subdued state; if it is detected that the communication between the user and other users is in an active state or a subdued state, storing at least a portion of the communication data as control parameters in a predetermined storage area; and changing the modal in the target device using at least a portion of the communication data stored in the storage area after a predetermined time has elapsed since the acquisition of the communication data.

6. The information processing method according to claim 2 or 4, wherein the user's behavioral data includes data relating to communication between the user and other users.

7. The information processing method according to claim 2 or 5, further comprising acquiring data relating to the location where communication between the user and other users took place, and changing the modal on a target device installed at the location where communication between the user and other users took place.

8. The information processing method according to any one of claims 2, 4, or 5, which determines the target device for which the modal is changed based on the relationship between the user and other users who communicate with the user.

9. The information processing method according to any one of claims 2, 4, or 5, wherein the timing for changing the modal of the target device is determined based on the status of communication between the user and other users as indicated by the user's behavioral data.

10. An information processing device comprising: an acquisition unit for acquiring user behavior data; a determination unit for determining control parameters for changing the modal in a target device based on the user behavior data; and a device control unit for changing the modal in the target device using the control parameters.

11. An information processing device comprising: an acquisition unit for acquiring user behavior data; a detection unit for detecting changes in the user based on the user behavior data; a determination unit for determining control parameters for changing the modal in a target device based on the changes in the user; and a device control unit for changing the modal in the target device using the control parameters.

12. An information processing device comprising: an acquisition unit that acquires user behavior data generated in accordance with the user's movements; a determination unit that determines control parameters for changing the modal in a target device installed in a space where a user other than the user exists, based on the user's behavior data; and a device control unit that changes the modal in the target device using the control parameters.

13. An information processing device comprising: an acquisition unit for acquiring user behavior data; an estimation unit for estimating events in which the user is involved and the user's emotions regarding those events based on the user behavior data; a determination unit for determining a message regarding the events in which the user is involved and the user's emotions; and a device control unit for notifying the user of the message by changing the modal in the target device.

14. An information processing device comprising: an acquisition unit that acquires communication data between a user and other users; a detection unit that detects, based on the communication data, whether the communication between the user and other users is in an active state or in a subdued state; a storage unit that stores at least a portion of the communication data in a predetermined storage area when it is detected that the communication between the user and other users is in an active state or in a subdued state; and a device control unit that changes the modal in a target device using at least a portion of the communication data stored in the storage area after a predetermined time has elapsed since the acquisition of the communication data.

Images