System

Abstract

The purpose of the present embodiment is to implement a smart home that can be operated autonomously even if a user does not issue an instruction. A system according to an embodiment includes a collection unit, an analysis unit, a decision unit, and an execution unit. The collection unit collects the state of the user. The analysis unit analyzes the data collected by the collection unit. The decision unit determines an appropriate operation on the basis of the analysis result obtained by the analysis unit. The execution unit executes the operation determined by the decision unit.

Description

Technical Field

[0001] The technology disclosed herein relates to a system. Background Technology

[0002] Patent Document 1 discloses a personalized chatbot control method executed by at least one processor, the method comprising: receiving user speech; adding the user speech to a prompt containing instructions related to a chatbot role; encoding the prompt; and inputting the encoded prompt into a language model to generate chatbot speech in response to the user speech.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2022-180282 Summary of the Invention

[0004] In existing technologies, users need to issue commands to smartphones or smart speakers when operating smart home devices, which can be cumbersome.

[0005] The purpose of this technical solution is to realize a smart home that can operate autonomously even without user commands.

[0006] The system of this technical solution includes a collection unit, an analysis unit, a decision-making unit, and an execution unit. The collection unit collects user status data. The analysis unit analyzes the data collected by the collection unit. The decision-making unit determines appropriate actions based on the analysis results obtained by the analysis unit. The execution unit executes the actions determined by the decision-making unit.

[0007] The system based on this technical solution can enable smart homes to operate autonomously even without user commands. Attached Figure Description

[0008] Figure 1 This is a conceptual diagram illustrating an example of the configuration of a data processing system according to the first embodiment.

[0009] Figure 2 This is a conceptual diagram illustrating an example of the main functions of the data processing apparatus and smart device according to the first embodiment.

[0010] Figure 3 This is a conceptual diagram illustrating an example of the data processing system configuration in the second embodiment.

[0011] Figure 4 This is a conceptual diagram illustrating an example of the main functions of the data processing device and smart glasses according to the second embodiment.

[0012] Figure 5 This is a conceptual diagram illustrating an example of the data processing system configuration in the third embodiment.

[0013] Figure 6This is a conceptual diagram illustrating an example of the main functions of the data processing device and head-mounted terminal according to the third embodiment.

[0014] Figure 7 This is a conceptual diagram illustrating an example of the data processing system configuration in the fourth embodiment.

[0015] Figure 8 This is a conceptual diagram illustrating an example of the functions of the main parts of the data processing device and robot according to the fourth embodiment.

[0016] Figure 9 It represents an emotion graph that maps multiple emotions.

[0017] Figure 10 It represents an emotion graph that maps multiple emotions.

[0018] Explanation of reference numerals in the attached figures

[0019] Data processing systems 10, 210, 310, and 410

[0020] 12 Data processing device

[0021] 14 Smart devices

[0022] 214 Smart Glasses

[0023] 314 Head-mounted terminal

[0024] 414 Robot Detailed Implementation

[0025] Hereinafter, an example of an implementation of the system involved in this disclosure will be described with reference to the accompanying drawings.

[0026] First, let's explain the terms used in the following description.

[0027] In the following embodiments, the processor (hereinafter referred to as "processor"), as indicated by the reference numerals, can be a single computing device or a combination of multiple computing devices. Furthermore, a processor can be a single computing device or a combination of multiple computing devices. Examples of computing devices include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit), etc.

[0028] In the following embodiments, RAM (Random Access Memory), as indicated in the figures, is a memory for temporary storage of information, which is used by the processor as working memory.

[0029] In the following embodiments, the memory, as indicated by the reference numerals, is one or more non-volatile storage devices used to store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), disk (e.g., hard disk), or magnetic tape, etc.

[0030] In the following embodiments, the communication I / F (Interface) with reference numerals is an interface including a communication processor and an antenna, etc. The communication I / F is responsible for communication between multiple computers. Examples of communication standards applicable to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0031] In the following implementation, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it can be only A, only B, or a combination of A and B. Furthermore, in this specification, when "and / or" connects more than three items, the same approach as "A and / or B" applies.

[0032] First Implementation Method

[0033] Figure 1 An example of the configuration of the data processing system 10 according to the first embodiment is shown.

[0034] like Figure 1 As shown, the data processing system 10 includes a data processing device 12 and an intelligent device 14. An example of the data processing device 12 is a server.

[0035] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. An example of the network 54 includes a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0036] The smart device 14 includes a computer 36, a receiver 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the receiver 38, output device 40, and camera 42 are also connected to the bus 52.

[0037] The receiving device 38 includes a touchscreen 38A and a microphone 38B, etc., for receiving user input. The touchscreen 38A receives user input generated by contact with an indicator (e.g., a pen or finger). The microphone 38B receives user input generated by sound by detecting the user's voice. The control unit 46A sends data representing user input received via the touchscreen 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, a specific processing unit 290 (see reference...) Figure 2 Get the data that represents the user input.

[0038] The output device 40 includes a display 40A and a speaker 40B, which present data to the user by outputting data in a user-perceptible format (e.g., sound and / or text). The display 40A displays visual information such as text and images according to instructions from the processor 46. The speaker 40B outputs sound according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0039] Communication I / F 44 is connected to network 54. Communication I / F 44 and 26 are responsible for sending and receiving various information between processor 46 and processor 28 via network 54.

[0040] Figure 2 An example of the main functions of the data processing device 12 and the smart device 14 is shown.

[0041] like Figure 2 As shown, in the data processing apparatus 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the memory 32. The specific processing program 56 is an example of a "program" as understood in this disclosure. The processor 28 reads the specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. Specific processing is implemented by the processor 28 operating as a specific processing unit 290 based on the specific processing program 56 executed on the RAM 30.

[0042] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. The emotion inference function using the emotion-specific model 59 (emotion-specific function) includes inferring and predicting the user's emotions, performing various inferences and predictions related to the user's emotions, but is not limited to this example. Furthermore, emotion inference and prediction also include, for example, emotion analysis (analysis).

[0043] In the smart device 14, specific processing is performed by the processor 46. A specific processing program 60 is stored in the memory 50. The specific processing program 60 is used in conjunction with the data processing system 10. The processor 46 reads the specific processing program 60 from the memory 50 and executes the read specific processing program 60 on the RAM 48. Specific processing is implemented by the processor 46 operating as a control unit 46A based on the specific processing program 60 executed on the RAM 48. Furthermore, the smart device 14 may also have the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and use these models to perform the same processing as the specific processing unit 290.

[0044] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. Furthermore, the data processing device 12 may be a server device or a user-owned terminal device (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of the processing performed by the data processing system 10 of the first embodiment will be described.

[0045] Implementation Method 1

[0046] The smart home system of this invention is a smart home system that does not require commands from a smart speaker equipped with a camera and generative AI. This smart home system collects the user's state through a camera mounted on a smart speaker, which is then analyzed and identified by the generative AI. Based on the analysis results, the generative AI determines the optimal action and can autonomously perform operations such as playing music. Thus, users can enjoy a comfortable environment without operating their smartphones or speaking to the speaker. Furthermore, it has the function of learning user behavior patterns and providing the optimal environment based on time periods and situations. In addition, to protect the privacy of the camera, it includes mechanisms for data encryption and obtaining user consent. For example, the camera mounted on the smart speaker collects the user's state. For example, the camera captures a user relaxing in the living room. This data is sent to the generative AI. The generative AI then analyzes the collected data and identifies the user's behavior. For example, if it determines that the user is relaxed, the generative AI selects music suitable for the situation. Based on the analysis results, the generative AI determines the optimal action. For example, it decides to play relaxing music for the relaxed user. The determined action is executed. For example, the smart speaker plays music. In this way, users can enjoy a comfortable environment without any operation. Furthermore, generative AI learns user behavior patterns and provides the optimal environment based on time of day and situation. For example, it plays relaxing music at night and easy-to-wake-up music in the morning. In addition, to protect camera privacy, mechanisms for data encryption and obtaining user consent are included. For example, collected data is encrypted and only parsed after obtaining user consent. Thus, the smart home system can provide hands-free smart living by automatically collecting, analyzing, deciding on, and executing user status.

[0047] The smart home system of this embodiment includes a collection unit, an analysis unit, a decision-making unit, and an execution unit. The collection unit collects the user's state. The user's state includes, but is not limited to, actions, facial expressions, and voice. For example, the collection unit can collect the user's actions via a camera. The collection unit can also collect the user's voice via a microphone. Furthermore, the collection unit can collect the user's facial expressions via sensors. For example, the collection unit can collect the user's actions in real time via a camera and send them to generative AI. The analysis unit uses generative AI to analyze the data collected by the collection unit. Analysis is based on, for example, data analysis algorithms or analysis precision, but is not limited to. For example, generative AI can use text generation AI (such as LLM) to analyze the data. The analysis unit can also use multimodal generation AI to analyze the data. The analysis unit can also use generative AI to extract important parts of the data and analyze them. For example, text generation AI has learned from a large amount of data and possesses advanced natural language processing capabilities. Multimodal generation AI can process not only text but also multiple modalities such as images and voice. Generative AI utilizes keyword extraction technology to select particularly important information from data and analyzes it accordingly. The decision-making unit determines the optimal action based on the analysis results obtained by the analysis unit. Actions may be based on things like appliance operation or notification sending, but are not limited to these. For example, the decision-making unit might decide to play relaxing music based on the analysis results. The decision-making unit can also adjust the lighting based on the analysis results. The decision-making unit can also adjust the temperature based on the analysis results. The execution unit executes the actions determined by the decision-making unit. Execution may be based on things like music playback or lighting adjustment, but is not limited to these. For example, the execution unit might play relaxing music determined by the decision-making unit. The execution unit can also adjust the lighting determined by the decision-making unit. The execution unit can also adjust the temperature determined by the decision-making unit. For example, the execution unit might play relaxing music determined by the decision-making unit. The execution unit can also adjust the lighting determined by the decision-making unit. The execution unit can also adjust the temperature determined by the decision-making unit. Therefore, the smart home system of this embodiment can provide a hands-free smart home by automatically collecting, analyzing, deciding and executing the user's status.

[0048] The parsing unit may include a learning unit for learning user behavior patterns. This learning unit, for example, learns user behavior patterns. Behavior patterns may include, but are not limited to, daily routines or specific events. The learning unit, for example, learns the user's behavior patterns and sends them to generative AI. By learning user behavior patterns, more appropriate actions can be provided. Some or all of the above processing in the learning unit can be performed by AI, or it can be performed without AI. For example, the learning unit can input the user's behavior patterns into the generative AI, and then have the generative AI perform the learning of the behavior patterns.

[0049] The collection unit may include an encryption unit for encrypting data collected via a camera. The encryption unit, for example, encrypts the data collected via a camera. Encryption may include, but is not limited to, the encryption algorithm or encryption strength used. The encryption unit, for example, encrypts the data collected via a camera and sends it to the generative AI. Privacy can be protected by encrypting the collected data. Some or all of the above processing in the encryption unit may be performed by AI, or it may not be performed by AI. For example, the encryption unit may input the data collected via a camera into the generative AI and have the generative AI perform data encryption.

[0050] The data collection unit may include a consent acquisition unit for obtaining user consent. The consent acquisition unit, for example, obtains the user's consent. Consent may include, but is not limited to, the form or scope of consent. The consent acquisition unit, for example, obtains the user's consent and sends it to a generative AI. Obtaining user consent ensures transparency in data collection. Some or all of the above-described processing in the consent acquisition unit may be performed by AI, or it may not be performed by AI. For example, the consent acquisition unit may input the user's consent into the generative AI and have the generative AI perform the consent acquisition.

[0051] The execution unit can play music. Music playback includes, but is not limited to, the type of music played or the timing of playback. For example, the execution unit can play relaxing music. It can also play upbeat music. Furthermore, the execution unit can play ambient sounds. By playing music, a comfortable environment can be provided for the user. Some or all of the above processing in the execution unit can be performed by AI, or it can be performed without AI. For example, the execution unit can input music playback information into generative AI and have the generative AI execute the music playback.

[0052] The execution unit can perform lighting adjustments. Lighting adjustments include, but are not limited to, brightness adjustments or color temperature changes. For example, the execution unit can adjust the lighting brightness. The execution unit can also change the lighting color temperature. Furthermore, the execution unit can control the switching of the lighting on and off. For example, the execution unit can adjust the lighting brightness to provide a relaxing environment. The execution unit can also change the lighting color temperature to provide an active environment. Additionally, the execution unit can control the switching of the lighting to provide an appropriate lighting environment. By performing lighting adjustments, an optimal environment can be provided for the user. Some or all of the above processing in the execution unit can be performed by AI, or it can be performed without AI. For example, the execution unit can input lighting adjustments into a generative AI, and have the generative AI perform the lighting adjustments.

[0053] The data collection department can analyze users' past behavior records and select the optimal data collection method. For example, it can analyze the time periods users spent relaxing in their living room and concentrate data collection during those periods. It can also analyze patterns of user activity in specific rooms and prioritize data collection in those rooms. Furthermore, by analyzing users' past behavior records, the department can collect data on specific days of the week or during specific time periods, achieving efficient data collection. Efficient data collection can be achieved by analyzing past behavior records. Some or all of the above processing in the collection department can be performed using AI, or it can be performed without AI. For example, the collection department can input users' past behavior records into generative AI and have the generative AI perform behavior record analysis.

[0054] The data collection department can filter data based on the user's current activity or environment. For example, when a user is watching TV in the living room, the department can reduce voice data collection and prioritize video data collection. Similarly, when a user is cooking in the kitchen, the department can prioritize voice data collection and wait for user instructions. Furthermore, the department can minimize data collection when a user is resting in the bedroom to protect privacy. Appropriate data collection can be performed by filtering data based on the user's current activity or environment. Some or all of the above processing in the collection department can be performed by AI, or it can be performed without AI. For example, the collection department can input the user's current activity or environmental data into generative AI, which can then perform data filtering.

[0055] When collecting data, the data collection department can consider the user's geographic location information and prioritize the collection of highly relevant data. For example, when the user is in the living room, the collection department can prioritize collecting environmental data for the living room. Similarly, when the user is in the kitchen, the collection department can prioritize collecting temperature and humidity data for the kitchen. Furthermore, when the user is in the bedroom, the collection department can prioritize collecting lighting and music data for the bedroom. By considering the user's geographic location information, highly relevant data can be prioritized. Some or all of the above processing in the collection department can be performed by AI, or it can be performed without AI. For example, the collection department can input the user's geographic location information into generative AI, which can then perform the collection of highly relevant data.

[0056] During data collection, the data collection department can analyze users' social media activities to gather relevant data. For example, when users post relaxed statuses on social media, the department can collect context-appropriate data. It can also collect context-appropriate data when users post about feeling stressed. Furthermore, it can collect context-appropriate data when users post active statuses on social media. By analyzing social media activity, relevant data can be collected efficiently. Some or all of the above processing in the collection department can be performed using AI, or it can be performed without AI. For example, the collection department can input users' social media activity data into generative AI, which can then perform the relevant data collection.

[0057] During parsing, the parsing unit can adjust the level of detail based on the importance of the data. For example, it can perform detailed parsing on highly important data, generating specific suggestions. It can also perform brief parsing on less important data, providing only a summary. Furthermore, it can perform moderately detailed parsing on moderately important data, providing balanced information. By adjusting the level of detail based on the data's importance, parsing can be performed efficiently. Some or all of the above processing in the parsing unit can be performed by AI, or it can be done without AI. For example, the parsing unit can input the importance of the data into a generative AI, which can then adjust the level of detail.

[0058] During data analysis, the parsing unit can apply different parsing algorithms based on the data category. For example, it can analyze music data based on genre and rhythm. It can also analyze lighting data based on color temperature and brightness. Furthermore, it can analyze environmental data based on temperature and humidity. By applying different parsing algorithms based on the data category, more appropriate parsing can be performed. Some or all of the above processing in the parsing unit can be performed by AI, or it can be performed without AI. For example, the parsing unit can input the data category into a generative AI, which can then execute the parsing algorithm.

[0059] During data analysis, the parsing unit can determine the analysis priority based on the timing of data collection. For example, it can prioritize analyzing the latest data to achieve real-time response. It can also refer to historical data to grasp long-term trends. Furthermore, it can prioritize analyzing data collected within a specific time period to provide a response appropriate for that timeframe. By prioritizing analysis based on the timing of data collection, real-time response can be achieved. Some or all of the above processing in the parsing unit can be performed by AI, or it can be performed without AI. For example, the parsing unit can input the data collection timing into a generative AI, which can then determine the analysis priority.

[0060] During parsing, the parsing unit can adjust the parsing order based on the relevance of the data. For example, it can prioritize parsing highly relevant data for rapid response. It can also postpone parsing less relevant data and prioritize important data. Furthermore, the parsing unit can dynamically adjust the parsing order based on data relevance for efficient parsing. By adjusting the parsing order based on data relevance, efficient parsing can be performed. Some or all of the above processing in the parsing unit can be performed by AI, or it can be done without AI. For example, the parsing unit can input the data relevance into a generative AI and have the generative AI adjust the parsing order.

[0061] When making action decisions, the decision-making unit can consider the interrelationships of the analytical results to improve decision accuracy. For example, the decision-making unit can coordinate music playback and lighting adjustment to provide an environment most suitable for the user's emotions. It can also coordinate ambient sound and temperature adjustment to maximize user comfort. Furthermore, it can coordinate lighting color temperature and music genre to provide an environment that matches the user's mood. By considering the interrelationships of the analytical results, decision accuracy can be improved. Some or all of the above processing in the decision-making unit can be performed by AI, or it can be performed without AI. For example, the decision-making unit can input the interrelationships of the analytical results into generative AI, thereby improving the accuracy of the generative AI's decision-making.

[0062] When making action decisions, the decision-making unit can consider user attribute information to determine the appropriate action. For example, it can select the appropriate music genre based on the user's age. It can also adjust the preferred lighting color temperature based on the user's gender. Furthermore, it can play appropriate ambient sounds based on the user's health status. By considering user attribute information, optimal actions can be provided. Some or all of the above processing in the decision-making unit can be performed by AI, or it can be performed without AI. For example, the decision-making unit can input user attribute information into generative AI and have the generative AI execute the action decision.

[0063] When making action decisions, the decision-making unit can consider the geographical distribution of data to determine the appropriate action. For example, if a user is in the living room, the decision-making unit can decide on an action suitable for the living room. Similarly, if a user is in the kitchen, the decision-making unit can decide on an action suitable for the kitchen. Furthermore, if a user is in the bedroom, the decision-making unit can decide on an action suitable for the bedroom. By considering the geographical distribution of data, optimal actions can be provided. Some or all of the above processing in the decision-making unit can be performed by AI, or it can be performed without AI. For example, the decision-making unit can input the geographical distribution of the data into generative AI and have the generative AI perform the action decision.

[0064] When making action decisions, the decision-making department can refer to relevant literature to improve decision-making accuracy. For example, it can refer to the latest research papers to select the optimal music type. It can also refer to literature on lighting adjustment to determine the optimal color temperature. Furthermore, it can refer to literature on ambient sound effects to select the optimal ambient sound. By referring to relevant literature, decision-making accuracy can be improved. Some or all of the above processing in the decision-making department can be performed by AI, or it can be performed without AI. For example, the decision-making department can input relevant literature into generative AI, allowing the generative AI to improve the accuracy of its decision-making.

[0065] During execution, the execution unit can consider the interrelationships of actions to improve execution accuracy. For example, the execution unit can coordinate music playback and lighting adjustment to provide an environment most suitable for the user's emotions. It can also coordinate ambient sound and temperature adjustment to maximize user comfort. Furthermore, it can coordinate lighting color temperature and music genre to provide an environment that matches the user's mood. By considering the interrelationships of actions, execution accuracy can be improved. Some or all of the above processing in the execution unit can be performed by AI, or it can be performed without AI. For example, the execution unit can input the interrelationships of actions into generative AI, thereby improving the execution accuracy of the generative AI.

[0066] During execution, the execution unit can consider user attribute information to customize actions. For example, it can select appropriate music genres based on the user's age. It can also adjust preferred lighting color temperature based on the user's gender. Furthermore, it can play appropriate ambient sounds based on the user's health status. By considering user attribute information, more appropriate actions can be provided. Some or all of the above processing in the execution unit can be performed by AI, or it can be performed without AI. For example, the execution unit can input user attribute information into generative AI and have the generative AI perform customized actions.

[0067] When executing actions, the execution unit can consider the geographical distribution of the data. For example, if the user is in the living room, the execution unit will execute actions suitable for the living room. Similarly, if the user is in the kitchen, the execution unit will execute actions suitable for the kitchen. Furthermore, if the user is in the bedroom, the execution unit will execute actions suitable for the bedroom. By considering the geographical distribution of the data, optimal actions can be provided. Some or all of the above processing in the execution unit can be performed by AI, or it can be performed without AI. For example, the execution unit can input the geographical distribution of the data into generative AI, and then have the generative AI execute the actions.

[0068] During execution, the execution unit can refer to relevant literature to improve accuracy. For example, it can consult the latest research papers to select the optimal music type. It can also refer to literature on lighting adjustment to determine the optimal color temperature. Furthermore, it can refer to literature on ambient sound effects to select the optimal ambient sound. By referring to relevant literature, execution accuracy can be improved. Some or all of the above processing in the execution unit can be performed by AI, or it can be performed without AI. For example, the execution unit can input relevant literature into generative AI, thereby improving the execution accuracy of the generative AI.

[0069] During learning, the learning department can refer to past learning data to optimize the learning algorithm. For example, the learning department can select the optimal learning algorithm based on past learning data. The learning department can also analyze past learning data and adjust the parameters of the learning algorithm. Furthermore, the learning department can refer to past learning data to improve the accuracy of the learning algorithm. By referring to past learning data, the accuracy of the learning algorithm can be improved. Some or all of the above processes in the learning department can be performed by AI, or they can be performed without AI. For example, the learning department can input past learning data into generative AI and have the generative AI perform the learning algorithm optimization.

[0070] During the learning process, the learning department can weight the learning data based on the timing of its collection. For example, it can assign higher weights to the most recent data for real-time response, and lower weights to past data to grasp long-term trends. Furthermore, it can assign appropriate weights to data collected within specific time periods for responses tailored to those periods. By weighting the learning data according to its collection timing, real-time response can be achieved. Some or all of the above processing within the learning department can be performed using AI, or it can be done without AI. For instance, the learning department can input the data collection timing into generative AI, which can then perform the weighting of the learning data.

[0071] During encryption, the encryption department can adjust the level of detail based on the importance of the data. For example, it can perform detailed encryption on highly important data to ensure data security. It can also perform simplified encryption on less important data to improve processing speed. Furthermore, it can encrypt moderately important data with appropriate detail to achieve balanced data protection. By adjusting the level of detail based on the importance of the data, data protection can be performed efficiently. Some or all of the above processing in the encryption department can be performed by AI, or it can be performed without AI. For example, the encryption department can input the importance of the data into a generative AI, which will then adjust the encryption detail.

[0072] During encryption, the encryption department can adjust the encryption order based on the timing of data collection. For example, it can prioritize encrypting the latest data for real-time response. It can also postpone encrypting past data, prioritizing important data. Furthermore, it can prioritize encrypting data collected within a specific time period for a response appropriate to that timeframe. By adjusting the encryption order based on the timing of data collection, real-time response can be achieved. Some or all of the above processing in the encryption department can be performed using AI, or it can be performed without AI. For example, the encryption department can input the data collection timing into a generative AI, which will then adjust the encryption order.

[0073] When obtaining consent, the consent acquisition department can refer to the user's past consent records to select the optimal consent acquisition method. For example, the consent acquisition department can select the optimal consent acquisition method based on the user's past consent methods. The consent acquisition department can also analyze the user's past consent records to optimize the consent acquisition process. Furthermore, the consent acquisition department can adjust the consent acquisition frequency by referring to the user's past consent records. By referring to the user's past consent records, the optimal consent acquisition method can be selected. Some or all of the above processing in the consent acquisition department can be performed by AI, or it can be performed without AI. For example, the consent acquisition department can input the user's past consent records into a generative AI and have the generative AI perform the consent acquisition method selection.

[0074] When obtaining consent, the consent acquisition department can select an appropriate consent acquisition method based on the user's device information. For example, when a user is using a smartphone, the consent acquisition department can provide a consent acquisition method suitable for the screen size. When a user is using a tablet, the consent acquisition department can also provide a consent acquisition method optimized for larger screens. Furthermore, when a user is using a smartwatch, the consent acquisition department can provide a simple and highly visual consent acquisition method. By considering the user's device information, the optimal consent acquisition method can be selected. Some or all of the above processing in the consent acquisition department can be performed by AI, or it can be performed without AI. For example, the consent acquisition department can input the user's device information into generative AI and have the generative AI perform the consent acquisition method selection.

[0075] The system described in this embodiment is not limited to the examples above. For example, various modifications can be made as follows.

[0076] The collection unit may include a health data collection unit for gathering user health data. This unit may collect vital signs such as heart rate, blood pressure, and body temperature. It may also collect data on user activity levels or sleep patterns. Furthermore, it may collect data on user diet or calorie intake. Based on the collected health data, the user's health status can be assessed and appropriate measures taken. Some or all of the above processing in the health data collection unit may be performed using AI, or it may not. For example, the health data collection unit can input the collected health data into generative AI, which can then perform health status analysis.

[0077] The analysis unit may include an interest preference learning unit to learn user interests and preferences. For example, the interest preference learning unit might learn a user's favorite music genres or movie types. It could also learn a user's favorite foods or drinks. Furthermore, it could learn a user's favorite travel destinations or activities. By basing its work on user interests and preferences, more personalized services can be provided. Some or all of the above processing in the interest preference learning unit can be performed using AI, or it can be performed without AI. For example, the interest preference learning unit can input user interest preference data into generative AI, which can then perform interest preference learning.

[0078] The encryption department can apply different encryption algorithms depending on the type of data. For example, advanced encryption algorithms can be used for personal information data, while lightweight encryption algorithms can be used for general environmental data. The encryption department can also apply dedicated voice encryption algorithms to voice data and dedicated image encryption algorithms to image data. Furthermore, the encryption department can apply dedicated text encryption algorithms to text data. By using optimal encryption based on the type of data, efficient data protection can be achieved. Some or all of the above processing in the encryption department can be performed by AI, or it can be performed without AI. For example, the encryption department can input the type of data into a generative AI and have the generative AI execute the encryption algorithm application.

[0079] When obtaining user consent, the consent acquisition department can refer to the user's past consent records to select the optimal consent acquisition method. For example, it can select the optimal consent acquisition method based on the user's past consent methods. The consent acquisition department can also analyze the user's past consent records to optimize the consent acquisition process. In addition, the consent acquisition department can also refer to the user's past consent records to adjust the consent acquisition frequency. By referring to the user's past consent records, the optimal consent acquisition method can be selected. Some or all of the above processing in the consent acquisition department can be performed by AI, or it can be performed without AI. For example, the consent acquisition department can input the user's past consent records into a generative AI and have the generative AI perform the consent acquisition method selection.

[0080] The execution unit can perform actions based on the user's health status. For example, it can play relaxing music when the user's heart rate is high, or lower the room temperature when the user's body temperature is high. Furthermore, it can provide appropriate lighting or music based on the user's sleep patterns. By basing actions on the user's health status, optimal actions can be provided. Some or all of the above processing in the execution unit can be performed by AI, or it can be performed without AI. For example, the execution unit can input the user's health data into generative AI and have the generative AI execute the actions.

[0081] The following is a brief description of the processing flow of Implementation Method 1.

[0082] Step 1: The data collection unit gathers the user's state. The user's state includes actions, facial expressions, and voice. The collection unit can collect the user's actions through a camera, voice through a microphone, and facial expressions through sensors. The collected data is then sent to the generative AI.

[0083] Step 2: The parsing department uses generative AI to analyze the data collected by the collection department. The parsing is based on data parsing algorithms or parsing precision. Generative AI uses text generation AI (such as LLM) or multimodal generation AI to parse the data and extract important parts for analysis.

[0084] Step 3: The decision-making department determines the optimal action based on the analysis results obtained from the analysis department. The action is determined based on factors such as appliance operation or notification sending. Examples include playing relaxing music, adjusting lighting, and setting the temperature.

[0085] Step 4: The execution department carries out the actions determined by the decision-making department. Execution is based on factors such as music playback, lighting adjustment, and temperature control. For example, this includes playing relaxing music, adjusting lighting, and adjusting the temperature.

[0086] Implementation Method 2

[0087] The smart home system of this invention is a smart home system that does not require commands from a smart speaker equipped with a camera and generative AI. This smart home system collects the user's state through a camera mounted on a smart speaker, which is then analyzed and identified by the generative AI. Based on the analysis results, the generative AI determines the optimal action and can autonomously perform operations such as playing music. Thus, users can enjoy a comfortable environment without operating their smartphones or speaking to the speaker. Furthermore, it has the function of learning user behavior patterns and providing the optimal environment based on time periods and situations. In addition, to protect the privacy of the camera, it includes mechanisms for data encryption and obtaining user consent. For example, the camera mounted on the smart speaker collects the user's state. For example, the camera captures a user relaxing in the living room. This data is sent to the generative AI. The generative AI then analyzes the collected data and identifies the user's behavior. For example, if it determines that the user is relaxed, the generative AI selects music suitable for the situation. Based on the analysis results, the generative AI determines the optimal action. For example, it decides to play relaxing music for the relaxed user. The determined action is executed. For example, the smart speaker plays music. In this way, users can enjoy a comfortable environment without any operation. Furthermore, generative AI learns user behavior patterns and provides the optimal environment based on time of day and situation. For example, it plays relaxing music at night and easy-to-wake-up music in the morning. In addition, to protect camera privacy, mechanisms for data encryption and obtaining user consent are included. For example, collected data is encrypted and only parsed after obtaining user consent. Thus, the smart home system can provide hands-free smart living by automatically collecting, analyzing, deciding on, and executing user status.

[0088] The smart home system of this embodiment includes a collection unit, an analysis unit, a decision-making unit, and an execution unit. The collection unit collects the user's state. The user's state includes, but is not limited to, actions, facial expressions, and voice. For example, the collection unit can collect the user's actions via a camera. The collection unit can also collect the user's voice via a microphone. Furthermore, the collection unit can collect the user's facial expressions via sensors. For example, the collection unit can collect the user's actions in real time via a camera and send them to generative AI. The analysis unit uses generative AI to analyze the data collected by the collection unit. Analysis is based on, for example, data analysis algorithms or analysis precision, but is not limited to. For example, generative AI can use text generation AI (such as LLM) to analyze the data. The analysis unit can also use multimodal generation AI to analyze the data. The analysis unit can also use generative AI to extract important parts of the data and analyze them. For example, text generation AI has learned from a large amount of data and possesses advanced natural language processing capabilities. Multimodal generation AI can process not only text but also multiple modalities such as images and voice. Generative AI utilizes keyword extraction technology to select particularly important information from data and analyzes it accordingly. The decision-making unit determines the optimal action based on the analysis results obtained by the analysis unit. Actions may be based on things like appliance operation or notification sending, but are not limited to these. For example, the decision-making unit might decide to play relaxing music based on the analysis results. The decision-making unit can also adjust the lighting based on the analysis results. The decision-making unit can also adjust the temperature based on the analysis results. The execution unit executes the actions determined by the decision-making unit. Execution may be based on things like music playback or lighting adjustment, but is not limited to these. For example, the execution unit might play relaxing music determined by the decision-making unit. The execution unit can also adjust the lighting determined by the decision-making unit. The execution unit can also adjust the temperature determined by the decision-making unit. For example, the execution unit might play relaxing music determined by the decision-making unit. The execution unit can also adjust the lighting determined by the decision-making unit. The execution unit can also adjust the temperature determined by the decision-making unit. Therefore, the smart home system of this embodiment can provide a hands-free smart home by automatically collecting, analyzing, deciding and executing the user's status.

[0089] The parsing unit may include a learning unit for learning user behavior patterns. This learning unit, for example, learns user behavior patterns. Behavior patterns may include, but are not limited to, daily routines or specific events. The learning unit, for example, learns user behavior patterns and sends them to generative AI. The learning unit may also learn user behavior patterns and send them to generative AI. Furthermore, the learning unit may learn user behavior patterns and send them to generative AI. By learning user behavior patterns, more appropriate actions can be provided. Some or all of the above processing in the learning unit can be performed by AI, or it may not be performed by AI. For example, the learning unit can input user behavior patterns into generative AI and cause the generative AI to perform behavior pattern learning.

[0090] The collection unit may include an encryption unit for encrypting data collected via a camera. The encryption unit, for example, encrypts the data collected via a camera. Encryption includes, but is not limited to, the encryption algorithm or encryption strength used. The encryption unit, for example, encrypts the data collected via a camera and sends it to the generative AI. The encryption unit may also encrypt the data collected via a camera and send it to the generative AI. Furthermore, the encryption unit may also encrypt the data collected via a camera and send it to the generative AI. By encrypting the collected data, privacy can be protected. Some or all of the above processing in the encryption unit may be performed by AI, or it may not be performed by AI. For example, the encryption unit may input the data collected via a camera into the generative AI and have the generative AI perform data encryption.

[0091] The data collection unit may include a consent acquisition unit for obtaining user consent. This consent acquisition unit, for example, obtains the user's consent. Consent may include, but is not limited to, the form or scope of consent. The consent acquisition unit may, for example, obtain user consent and send it to a generative AI. Furthermore, the consent acquisition unit may also obtain user consent and send it to a generative AI. Obtaining user consent ensures transparency in data collection. Some or all of the above-described processing in the consent acquisition unit may be performed by AI, or it may not be performed by AI. For example, the consent acquisition unit may input user consent into the generative AI and have the generative AI perform the consent acquisition.

[0092] The execution unit can play music. Music playback includes, but is not limited to, the type of music played or the timing of playback. For example, the execution unit can play relaxing music. It can also play upbeat music. Furthermore, the execution unit can play ambient sounds. By playing music, a comfortable environment can be provided for the user. Some or all of the above processing in the execution unit can be performed by AI, or it can be performed without AI. For example, the execution unit can input music playback information into generative AI and have the generative AI execute the music playback.

[0093] The execution unit can perform lighting adjustments. Lighting adjustments include, but are not limited to, brightness adjustments or color temperature changes. For example, the execution unit can adjust the lighting brightness. The execution unit can also change the lighting color temperature. Furthermore, the execution unit can control the switching of the lighting on and off. For example, the execution unit can adjust the lighting brightness to provide a relaxing environment. The execution unit can also change the lighting color temperature to provide an active environment. Additionally, the execution unit can control the switching of the lighting to provide an appropriate lighting environment. By performing lighting adjustments, an optimal environment can be provided for the user. Some or all of the above processing in the execution unit can be performed by AI, or it can be performed without AI. For example, the execution unit can input lighting adjustments into a generative AI, and have the generative AI perform the lighting adjustments.

[0094] The data collection department can infer users' emotions and adjust the timing of data collection based on these inferred emotions. For example, when a user is presumed to be relaxed, the data collection frequency can be reduced to respect user privacy. When a user is presumed to be stressed, the data collection frequency can be increased to respond quickly and appropriately. Furthermore, when a user is presumed to be active, the data collection interval can be shortened to achieve real-time response. By adjusting the timing of data collection based on user emotions, appropriate data collection can be conducted while respecting privacy. Emotion inference can be achieved, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the collection department can be performed by AI, or it can be performed without AI. For example, the collection department can input users' emotional data into generative AI and have the generative AI perform emotion inference.

[0095] The data collection department can analyze users' past behavior records and select the optimal data collection method. For example, it can analyze the time periods users spent relaxing in their living room and concentrate data collection during those periods. It can also analyze patterns of user activity in specific rooms and prioritize data collection in those rooms. Furthermore, by analyzing users' past behavior records, the department can collect data on specific days of the week or during specific time periods, achieving efficient data collection. Efficient data collection can be achieved by analyzing past behavior records. Some or all of the above processing in the collection department can be performed using AI, or it can be performed without AI. For example, the collection department can input users' past behavior records into generative AI and have the generative AI perform behavior record analysis.

[0096] The data collection department can filter data based on the user's current activity or environment. For example, when a user is watching TV in the living room, the department can reduce voice data collection and prioritize video data collection. Similarly, when a user is cooking in the kitchen, the department can prioritize voice data collection and wait for user instructions. Furthermore, the department can minimize data collection when a user is resting in the bedroom to protect privacy. Appropriate data collection can be performed by filtering data based on the user's current activity or environment. Some or all of the above processing in the collection department can be performed by AI, or it can be performed without AI. For example, the collection department can input the user's current activity or environmental data into generative AI, which can then perform data filtering.

[0097] The data collection unit can infer a user's emotions and prioritize data collection based on these inferences. For example, when a user is presumed to be relaxed, music and lighting data can be collected first. When a user is presumed to be stressed, ambient sound and temperature data can be collected first. Furthermore, when a user is presumed to be active, activity level and heart rate data can be collected first. By prioritizing data based on user emotions, more appropriate data collection can be performed. Emotion inference can be achieved, for example, through emotion engines or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the collection unit can be performed by AI, or it can be performed without AI. For example, the collection unit can input the user's emotional data into generative AI and have the generative AI perform emotion inference.

[0098] When collecting data, the data collection department can consider the user's geographic location information and prioritize the collection of highly relevant data. For example, when the user is in the living room, the collection department can prioritize collecting environmental data for the living room. Similarly, when the user is in the kitchen, the collection department can prioritize collecting temperature and humidity data for the kitchen. Furthermore, when the user is in the bedroom, the collection department can prioritize collecting lighting and music data for the bedroom. By considering the user's geographic location information, highly relevant data can be prioritized. Some or all of the above processing in the collection department can be performed by AI, or it can be performed without AI. For example, the collection department can input the user's geographic location information into generative AI, which can then perform the collection of highly relevant data.

[0099] During data collection, the data collection department can analyze users' social media activities to gather relevant data. For example, when users post relaxed statuses on social media, the department can collect context-appropriate data. It can also collect context-appropriate data when users post about feeling stressed. Furthermore, it can collect context-appropriate data when users post active statuses on social media. By analyzing social media activity, relevant data can be collected efficiently. Some or all of the above processing in the collection department can be performed using AI, or it can be performed without AI. For example, the collection department can input users' social media activity data into generative AI, which can then perform the relevant data collection.

[0100] The analysis unit can infer the user's emotions and adjust the presentation of the analysis based on the inferred emotions. For example, when the analysis unit infers that the user is relaxed, it displays the analysis results in a visually gentle color scheme. When the analysis unit infers that the user is stressed, it displays the analysis results in a concise and highly visual format. Furthermore, when the analysis unit infers that the user is active, it displays the analysis results using dynamic charts or animations. By adjusting the analysis presentation according to the user's emotions, more appropriate analysis results can be provided. Emotion inference can be achieved, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text generation AI (such as LLM) or multimodal generation AI, but is not limited to these. Some or all of the above processing in the analysis unit can be performed by AI, or it can be performed without AI. For example, the analysis unit can input the user's emotion data into the generative AI and have the generative AI perform emotion inference.

[0101] During parsing, the parsing unit can adjust the level of detail based on the importance of the data. For example, it can perform detailed parsing on highly important data, generating specific suggestions. It can also perform brief parsing on less important data, providing only a summary. Furthermore, it can perform moderately detailed parsing on moderately important data, providing balanced information. By adjusting the level of detail based on the data's importance, parsing can be performed efficiently. Some or all of the above processing in the parsing unit can be performed by AI, or it can be done without AI. For example, the parsing unit can input the importance of the data into a generative AI, which can then adjust the level of detail.

[0102] During data analysis, the parsing unit can apply different parsing algorithms based on the data category. For example, it can analyze music data based on genre and rhythm. It can also analyze lighting data based on color temperature and brightness. Furthermore, it can analyze environmental data based on temperature and humidity. By applying different parsing algorithms based on the data category, more appropriate parsing can be performed. Some or all of the above processing in the parsing unit can be performed by AI, or it can be performed without AI. For example, the parsing unit can input the data category into a generative AI, which can then execute the parsing algorithm.

[0103] The parsing unit can infer the user's emotions and adjust the length of the parsing based on the inferred emotions. For example, when the parsing unit infers the user to be relaxed, it provides detailed parsing results. When the parsing unit infers the user to be stressed, it provides concise parsing results. Furthermore, when the parsing unit infers the user to be active, it provides concise parsing results highlighting key points. By adjusting the parsing length according to the user's emotions, more appropriate parsing results can be provided. Emotion inference can be achieved, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text generation AI (such as LLM) or multimodal generation AI, but is not limited to these. Some or all of the above processing in the parsing unit can be performed by AI, or it can be performed without AI. For example, the parsing unit can input the user's emotion data into the generative AI and have the generative AI perform emotion inference.

[0104] During data analysis, the parsing unit can determine the analysis priority based on the timing of data collection. For example, it can prioritize analyzing the latest data to achieve real-time response. It can also refer to historical data to grasp long-term trends. Furthermore, it can prioritize analyzing data collected within a specific time period to provide a response appropriate for that timeframe. By prioritizing analysis based on the timing of data collection, real-time response can be achieved. Some or all of the above processing in the parsing unit can be performed by AI, or it can be performed without AI. For example, the parsing unit can input the data collection timing into a generative AI, which can then determine the analysis priority.

[0105] During parsing, the parsing unit can adjust the parsing order based on the relevance of the data. For example, it can prioritize parsing highly relevant data for rapid response. It can also postpone parsing less relevant data and prioritize important data. Furthermore, the parsing unit can dynamically adjust the parsing order based on data relevance for efficient parsing. By adjusting the parsing order based on data relevance, efficient parsing can be performed. Some or all of the above processing in the parsing unit can be performed by AI, or it can be done without AI. For example, the parsing unit can input the data relevance into a generative AI and have the generative AI adjust the parsing order.

[0106] The decision-making unit can infer a user's emotions and adjust the criteria for action decisions based on these inferred emotions. For example, when the decision-making unit infers that the user is relaxed, it can play relaxing music. When the decision-making unit infers that the user is stressed, it can play stress-relieving ambient sounds. Furthermore, when the decision-making unit infers that the user is active, it can play upbeat music that supports the activity. By adjusting the action decision criteria based on the user's emotions, more appropriate actions can be provided. Emotion inference can be achieved, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the decision-making unit can be performed by AI, or it can be performed without AI. For example, the decision-making unit can input the user's emotion data into the generative AI and have the generative AI perform emotion inference.

[0107] When making action decisions, the decision-making unit can consider the interrelationships of the analytical results to improve decision accuracy. For example, the decision-making unit can coordinate music playback and lighting adjustment to provide an environment most suitable for the user's emotions. It can also coordinate ambient sound and temperature adjustment to maximize user comfort. Furthermore, it can coordinate lighting color temperature and music genre to provide an environment that matches the user's mood. By considering the interrelationships of the analytical results, decision accuracy can be improved. Some or all of the above processing in the decision-making unit can be performed by AI, or it can be performed without AI. For example, the decision-making unit can input the interrelationships of the analytical results into generative AI, thereby improving the accuracy of the generative AI's decision-making.

[0108] When making action decisions, the decision-making unit can consider user attribute information to determine the appropriate action. For example, it can select the appropriate music genre based on the user's age. It can also adjust the preferred lighting color temperature based on the user's gender. Furthermore, it can play appropriate ambient sounds based on the user's health status. By considering user attribute information, optimal actions can be provided. Some or all of the above processing in the decision-making unit can be performed by AI, or it can be performed without AI. For example, the decision-making unit can input user attribute information into generative AI and have the generative AI execute the action decision.

[0109] The decision-making unit can infer a user's emotions and adjust the display order of action decision results based on the inferred emotions. For example, if the decision-making unit infers that the user is relaxed, it will prioritize displaying relaxation actions. If the decision-making unit infers that the user is stressed, it will prioritize displaying stress-relieving actions. Furthermore, if the decision-making unit infers that the user is active, it will prioritize displaying actions that support the activity. By adjusting the display order of action decision results according to the user's emotions, more appropriate actions can be provided. Emotion inference can be achieved, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the decision-making unit can be performed by AI, or it can be performed without AI. For example, the decision-making unit can input the user's emotion data into the generative AI and have the generative AI perform emotion inference.

[0110] When making action decisions, the decision-making unit can consider the geographical distribution of data to determine the appropriate action. For example, if a user is in the living room, the decision-making unit can decide on an action suitable for the living room. Similarly, if a user is in the kitchen, the decision-making unit can decide on an action suitable for the kitchen. Furthermore, if a user is in the bedroom, the decision-making unit can decide on an action suitable for the bedroom. By considering the geographical distribution of data, optimal actions can be provided. Some or all of the above processing in the decision-making unit can be performed by AI, or it can be performed without AI. For example, the decision-making unit can input the geographical distribution of the data into generative AI and have the generative AI perform the action decision.

[0111] When making action decisions, the decision-making department can refer to relevant literature to improve decision-making accuracy. For example, it can refer to the latest research papers to select the optimal music type. It can also refer to literature on lighting adjustment to determine the optimal color temperature. Furthermore, it can refer to literature on ambient sound effects to select the optimal ambient sound. By referring to relevant literature, decision-making accuracy can be improved. Some or all of the above processing in the decision-making department can be performed by AI, or it can be performed without AI. For example, the decision-making department can input relevant literature into generative AI, allowing the generative AI to improve the accuracy of its decision-making.

[0112] The execution unit can infer the user's emotions and determine the priority of actions based on these inferred emotions. For example, if the execution unit infers that the user is relaxed, it will prioritize playing relaxing music. If the execution unit infers that the user is stressed, it will prioritize playing stress-relieving ambient sounds. Furthermore, if the execution unit infers that the user is active, it will prioritize playing music that supports the activity. By prioritizing actions based on the user's emotions, more appropriate actions can be provided. Emotion inference can be achieved, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the execution unit can be performed by AI, or it can be performed without AI. For example, the execution unit can input the user's emotion data into the generative AI and have the generative AI perform emotion inference.

[0113] During execution, the execution unit can consider the interrelationships of actions to improve execution accuracy. For example, the execution unit can coordinate music playback and lighting adjustment to provide an environment most suitable for the user's emotions. It can also coordinate ambient sound and temperature adjustment to maximize user comfort. Furthermore, it can coordinate lighting color temperature and music genre to provide an environment that matches the user's mood. By considering the interrelationships of actions, execution accuracy can be improved. Some or all of the above processing in the execution unit can be performed by AI, or it can be performed without AI. For example, the execution unit can input the interrelationships of actions into generative AI, thereby improving the execution accuracy of the generative AI.

[0114] During execution, the execution unit can consider user attribute information to customize actions. For example, it can select appropriate music genres based on the user's age. It can also adjust preferred lighting color temperature based on the user's gender. Furthermore, it can play appropriate ambient sounds based on the user's health status. By considering user attribute information, more appropriate actions can be provided. Some or all of the above processing in the execution unit can be performed by AI, or it can be performed without AI. For example, the execution unit can input user attribute information into generative AI and have the generative AI perform customized actions.

[0115] The execution unit can infer the user's emotions and adjust the display of actions based on these inferences. For example, when the user is presumed to be relaxed, the execution unit displays actions in a gentle color scheme. When the user is presumed to be stressed, the execution unit displays actions in a concise and highly visual format. Furthermore, when the user is presumed to be active, the execution unit can display actions using dynamic charts or animations. By adjusting the action display based on the user's emotions, more appropriate actions can be provided. Emotion inference can be achieved, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the execution unit can be performed by AI, or it can be performed without AI. For example, the execution unit can input the user's emotion data into the generative AI and have the generative AI perform emotion inference.

[0116] When executing actions, the execution unit can consider the geographical distribution of the data. For example, if the user is in the living room, the execution unit will execute actions suitable for the living room. Similarly, if the user is in the kitchen, the execution unit will execute actions suitable for the kitchen. Furthermore, if the user is in the bedroom, the execution unit will execute actions suitable for the bedroom. By considering the geographical distribution of the data, optimal actions can be provided. Some or all of the above processing in the execution unit can be performed by AI, or it can be performed without AI. For example, the execution unit can input the geographical distribution of the data into generative AI, and then have the generative AI execute the actions.

[0117] During execution, the execution unit can refer to relevant literature to improve accuracy. For example, it can consult the latest research papers to select the optimal music type. It can also refer to literature on lighting adjustment to determine the optimal color temperature. Furthermore, it can refer to literature on ambient sound effects to select the optimal ambient sound. By referring to relevant literature, execution accuracy can be improved. Some or all of the above processing in the execution unit can be performed by AI, or it can be performed without AI. For example, the execution unit can input relevant literature into generative AI, thereby improving the execution accuracy of the generative AI.

[0118] The learning unit can infer a user's emotions and select learning data based on these inferred emotions. For example, if the user is presumed to be relaxed, learning data suitable for that situation can be selected. If the user is presumed to be stressed, learning data that helps alleviate stress can be selected. Furthermore, if the user is presumed to be active, learning data that supports the activity can be selected. By selecting learning data based on user emotions, more appropriate learning can be achieved. Emotion inference can be implemented, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the learning unit can be performed by AI, or it can be performed without AI. For example, the learning unit can input the user's emotion data into generative AI and have the generative AI perform emotion inference.

[0119] During learning, the learning department can refer to past learning data to optimize the learning algorithm. For example, the learning department can select the optimal learning algorithm based on past learning data. The learning department can also analyze past learning data and adjust the parameters of the learning algorithm. Furthermore, the learning department can refer to past learning data to improve the accuracy of the learning algorithm. By referring to past learning data, the accuracy of the learning algorithm can be improved. Some or all of the above processes in the learning department can be performed by AI, or they can be performed without AI. For example, the learning department can input past learning data into generative AI and have the generative AI perform the learning algorithm optimization.

[0120] The learning unit can infer a user's emotions and adjust the learning frequency based on these inferences. For example, when the learning unit infers a user to be relaxed, it can reduce the learning frequency to respect user privacy. When the learning unit infers a user to be stressed, it can increase the learning frequency to respond quickly and appropriately. Furthermore, when the learning unit infers a user to be active, it can shorten the learning frequency to achieve real-time response. By adjusting the learning frequency according to the user's emotions, more appropriate learning can be achieved. Emotion inference can be achieved, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the learning unit can be performed by AI, or it can be performed without AI. For example, the learning unit can input the user's emotion data into the generative AI and have the generative AI perform emotion inference.

[0121] During the learning process, the learning department can weight the learning data based on the timing of its collection. For example, it can assign higher weights to the most recent data for real-time response, and lower weights to past data to grasp long-term trends. Furthermore, it can assign appropriate weights to data collected within specific time periods for responses tailored to those periods. By weighting the learning data according to its collection timing, real-time response can be achieved. Some or all of the above processing within the learning department can be performed using AI, or it can be done without AI. For instance, the learning department can input the data collection timing into generative AI, which can then perform the weighting of the learning data.

[0122] The encryption unit can infer a user's emotions and adjust the encryption strength based on the inferred emotions. For example, it might use standard encryption strength when the user is presumed to be relaxed. It could also use high encryption strength when the user is presumed to be stressed, ensuring data security. Furthermore, it could use moderate encryption strength when the user is presumed to be active, maintaining data processing speed. By adjusting the encryption strength according to the user's emotions, data security can be ensured. Emotion inference can be achieved, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the encryption unit can be performed by AI, or it can be performed without AI. For example, the encryption unit can input the user's emotion data into generative AI and have the generative AI perform emotion inference.

[0123] During encryption, the encryption department can adjust the level of detail based on the importance of the data. For example, it can perform detailed encryption on highly important data to ensure data security. It can also perform simplified encryption on less important data to improve processing speed. Furthermore, it can encrypt moderately important data with appropriate detail to achieve balanced data protection. By adjusting the level of detail based on the importance of the data, data protection can be performed efficiently. Some or all of the above processing in the encryption department can be performed by AI, or it can be performed without AI. For example, the encryption department can input the importance of the data into a generative AI, which will then adjust the encryption detail.

[0124] The encryption unit can infer a user's emotions and determine encryption priorities based on these inferences. For example, it might prioritize standard encryption when the user is presumed to be relaxed, or prioritize strong encryption when the user is presumed to be stressed, or prioritize moderate encryption when the user is presumed to be active. By prioritizing encryption based on user emotions, data protection can be achieved efficiently. Emotion inference can be implemented, for example, through emotion engines or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the encryption unit can be performed by AI, or it can be performed without AI. For example, the encryption unit can input the user's emotion data into generative AI and have the generative AI perform emotion inference.

[0125] During encryption, the encryption department can adjust the encryption order based on the timing of data collection. For example, it can prioritize encrypting the latest data for real-time response. It can also postpone encrypting past data, prioritizing important data. Furthermore, it can prioritize encrypting data collected within a specific time period for a response appropriate to that timeframe. By adjusting the encryption order based on the timing of data collection, real-time response can be achieved. Some or all of the above processing in the encryption department can be performed using AI, or it can be performed without AI. For example, the encryption department can input the data collection timing into a generative AI, which will then adjust the encryption order.

[0126] The consent acquisition department can infer a user's emotions and adjust the consent acquisition method based on the inferred emotions. For example, when the user is presumed to be relaxed, the consent acquisition department provides a detailed consent acquisition method. When the user is presumed to be stressed, the consent acquisition department can provide a concise and easy-to-understand consent acquisition method. Furthermore, when the user is presumed to be active, the consent acquisition department can provide a method for quickly obtaining consent. By adjusting the consent acquisition method according to the user's emotions, more appropriate consent acquisition can be achieved. Emotion inference can be implemented, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the consent acquisition department can be performed by AI, or it can be performed without AI. For example, the consent acquisition department can input the user's emotion data into generative AI and have the generative AI perform emotion inference.

[0127] When obtaining consent, the consent acquisition department can refer to the user's past consent records to select the optimal consent acquisition method. For example, the consent acquisition department can select the optimal consent acquisition method based on the user's past consent methods. The consent acquisition department can also analyze the user's past consent records to optimize the consent acquisition process. Furthermore, the consent acquisition department can adjust the consent acquisition frequency by referring to the user's past consent records. By referring to the user's past consent records, the optimal consent acquisition method can be selected. Some or all of the above processing in the consent acquisition department can be performed by AI, or it can be performed without AI. For example, the consent acquisition department can input the user's past consent records into a generative AI and have the generative AI perform the consent acquisition method selection.

[0128] The consent acquisition department can infer a user's emotions and determine the priority of consent acquisition based on the inferred emotions. For example, when the user is presumed to be relaxed, the consent acquisition department prioritizes obtaining consent with detailed explanations. When the user is presumed to be under stress, the consent acquisition department prioritizes obtaining consent with concise and easy-to-understand explanations. Furthermore, when the user is presumed to be active, the consent acquisition department prioritizes methods that can quickly obtain consent. By determining the consent acquisition priority based on the user's emotions, more appropriate consent acquisition can be achieved. Emotion inference can be implemented, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the consent acquisition department can be performed by AI, or it can be performed without AI. For example, the consent acquisition department can input the user's emotion data into generative AI and have the generative AI perform emotion inference.

[0129] When obtaining consent, the consent acquisition department can select an appropriate consent acquisition method based on the user's device information. For example, when a user is using a smartphone, the consent acquisition department can provide a consent acquisition method suitable for the screen size. When a user is using a tablet, the consent acquisition department can also provide a consent acquisition method optimized for larger screens. Furthermore, when a user is using a smartwatch, the consent acquisition department can provide a simple and highly visual consent acquisition method. By considering the user's device information, the optimal consent acquisition method can be selected. Some or all of the above processing in the consent acquisition department can be performed by AI, or it can be performed without AI. For example, the consent acquisition department can input the user's device information into a generative AI, and have the generative AI perform the consent acquisition method selection.

[0130] ===Hardware Implementation 1-1===

[0131] Each of the aforementioned elements—collection unit, analysis unit, decision-making unit, and execution unit—is implemented, for example, by at least one of the smart device 14 and the data processing device 12. For example, the collection unit collects the user's status using the camera 42 or microphone 38B of the smart device 14 and sends it to the data processing device 12 via the control unit 46A. The analysis unit, for example, is implemented by a specific processing unit 290 of the data processing device 12, and analyzes the collected data. The decision-making unit, for example, is implemented by the specific processing unit 290 of the data processing device 12, and determines the optimal action based on the analysis results. The execution unit, for example, is implemented by the control unit 46A of the smart device 14, and executes the determined action.

[0132] ===Hardware Implementation 1-2===

[0133] Each of the aforementioned elements—collection unit, analysis unit, decision-making unit, and execution unit—is implemented, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the collection unit collects the user's status using the camera 42 or microphone 238 of the smart glasses 214 and sends it to the data processing device 12 via the control unit 46A. The analysis unit, for example, is implemented by a specific processing unit 290 of the data processing device 12, and analyzes the collected data. The decision-making unit, for example, is implemented by the specific processing unit 290 of the data processing device 12, and determines the optimal action based on the analysis results. The execution unit, for example, is implemented by the control unit 46A of the smart glasses 214, and executes the determined action.

[0134] ===Hardware Implementation 1-3===

[0135] Each of the aforementioned elements—collection unit, analysis unit, decision-making unit, and execution unit—is implemented, for example, by at least one of the head-mounted terminal 314 and the data processing device 12. For example, the collection unit collects the user's status using the camera 42 or microphone 238 of the head-mounted terminal 314 and sends it to the data processing device 12 via the control unit 46A. The analysis unit, for example, is implemented via a specific processing unit 290 of the data processing device 12, and analyzes the collected data. The decision-making unit, for example, is implemented via the specific processing unit 290 of the data processing device 12, and determines the optimal action based on the analysis results. The execution unit, for example, is implemented via the control unit 46A of the head-mounted terminal 314, and executes the determined action.

[0136] ===Hardware Implementation 1-4===

[0137] Each of the aforementioned elements—collection unit, analysis unit, decision-making unit, and execution unit—is implemented, for example, by at least one of the robot 414 and the data processing device 12. For example, the collection unit collects the user's status using the robot 414's camera 42 or microphone 238 and sends it to the data processing device 12 via the control unit 46A. The analysis unit, for example, is implemented via a specific processing unit 290 of the data processing device 12, and analyzes the collected data. The decision-making unit, for example, is implemented via the specific processing unit 290 of the data processing device 12, and determines the optimal action based on the analysis results. The execution unit, for example, is implemented via the control unit 46A of the robot 414, and executes the determined action.

[0138] The system described in this embodiment is not limited to the examples above. For example, various modifications can be made as follows.

[0139] The collection unit may include a health data collection unit for gathering user health data. This unit may collect vital signs such as heart rate, blood pressure, and body temperature. It may also collect data on user activity levels or sleep patterns. Furthermore, it may collect data on user diet or calorie intake. Based on the collected health data, the user's health status can be assessed and appropriate measures taken. Some or all of the above processing in the health data collection unit may be performed using AI, or it may not. For example, the health data collection unit can input the collected health data into generative AI, which can then perform health status analysis.

[0140] The analysis unit may include an interest preference learning unit to learn user interests and preferences. For example, the interest preference learning unit might learn a user's favorite music genres or movie types. It could also learn a user's favorite foods or drinks. Furthermore, it could learn a user's favorite travel destinations or activities. By basing its work on user interests and preferences, more personalized services can be provided. Some or all of the above processing in the interest preference learning unit can be performed using AI, or it can be performed without AI. For example, the interest preference learning unit can input user interest preference data into generative AI, which can then perform interest preference learning.

[0141] The encryption department can apply different encryption algorithms depending on the type of data. For example, advanced encryption algorithms can be used for personal information data, while lightweight encryption algorithms can be used for general environmental data. The encryption department can also apply dedicated voice encryption algorithms to voice data and dedicated image encryption algorithms to image data. Furthermore, the encryption department can apply dedicated text encryption algorithms to text data. By using optimal encryption based on the type of data, efficient data protection can be achieved. Some or all of the above processing in the encryption department can be performed by AI, or it can be performed without AI. For example, the encryption department can input the type of data into a generative AI and have the generative AI execute the encryption algorithm application.

[0142] When obtaining user consent, the consent acquisition department can refer to the user's past consent records to select the optimal consent acquisition method. For example, it can select the optimal consent acquisition method based on the user's past consent methods. The consent acquisition department can also analyze the user's past consent records to optimize the consent acquisition process. In addition, the consent acquisition department can also refer to the user's past consent records to adjust the consent acquisition frequency. By referring to the user's past consent records, the optimal consent acquisition method can be selected. Some or all of the above processing in the consent acquisition department can be performed by AI, or it can be performed without AI. For example, the consent acquisition department can input the user's past consent records into a generative AI and have the generative AI perform the consent acquisition method selection.

[0143] The execution unit can perform actions based on the user's health status. For example, it can play relaxing music when the user's heart rate is high, or lower the room temperature when the user's body temperature is high. Furthermore, it can provide appropriate lighting or music based on the user's sleep patterns. By basing actions on the user's health status, optimal actions can be provided. Some or all of the above processing in the execution unit can be performed by AI, or it can be performed without AI. For example, the execution unit can input the user's health data into generative AI and have the generative AI execute the actions.

[0144] The data collection department can infer users' emotions and adjust the timing of data collection based on these inferred emotions. For example, when a user is presumed to be relaxed, the data collection frequency can be reduced to respect user privacy. When a user is presumed to be stressed, the data collection frequency can be increased to respond quickly and appropriately. Furthermore, when a user is presumed to be active, the data collection interval can be shortened to achieve real-time response. By adjusting the timing of data collection based on user emotions, appropriate data collection can be conducted while respecting privacy. Emotion inference can be achieved, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the collection department can be performed by AI, or it can be performed without AI. For example, the collection department can input users' emotional data into generative AI and have the generative AI perform emotion inference.

[0145] The analysis unit can infer the user's emotions and adjust the presentation of the analysis based on the inferred emotions. For example, when the user is presumed to be relaxed, the analysis results can be displayed in a visually gentle color scheme. When the user is presumed to be stressed, the analysis results can be displayed in a concise and highly visual format. Furthermore, when the user is presumed to be active, the analysis results can be displayed using dynamic charts or animations. By adjusting the analysis presentation according to the user's emotions, more appropriate analysis results can be provided. Emotion inference can be achieved, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text generation AI (such as LLM) or multimodal generation AI, but is not limited to these. Some or all of the above processing in the analysis unit can be performed by AI, or it can be performed without AI. For example, the analysis unit can input the user's emotion data into the generative AI and have the generative AI perform emotion inference.

[0146] The decision-making unit can infer a user's emotions and adjust the criteria for action decisions based on these inferred emotions. For example, relaxing music can be played when the user is presumed to be relaxed. Stress-relieving ambient sounds can be played when the user is presumed to be stressed. Furthermore, upbeat music supporting the activity can be played when the user is presumed to be active. By adjusting the action decision criteria based on the user's emotions, more appropriate actions can be provided. Emotion inference can be achieved, for example, through emotion engines or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the decision-making unit can be performed by AI, or it can be performed without AI. For example, the decision-making unit can input the user's emotion data into generative AI and have the generative AI perform emotion inference.

[0147] The execution unit can infer the user's emotions and determine the priority of actions based on these inferred emotions. For example, if the user is presumed to be relaxed, relaxing music can be played first. If the user is presumed to be stressed, stress-relieving ambient sounds can also be played first. Furthermore, if the user is presumed to be active, music supporting the activity can be played first. By prioritizing actions based on user emotions, more appropriate actions can be provided. Emotion inference can be achieved, for example, through emotion engines or generative AI. Generative AI can be text-generating AI (such as LLM) or multimodal generative AI, but is not limited to these. Some or all of the above processing in the execution unit can be performed by AI, or it can be performed without AI. For example, the execution unit can input the user's emotion data into generative AI and have the generative AI perform emotion inference.

[0148] The learning unit can infer a user's emotions and select learning data based on these inferred emotions. For example, if the user is presumed to be relaxed, learning data suitable for that situation can be selected. If the user is presumed to be stressed, learning data that helps alleviate stress can also be selected. Furthermore, if the user is presumed to be active, learning data that supports the activity can also be selected. By selecting learning data based on the user's emotions, more appropriate learning can be achieved. Emotion inference can be implemented, for example, through emotion inference functions such as emotion engines or generative AI. Generative AI can be text generation AI (such as LLM) or multimodal generation AI, but is not limited to these. Some or all of the above processing in the learning unit can be performed by AI, or it can be performed without AI. For example, the learning unit can input the user's emotion data into generative AI and have the generative AI perform emotion inference.

[0149] The following is a brief description of the processing flow of Implementation Method 2.

[0150] Step 1: The data collection unit gathers the user's state. The user's state includes actions, facial expressions, and voice. The collection unit can collect the user's actions through a camera, voice through a microphone, and facial expressions through sensors. The collected data is then sent to the generative AI.

[0151] Step 2: The parsing department uses generative AI to analyze the data collected by the collection department. The parsing is based on data parsing algorithms or parsing precision. Generative AI uses text generation AI (such as LLM) or multimodal generation AI to parse the data and extract important parts for analysis.

[0152] Step 3: The decision-making department determines the optimal action based on the analysis results obtained from the analysis department. The action is determined based on factors such as appliance operation or notification sending. Examples include playing relaxing music, adjusting lighting, and setting the temperature.

[0153] Step 4: The execution department carries out the actions determined by the decision-making department. Execution is based on factors such as music playback, lighting adjustment, and temperature control. For example, this includes playing relaxing music, adjusting lighting, and adjusting the temperature.

[0154] The specific processing unit 290 sends the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires voice representing the user's input to the result of the specific processing. The control unit 46A sends the voice data representing the user's input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0155] Data generation model 58 is what is known as generative AI (Artificial Intelligence). An example of data generation model 58 includes ChatGPT (registered trademark) (Internet search).<URL:https: / / openai.com / blog / chatgpt> Generative AI, such as data generation model 58, is obtained by performing deep learning on a neural network. The data generation model 58 is input with a prompt containing instructions, and with inference data such as speech data representing speech, text data representing text, and image data representing images (e.g., data of still images or data of moving images). The data generation model 58 infers the input inference data according to the instructions shown in the prompt, and outputs the inference result in one or more data forms such as speech data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the above-described specific processing while using the data generation model 58. The data generation model 58 can also be a fine-tuned model to output inference results from prompts without instructions; in this case, the data generation model 58 can output inference results from prompts without instructions. The data processing device 12, etc., includes multiple data generation models 58, including AI other than generative AI. AI other than generative AI includes, but is not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes. Furthermore, AI can also act as an AI agent. Additionally, when the processing described above is performed by AI, this processing may be partially or entirely performed by AI, but is not limited to this example. Moreover, processing performed by AI, including generative AI, can be replaced by rule-based processing, and vice versa.

[0156] Furthermore, the processing performed by the aforementioned data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it can also be executed jointly by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Additionally, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or external devices, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or external devices.

[0157] The correspondence between the various parts and the devices or control units is not limited to the examples mentioned above, and various changes can be made.

[0158] Second Implementation Method

[0159] Figure 3 An example of the configuration of the data processing system 210 according to the second embodiment is shown.

[0160] like Figure 3 As shown, the data processing system 210 includes a data processing device 12 and smart glasses 214. One example of the data processing device 12 is a server.

[0161] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. An example of the network 54 includes a WAN and / or LAN, etc.

[0162] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0163] Microphone 238 receives user commands by receiving the user's voice. Microphone 238 captures the user's voice, converts the captured sound into speech data, and outputs it to processor 46. Speaker 240 outputs sound according to commands from processor 46.

[0164] The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, used to photograph the user's surroundings (e.g., the shooting range defined by an angle of view equivalent to the field of vision of an average healthy person).

[0165] Communication I / F 44 is connected to network 54. Communication I / F 44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F 44 and 26 is performed in a secure state.

[0166] Figure 4 An example of the main functions of the data processing device 12 and the smart glasses 214 is shown. Figure 4 As shown, in the data processing device 12, specific processing is performed by the processor 28. The memory 32 stores a specific processing program 56.

[0167] The processor 28 reads a specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is implemented by the processor 28 operating as a specific processing unit 290 based on the specific processing program 56 executed on the RAM 30.

[0168] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. The emotion inference function using the emotion-specific model 59 (emotion-specific function) includes inferring and predicting the user's emotions, performing various inferences and predictions related to the user's emotions, but is not limited to this example. Furthermore, emotion inference and prediction also include, for example, emotion analysis (analysis).

[0169] In the smart glasses 214, specific processing is performed by the processor 46. A specific processing program 60 is stored in the memory 50. The processor 46 reads the specific processing program 60 from the memory 50 and executes the read specific processing program 60 on the RAM 48. Specific processing is implemented by the processor 46 operating as a control unit 46A based on the specific processing program 60 executed on the RAM 48. Furthermore, the smart glasses 214 may also have the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and use these models to perform the same processing as the specific processing unit 290.

[0170] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. In addition, the data processing device 12 may be a server device or a user-owned terminal device (e.g., a mobile phone, robot, home appliance, etc.).

[0171] The specific processing unit 290 sends the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires voice input representing the user's input to the specific processing result. The control unit 46A sends the voice data representing the user's input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0172] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 includes generative AIs such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing instructions is input to the data generation model 58, along with inference data such as speech data representing speech, text data representing text, and image data representing images (e.g., data from still images or moving images). The data generation model 58 infers from the input inference data according to the instructions shown in the prompt, and outputs the inference result in one or more data forms such as speech data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the aforementioned specific processing while using the data generation model 58. The data generation model 58 can also be a fine-tuned model to output inference results from prompts that do not contain instructions; in this case, the data generation model 58 can output inference results from prompts that do not contain instructions. The data processing device 12, etc., includes various data generation models 58, including AIs other than generative AI. AI other than generative AI includes, but is not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes. Furthermore, AI can also act as an AI agent. Additionally, when the processing described above is performed by AI, this processing may be partially or entirely performed by AI, but is not limited to this example. Moreover, processing performed by AI, including generative AI, can be replaced by rule-based processing, and vice versa.

[0173] The data processing system 210 of the second embodiment performs the same processing as the data processing system 10 of the first embodiment. The processing performed by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it can also be executed jointly by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or external devices, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or external devices.

[0174] The correspondence between the various parts and the devices or control units is not limited to the examples mentioned above, and various changes can be made.

[0175] Third Implementation Method

[0176] Figure 5 An example of the configuration of the data processing system 310 according to the third embodiment is shown.

[0177] like Figure 5 As shown, the data processing system 310 includes a data processing device 12 and a head-mounted terminal 314. An example of the data processing device 12 is a server.

[0178] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. An example of the network 54 includes a WAN and / or LAN, etc.

[0179] The head-mounted terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0180] Microphone 238 receives user commands by receiving the user's voice. Microphone 238 captures the user's voice, converts the captured sound into speech data, and outputs it to processor 46. Speaker 240 outputs sound according to commands from processor 46.

[0181] The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, used to photograph the user's surroundings (e.g., the shooting range defined by an angle of view equivalent to the field of vision of an average healthy person).

[0182] Communication I / F 44 is connected to network 54. Communication I / F 44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F 44 and 26 is performed in a secure state.

[0183] Figure 6 An example of the main functions of the data processing device 12 and the head-mounted terminal 314 is shown. Figure 6 As shown, in the data processing device 12, specific processing is performed by the processor 28. The memory 32 stores a specific processing program 56.

[0184] The processor 28 reads a specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is implemented by the processor 28 operating as a specific processing unit 290 based on the specific processing program 56 executed on the RAM 30.

[0185] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. The emotion inference function using the emotion-specific model 59 (emotion-specific function) includes inferring and predicting the user's emotions, performing various inferences and predictions related to the user's emotions, but is not limited to this example. Furthermore, emotion inference and prediction also include, for example, emotion analysis (analysis).

[0186] In the head-mounted terminal 314, specific processing is performed by the processor 46. A specific program 60 is stored in the memory 50. The processor 46 reads the specific program 60 from the memory 50 and executes the read specific program 60 on the RAM 48. Specific processing is implemented by the processor 46 operating as a control unit 46A based on the specific program 60 executed on the RAM 48. Furthermore, the head-mounted terminal 314 may also have the same data generation model and emotion-specific model as the data generation model 58 and the emotion-specific model 59, and use these models to perform the same processing as the specific processing unit 290.

[0187] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. In addition, the data processing device 12 may be a server device or a user-owned terminal device (e.g., a mobile phone, robot, home appliance, etc.).

[0188] The specific processing unit 290 sends the result of the specific processing to the head-mounted terminal 314. In the head-mounted terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires voice representing the user's input to the specific processing result. The control unit 46A sends the voice data representing the user's input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0189] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 includes generative AIs such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing instructions is input to the data generation model 58, along with inference data such as speech data representing speech, text data representing text, and image data representing images (e.g., data from still images or moving images). The data generation model 58 infers from the input inference data according to the instructions shown in the prompt, and outputs the inference result in one or more data forms such as speech data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the aforementioned specific processing while using the data generation model 58. The data generation model 58 can also be a fine-tuned model to output inference results from prompts that do not contain instructions; in this case, the data generation model 58 can output inference results from prompts that do not contain instructions. The data processing device 12, etc., includes various data generation models 58, including AIs other than generative AI. AI other than generative AI includes, but is not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes. Furthermore, AI can also act as an AI agent. Additionally, when the processing described above is performed by AI, this processing may be partially or entirely performed by AI, but is not limited to this example. Moreover, processing performed by AI, including generative AI, can be replaced by rule-based processing, and vice versa.

[0190] The data processing system 310 of the third embodiment performs the same processing as the data processing system 10 of the first embodiment. The processing performed by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the head-mounted terminal 314, but it can also be executed jointly by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the head-mounted terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the head-mounted terminal 314 or external devices, and the head-mounted terminal 314 acquires or collects information required for processing from the data processing device 12 or external devices.

[0191] The correspondence between the various parts and the devices or control units is not limited to the examples mentioned above, and various changes can be made.

[0192] Fourth Implementation Method

[0193] Figure 7 An example of the configuration of the data processing system 410 according to the fourth embodiment is shown.

[0194] like Figure 7 As shown, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0195] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, RAM 30, and memory 32. The processor 28, RAM 30, and memory 32 are connected to a bus 34. Furthermore, the database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. An example of the network 54 includes a WAN and / or LAN, etc.

[0196] Robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control object 443. The computer 36 includes a processor 46, RAM 48, and memory 50. The processor 46, RAM 48, and memory 50 are connected to a bus 52. Furthermore, the microphone 238, speaker 240, camera 42, and control object 443 are also connected to the bus 52.

[0197] Microphone 238 receives user commands by receiving the user's voice. Microphone 238 captures the user's voice, converts the captured sound into speech data, and outputs it to processor 46. Speaker 240 outputs sound according to commands from processor 46.

[0198] The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, used to photograph the user's surroundings (e.g., the shooting range defined by an angle of view equivalent to the field of vision of an average healthy person).

[0199] Communication I / F 44 is connected to network 54. Communication I / F 44 and 26 are responsible for the transmission and reception of various information between processor 46 and processor 28 via network 54. The transmission and reception of various information between processor 46 and processor 28 using communication I / F 44 and 26 is performed in a secure state.

[0200] The controlled object 443 includes a display device, LEDs for the eyes, and motors for driving the arms, hands, and feet. The posture and movements of the robot 414 are controlled by controlling the motors for the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. In addition, facial expressions of the robot 414 can also be expressed by controlling the illumination state of the LEDs for the robot 414's eyes.

[0201] Figure 8 An example of the main functions of the data processing device 12 and the robot 414 is shown. For example... Figure 8 As shown, in the data processing device 12, specific processing is performed by the processor 28. The memory 32 stores a specific processing program 56.

[0202] The processor 28 reads a specific processing program 56 from the memory 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is implemented by the processor 28 operating as a specific processing unit 290 based on the specific processing program 56 executed on the RAM 30.

[0203] The memory 32 stores a data generation model 58 and an emotion-specific model 59. The data generation model 58 and the emotion-specific model 59 are used by the specific processing unit 290. The specific processing unit 290 can use the emotion-specific model 59 to infer the user's emotions and perform specific processing based on the user's emotions. The emotion inference function using the emotion-specific model 59 (emotion-specific function) includes inferring and predicting the user's emotions, performing various inferences and predictions related to the user's emotions, but is not limited to this example. Furthermore, emotion inference and prediction also include, for example, emotion analysis (analysis).

[0204] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in memory 50. Processor 46 reads the specific program 60 from memory 50 and executes the read specific program 60 on RAM 48. Specific processing is achieved by processor 46 acting as control unit 46A based on the specific program 60 executed on RAM 48. Furthermore, robot 414 may also have the same data generation model and emotion-specific model as data generation model 58 and emotion-specific model 59, and use these models to perform the same processing as specific processing unit 290.

[0205] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains the processing results (prediction results, etc.) using the data generation model 58 by communicating with the server device that has the data generation model 58. In addition, the data processing device 12 may be a server device or a user-owned terminal device (e.g., a mobile phone, robot, home appliance, etc.).

[0206] The specific processing unit 290 sends the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires voice representing the user's input regarding the result of the specific processing. The control unit 46A sends the voice data representing the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0207] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 includes generative AIs such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing instructions is input to the data generation model 58, along with inference data such as speech data representing speech, text data representing text, and image data representing images (e.g., data from still images or moving images). The data generation model 58 infers from the input inference data according to the instructions shown in the prompt, and outputs the inference result in one or more data forms such as speech data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the aforementioned specific processing while using the data generation model 58. The data generation model 58 can also be a fine-tuned model to output inference results from prompts that do not contain instructions; in this case, the data generation model 58 can output inference results from prompts that do not contain instructions. The data processing device 12, etc., includes various data generation models 58, including AIs other than generative AI. AI other than generative AI includes, but is not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or Naive Bayes. Furthermore, AI can also act as an AI agent. Additionally, when the processing described above is performed by AI, this processing may be partially or entirely performed by AI, but is not limited to this example. Moreover, processing performed by AI, including generative AI, can be replaced by rule-based processing, and vice versa.

[0208] The data processing system 410 of the fourth embodiment performs the same processing as the data processing system 10 of the first embodiment. The processing performed by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it can also be executed jointly by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or external devices, and the robot 414 acquires or collects information required for processing from the data processing device 12 or external devices.

[0209] The correspondence between the various parts and the devices or control units is not limited to the examples mentioned above, and various changes can be made.

[0210] Furthermore, the emotion-specific model 59, serving as an emotion engine, can determine the user's emotion based on a specific mapping. Specifically, the emotion-specific model 59 can determine the user's emotion based on an emotion graph that serves as a specific mapping (see...). Figure 9 The robot's emotions can be determined by the emotion-specific model 59. In addition, the emotion-specific model 59 can also determine the robot's emotions in the same way, and the specific processing unit 290 can also perform specific processing using the robot's emotions.

[0211] Figure 9 This is a diagram representing an emotion map 400 that maps various emotions. In the emotion map 400, emotions are arranged radially from the center in concentric circles. The closer to the center of the concentric circles, the more primitive the emotion is. Further out on the concentric circles, emotions are arranged representing states or actions arising from mood. Emotion is a concept that includes both feelings and mental states. To the left of the concentric circles, emotions generated by reactions occurring in the brain are arranged roughly. To the right of the concentric circles, emotions guided by situational judgments are arranged roughly. Above and below the concentric circles, emotions generated by reactions occurring in the brain and guided by situational judgments are arranged roughly. Furthermore, the emotion of "pleasure" is arranged above the concentric circles, and the emotion of "unpleasantness" is arranged below. Thus, in the emotion map 400, various emotions are mapped according to the structure of emotion generation, while easily generated emotions are mapped nearby.

[0212] These emotions are distributed at the 3 o'clock position on the Emotion Chart 400, and usually fluctuate between peace and unease. In the right half of the Emotion Chart 400, because situational awareness is more dominant than internal feelings, it gives a sense of calm.

[0213] The inner side of the emotion diagram 400 represents the mind, and the outer side of the emotion diagram 400 represents actions. Therefore, the further you go to the outer side of the emotion diagram 400, the more the emotion can be seen (manifested in actions).

[0214] Here, human emotions are based on a balance of various factors such as posture and blood sugar levels. When these balances deviate from the ideal, it indicates unhappiness; when they approach the ideal, it indicates pleasure. In robots, cars, and motorcycles, emotions can also be created based on a balance of factors such as posture and remaining battery power. When these balances deviate from the ideal, it indicates unhappiness; when they approach the ideal, it indicates pleasure. Emotion maps can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Speech Emotion Recognition and Brain Physiological Signal Analysis Systems for Emotion, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). In the left half of the emotion map, emotions belonging to the "Reaction" domain, where sensation is dominant, are arranged. Furthermore, in the right half of the emotion map, emotions belonging to the "Situation" domain, where situational cognition is dominant, are arranged.

[0215] The emotion map defines two types of emotions that promote learning. One is a negative emotion located near the middle of "repentance" or "reflection" on the situation side. That is, when the robot experiences negative emotions such as "I never want to feel this way again" or "I never want to be scolded again." The other is a positive emotion located near "desire" on the response side. That is, when the robot experiences positive feelings such as "wanting more" or "wanting to know more."

[0216] The emotion-specific model 59 feeds user input into a pre-learned neural network to obtain emotion values ​​representing each emotion shown in the emotion graph 400, and determines the user's emotion. This neural network is pre-learned based on multiple learning data sets that combine user input with emotion values ​​representing each emotion shown in the emotion graph 400. Furthermore, this neural network is learned to... Figure 10 As shown in sentiment graph 900, sentiment values ​​in nearby configurations are similar to each other. Figure 10 Examples show that multiple emotions such as "peace of mind", "stability", and "reassurance" have similar emotional values.

[0217] In the above embodiments, a specific processing is described by a single computer 22, but the technology disclosed herein is not limited to this, and distributed processing by multiple computers, including computer 22, is also possible.

[0218] In the above embodiments, an example of storing a specific processing program 56 in memory 32 is illustrated, but the technology disclosed herein is not limited thereto. For example, the specific processing program 56 may also be stored in a portable computer-readable non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed into the computer 22 of the data processing device 12. The processor 28 performs specific processing according to the specific processing program 56.

[0219] Alternatively, the specific processing program 56 can be stored in a storage device such as a server connected to the data processing device 12 via a network 54, and the specific processing program 56 can be downloaded and installed into the computer 22 upon request from the data processing device 12.

[0220] Furthermore, it is not necessary to store the entire specific process 56 in a storage device such as a server connected to the data processing device 12 via the network 54, nor is it necessary to store the entire specific process 56 in the memory 32; a portion of the specific process 56 may also be stored.

[0221] As a hardware resource for performing specific processing, various processors can be used. For example, a CPU is a general-purpose processor that functions as a hardware resource for performing specific processing by executing software, i.e., programs. Additionally, a dedicated circuit can be listed as a processor; it is a processor with a circuit structure specifically designed for performing specific processing, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit). Every processor has built-in or connected memory, and every processor executes specific processing by using memory.

[0222] The hardware resources for performing a specific process can consist of one of these various processors, or a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Furthermore, the hardware resources for performing a specific process can also be a single processor.

[0223] As an example of a single processor, the first type consists of a combination of one or more CPUs and software, which functions as a hardware resource to perform specific processing. The second type uses a processor, such as a System-on-a-chip (SoC), which implements the entire system functionality, including multiple hardware resources for performing specific processing, using a single IC chip. In this case, the specific processing is implemented using one or more of the aforementioned processors as hardware resources.

[0224] Furthermore, as the hardware architecture of these various processors, more specifically, circuits combining semiconductor elements and other circuit components can be used. Moreover, the specific process described above is merely an example. Therefore, it goes without saying that, without departing from the main point, unnecessary steps can be removed, new steps can be added, or the processing order can be changed.

[0225] Furthermore, although the above examples have been described in terms of first to fourth embodiments, some or all of these embodiments can be combined. Additionally, the smart device 14, smart glasses 214, head-mounted terminal 314, and robot 414 are just examples and can be combined separately, or other devices may be used. Furthermore, although the above examples have been described in terms of morphological example 1 and morphological example 2, these can also be combined.

[0226] The foregoing descriptions and illustrations are detailed explanations of the parts covered by this disclosure and are merely one example of this disclosure. For instance, the descriptions of the above-described structure, function, role, and effect are just one example of the structure, function, role, and effect of the parts covered by this disclosure. Therefore, it goes without saying that, without departing from the spirit of this disclosure, unnecessary parts can be deleted, new elements can be added, or replacements can be made to the foregoing descriptions and illustrations. Furthermore, to avoid confusion and facilitate understanding of the parts covered by this disclosure, explanations of technical common sense that does not require special explanation for implementing this disclosure have been omitted from the foregoing descriptions and illustrations.

[0227] All documents, patent applications and technical standards described in this specification are incorporated herein by reference as if they were specifically and individually described as incorporated by reference.

Claims

1. A system, characterized by comprising: a collection section for collecting a state of a user; an analysis section for analyzing data collected by the collection section; a decision section for deciding an appropriate action based on an analysis result obtained by the analysis section; an execution section for executing an action decided by the decision section.

2. The system according to claim 1, wherein the analysis section includes a learning section for learning a user behavior pattern.

3. The system according to claim 1, wherein the collection section includes an encryption section for encrypting data collected by a camera.

4. The system according to claim 1, wherein the collection section includes a consent acquisition section for acquiring a user consent.

5. The system according to claim 1, wherein the execution section executes music play.

6. The system according to claim 1, wherein the execution section executes lighting adjustment.

7. The system according to claim 1, wherein the collection section infers a user's emotion, and adjusts a timing of data collection based on the inferred user's emotion.

8. The system according to claim 1, wherein the collection section analyzes a past behavior record of a user, and selects an appropriate data collection method.

Images