system

Abstract

The system according to the embodiment aims to enable remote control of IoT home appliances from different manufacturers using a single app. [Solution] A system according to an embodiment includes an acquisition unit, an analysis unit, and a generation unit. The acquisition unit acquires information about each IoT home appliance. The analysis unit analyzes the protocol of each IoT home appliance based on the information acquired by the acquisition unit. The generation unit generates a unified operation interface based on the protocol analyzed by the analysis unit.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional technology, it was difficult to remotely control IoT home appliances from different manufacturers using a single app.

[0005] The system according to the embodiment aims to enable remote control of IoT home appliances from different manufacturers using a single app. [Means for solving the problem]

[0006] The system according to the embodiment includes an acquisition unit, an analysis unit, and a generation unit. The acquisition unit acquires information about each IoT home appliance. The analysis unit analyzes the protocol of each IoT home appliance based on the information acquired by the acquisition unit. The generation unit generates a unified operation interface based on the protocol analyzed by the analysis unit. [Effects of the Invention]

[0007] The system according to the embodiment enables IoT home appliances from different manufacturers to be remotely controlled using a single app. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) An IoT home appliance control system according to an embodiment of the present invention remotely controls IoT home appliances from different manufacturers using a single app. This system begins when a user installs the app and registers each IoT home appliance in the app. The app then analyzes the protocols of each IoT home appliance and generates a unified operation interface. This allows the user to remotely control IoT home appliances from different manufacturers using a single app. For example, a user installs the app and registers IoT home appliances, such as a refrigerator, air conditioner, and lighting fixtures, in the app. The user then enters information about each IoT home appliance and saves it in the app. The app then analyzes the communication protocols of each IoT home appliance and generates a unified operation interface. For example, the user can perform operations such as setting the refrigerator temperature, adjusting the air conditioner's airflow, and adjusting the lighting brightness using a unified interface. Based on the generated operation interface, the user can remotely control IoT home appliances from different manufacturers using a single app. For example, the user can use the app to set the refrigerator temperature, adjust the air conditioner's airflow, and adjust the lighting brightness. In this way, the user can easily control IoT home appliances from different manufacturers using a single app. This mechanism allows the user to remotely control IoT home appliances from different manufacturers using a single app. This simplifies operation and improves user convenience. For example, users can control all IoT home appliances with one app, eliminating the need to use multiple apps. The unified operation interface also makes operation intuitive and simple. For example, a single app can be used to set the refrigerator temperature, adjust the air conditioner's fan speed, and adjust the brightness of the lights. This allows the IoT home appliance operation system to remotely control IoT home appliances from different manufacturers with a single app. This eliminates the need for users to use multiple apps, making operation simple and intuitive.

[0029] An IoT home appliance operation system according to an embodiment includes an acquisition unit, an analysis unit, and a generation unit. The acquisition unit acquires information about each IoT home appliance. For example, the acquisition unit can acquire information about a refrigerator, air conditioner, lighting, and the like entered by a user. The acquisition unit can acquire information about each IoT home appliance by, for example, scanning a two-dimensional code (e.g., a QR code (registered trademark)) or using Bluetooth (registered trademark). For example, the acquisition unit can acquire information about the IoT home appliance by scanning the two-dimensional code with a camera. The acquisition unit can also acquire information by pairing with an IoT home appliance using Bluetooth. The analysis unit analyzes the protocol of each IoT home appliance based on the information acquired by the acquisition unit. For example, the analysis unit can analyze the communication protocol of a refrigerator to identify a temperature setting method. The analysis unit can also analyze the communication protocol of an air conditioner to identify a fan speed adjustment method. The analysis unit can also analyze the communication protocol of a lighting device to identify a brightness adjustment method. The generation unit generates a unified operation interface based on the protocols analyzed by the analysis unit. For example, the generation unit enables operations such as setting the refrigerator temperature, adjusting the airflow of an air conditioner, and adjusting the brightness of lights to be performed using a unified interface. The generation unit, for example, generates a GUI (Graphical User Interface) to enable users to operate intuitively. The generation unit can also generate an interface that is operated using voice commands. For example, the generation unit enables a user to set the refrigerator temperature by giving a voice command such as "Set the refrigerator temperature to 5 degrees." This allows the IoT home appliance operation system according to the embodiment to remotely control IoT home appliances from different manufacturers using a single app. This eliminates the need for users to use multiple apps, making operation simple and intuitive.

[0030] The IoT home appliance operation system includes an encryption unit that encrypts communications with each IoT home appliance. The encryption unit encrypts communications with each IoT home appliance. For example, the encryption unit can encrypt communications using the Advanced Encryption Standard (AES). AES is an advanced encryption standard that is widely used to ensure data security. The encryption unit can also encrypt communications using Rivest-Shamir-Adleman (RSA). RSA is a public key cryptography method used to encrypt and decrypt data. The encryption unit can also encrypt communications using Transport Layer Security (TLS). TLS is a protocol used to ensure the security of data communications over the Internet. This allows the encryption unit to encrypt communications with each IoT home appliance and ensure data security. For example, the encryption unit can encrypt communications with a refrigerator using AES, communications with an air conditioner using RSA, and communications with lighting using TLS. This allows the IoT home appliance operation system to encrypt communications with each IoT home appliance and ensure data security.

[0031] The IoT home appliance operation system includes an authentication unit that performs authentication with each IoT home appliance. The authentication unit performs authentication with each IoT home appliance. For example, the authentication unit can perform authentication with each IoT home appliance using password authentication. Password authentication is a method of authentication using a password set by the user. The authentication unit can also perform authentication with each IoT home appliance using biometric authentication. Biometric authentication is a method of authentication using biometric information such as fingerprint authentication or facial recognition. The authentication unit can also perform authentication with each IoT home appliance using token authentication. Token authentication is a method of authentication using a physical token or digital token held by the user. In this way, the authentication unit can perform authentication with each IoT home appliance and ensure security. For example, the authentication unit can perform authentication with a refrigerator using password authentication, authentication with an air conditioner using biometric authentication, and authentication with lighting using token authentication. In this way, the IoT home appliance operation system can perform authentication with each IoT home appliance and ensure security.

[0032] The IoT home appliance operation system includes a registration unit that supports the user's steps when registering an IoT home appliance. The registration unit supports the user's steps when registering an IoT home appliance. For example, the registration unit may provide a guided wizard to allow the user to register an IoT home appliance step by step. The guided wizard may guide the user through the registration process by showing the user the next steps to take. The registration unit may also display help messages to provide support if the user encounters difficulty during the registration process. The help messages may allow the user to refer to frequently asked questions and troubleshooting information. The registration unit may also provide tutorials to allow the user to learn the registration process. The tutorials may explain the registration process to the user and allow the user to learn by actually operating the appliance. In this way, the registration unit supports the user's steps when registering an IoT home appliance and simplifies the registration process. For example, the registration unit may support the refrigerator registration process with a guided wizard, the air conditioner registration process with help messages, and the lighting registration process with a tutorial. In this way, the IoT home appliance operation system allows the user to easily register IoT home appliances.

[0033] The acquisition unit can acquire information about each IoT home appliance by scanning a two-dimensional code or by using Bluetooth. The acquisition unit acquires information about each IoT home appliance by, for example, scanning a two-dimensional code. For example, the acquisition unit may use a camera to scan a two-dimensional code to acquire information about the IoT home appliance. The two-dimensional code includes identification information and setting information about the IoT home appliance, and the acquisition unit acquires information by reading the code. The acquisition unit can also acquire information about each IoT home appliance by using Bluetooth. For example, the acquisition unit may use Bluetooth to pair with the IoT home appliance and acquire information. Using Bluetooth allows information to be acquired wirelessly. This allows the acquisition unit to acquire information about each IoT home appliance by scanning a two-dimensional code or by using Bluetooth. For example, the acquisition unit can scan the two-dimensional code on a refrigerator to acquire information, pair with an air conditioner via Bluetooth to acquire information, and scan the two-dimensional code on a light to acquire information. This allows the acquisition unit to acquire information about IoT home appliances in a variety of ways.

[0034] The generation unit can abstract the functions of each IoT home appliance and set common operation items. For example, the generation unit abstracts the functions of each IoT home appliance and sets common operation items. For example, the generation unit abstracts operations such as refrigerator temperature setting, air conditioner airflow adjustment, and lighting brightness adjustment and sets them as common operation items. This allows users to operate IoT home appliances from different manufacturers with a unified operation interface. For example, the generation unit abstracts the refrigerator temperature setting as "temperature adjustment," the air conditioner airflow adjustment as "airflow adjustment," and the lighting brightness adjustment as "brightness adjustment." This allows users to operate IoT home appliances from different manufacturers using common operation items such as "temperature adjustment," "airflow adjustment," and "brightness adjustment." Furthermore, the generation unit can categorize the functions of each IoT home appliance when setting common operation items. For example, the generation unit categorizes home appliances such as refrigerators, air conditioners, and lighting into categories such as "temperature adjustment," "airflow adjustment," and "brightness adjustment." This allows users to select and operate operation items for each category. This allows the generation unit to abstract the functions of each IoT home appliance and set common operation items, making it possible to operate IoT home appliances from different manufacturers using a unified operation interface.

[0035] The acquisition unit can analyze the use frequency of each IoT home appliance and select the optimal information acquisition method. For example, the acquisition unit can analyze the use frequency of each IoT home appliance and select the optimal information acquisition method. For example, the acquisition unit can prioritize acquisition of information from frequently used home appliances. The acquisition unit can also reduce the acquisition of information from infrequently used home appliances. Furthermore, the acquisition unit can dynamically adjust the timing of information acquisition according to the use frequency. This allows the acquisition unit to analyze the use frequency of each IoT home appliance and select the optimal information acquisition method. The analysis of the use frequency is performed, for example, using log data analysis or statistical methods. For example, the acquisition unit can collect the use history of each IoT home appliance as log data and analyze it to identify the use frequency. The acquisition unit can also evaluate the use frequency of each IoT home appliance using statistical methods. This allows the acquisition unit to optimize the information acquisition method according to the use frequency and acquire information efficiently. For example, the acquisition unit can prioritize acquisition of information from the refrigerator if the refrigerator is used frequently, and reduce acquisition of information from the air conditioner if the air conditioner is used infrequently. This allows the acquisition unit to optimize the information acquisition method according to the frequency of use, and acquire information efficiently.

[0036] The acquisition unit can filter information about IoT home appliances based on the user's current living situation and usage pattern when acquiring the information. For example, the acquisition unit can filter information about IoT home appliances based on the user's current living situation and usage pattern when acquiring the information. For example, when the user is at home, the acquisition unit acquires detailed information about the status of the home appliance. Furthermore, when the user is out, the acquisition unit can acquire only important information. Furthermore, the acquisition unit can acquire only necessary information based on the user's usage pattern. This enables the acquisition unit to acquire information according to the user's living situation and usage pattern. The acquisition of living situations is performed based on, for example, daily behavior patterns and schedule information. For example, the acquisition unit acquires the user's schedule information and determines whether the user is at home or out. The acquisition unit can also analyze the user's behavior patterns to identify the user's current living situation. The acquisition of usage patterns is performed based on, for example, device usage history and operation logs. For example, the acquisition unit collects usage history of each IoT home appliance and analyzes it to identify the usage pattern. The acquisition unit can also analyze operation logs to identify the user's usage pattern. This enables the acquisition unit to acquire information according to the user's living situation and usage pattern, and enables the user to efficiently acquire information necessary for the user.

[0037] When acquiring information about IoT home appliances, the acquisition unit can prioritize acquiring highly relevant information by taking into account the user's geographical location information. For example, when acquiring information about IoT home appliances, the acquisition unit prioritizes acquiring highly relevant information by taking into account the user's geographical location information. For example, when the user is at home, the acquisition unit prioritizes acquiring information about home appliances. Furthermore, when the user is at work, the acquisition unit can prioritize acquiring information about workplace appliances. Furthermore, when the user is traveling, the acquisition unit can prioritize acquiring information about appliances at the user's travel destination. This allows the acquisition unit to acquire highly relevant information based on the user's geographical location information. The acquisition of geographical location information is performed based on, for example, GPS data or Wi-Fi location information. For example, the acquisition unit acquires GPS data from the user's smartphone to identify the user's current location. Furthermore, the acquisition unit can also identify the user's current location using Wi-Fi location information. This allows the acquisition unit to acquire highly relevant information based on the user's geographical location information and prioritize providing information necessary to the user.

[0038] The acquisition unit can analyze the user's social media activity and acquire related information when acquiring information about an IoT home appliance. For example, when acquiring information about an IoT home appliance, the acquisition unit analyzes the user's social media activity and acquires related information. For example, if the user mentions a specific home appliance on social media, the acquisition unit preferentially acquires information about the home appliance. The acquisition unit can also acquire information about home appliances of interest from the user's social media activity. Furthermore, the acquisition unit can acquire information about related home appliances based on the content of the user's social media posts. This allows the acquisition unit to acquire related information based on the user's social media activity. The analysis of social media activity is performed based on, for example, the content of posts, like history, follower information, etc. For example, the acquisition unit collects post content from the user's social media account and analyzes it to identify information about related home appliances. The acquisition unit can also analyze the user's like history to identify information about home appliances of interest. Furthermore, the acquisition unit can analyze follower information to identify information about home appliances in which the user is interested. This allows the acquisition unit to acquire related information based on the user's social media activity and provide the user with necessary information.

[0039] The analysis unit can analyze the complexity of the protocol of each IoT home appliance and select the optimal analysis algorithm. For example, the analysis unit analyzes the complexity of the protocol of each IoT home appliance and selects the optimal analysis algorithm. For example, the analysis unit applies an advanced analysis algorithm to home appliances with complex protocols. The analysis unit can also apply a simple analysis algorithm to home appliances with simple protocols. Furthermore, the analysis unit can dynamically select an analysis algorithm depending on the complexity of the protocol. This allows the analysis unit to select the optimal analysis algorithm depending on the complexity of the protocol. Protocol complexity analysis is performed based on, for example, the length of the message or the number of fields. For example, the analysis unit measures the length of the protocol message of each IoT home appliance and evaluates the complexity based on this. The analysis unit can also count the number of fields in the protocol message and evaluate the complexity based on this. This allows the analysis unit to select the optimal analysis algorithm depending on the complexity of the protocol and analyze the protocol efficiently. For example, the analysis unit can apply an advanced analysis algorithm if the protocol of a refrigerator is complex, and apply a simple analysis algorithm if the protocol of an air conditioner is simple. This allows the analysis unit to select the optimal analysis algorithm depending on the complexity of the protocol, and to analyze the protocol efficiently.

[0040] The analysis unit can improve the analysis accuracy by referring to past analysis data when analyzing a protocol. The analysis unit can improve the analysis accuracy by referring to past analysis data, for example, when analyzing a protocol. For example, the analysis unit learns protocol patterns based on the past analysis data and improves the analysis accuracy. The analysis unit can also refer to the past analysis data to efficiently analyze home appliances with similar protocols. Furthermore, the analysis unit can utilize the past analysis data to improve the accuracy of the analysis algorithm. In this way, the analysis unit can improve the analysis accuracy by referring to the past analysis data. The reference to the past analysis data is performed based on, for example, past log data and a history of analysis results. For example, the analysis unit collects past log data and analyzes it to identify protocol patterns. The analysis unit can also refer to a history of past analysis results to efficiently analyze home appliances with similar protocols. In this way, the analysis unit can improve the analysis accuracy by referring to the past analysis data and efficiently analyze protocols. For example, the analysis unit can refer to past analysis data of a refrigerator and learn protocol patterns to improve the analysis accuracy. In this way, the analysis unit can improve the analysis accuracy by referring to the past analysis data and efficiently analyze protocols.

[0041] The analysis unit can take into account the geographical distribution of IoT home appliances when analyzing protocols. For example, the analysis unit can perform protocol analysis while taking into account the geographical distribution of IoT home appliances. For example, the analysis unit prioritizes analysis of protocols of geographically close home appliances. The analysis unit can also thin out protocol analysis of geographically distant home appliances. Furthermore, the analysis unit can dynamically adjust the analysis priority based on the geographical distribution. This allows the analysis unit to perform protocol analysis based on the geographical distribution, thereby enabling efficient analysis. The geographical distribution can be taken into account, for example, based on the installation location of devices and their distribution by region. For example, the analysis unit can identify the installation location of each IoT home appliance and evaluate the geographical distribution based on this. The analysis unit can also analyze the distribution of devices by region and determine the analysis priority based on this. This allows the analysis unit to perform protocol analysis based on the geographical distribution and efficiently analyze protocols. For example, the analysis unit can identify the location of a refrigerator and determine the protocol analysis priority based on this. This allows the analysis unit to perform protocol analysis based on the geographical distribution and efficiently analyze protocols.

[0042] The analysis unit can improve the accuracy of the analysis by referring to related technical literature during protocol analysis. For example, the analysis unit improves the accuracy of the analysis by referring to related technical literature during protocol analysis. For example, the analysis unit learns protocol patterns based on related technical literature and improves the analysis accuracy. The analysis unit can also efficiently analyze home appliances with similar protocols by referring to technical literature. Furthermore, the analysis unit can utilize technical literature to improve the accuracy of the analysis algorithm. As a result, the analysis unit can improve the accuracy of the analysis by referring to related technical literature. Referencing technical literature is performed based on, for example, patent documents, academic papers, and technical reports. For example, the analysis unit collects patent documents and analyzes them to identify protocol patterns. The analysis unit can also efficiently analyze home appliances with similar protocols by referring to academic papers. Furthermore, the analysis unit can utilize technical reports to improve the accuracy of the analysis algorithm. As a result, the analysis unit can improve the accuracy of the analysis by referring to related technical literature and efficiently analyze protocols. For example, the analysis unit can improve the accuracy of the analysis by referring to related patent documents when analyzing a refrigerator protocol. This allows the analysis unit to improve the accuracy of analysis by referring to related technical literature, and to analyze protocols efficiently.

[0043] The generation unit can analyze the functions of each IoT home appliance in detail and generate an optimal operation interface. For example, the generation unit can analyze the functions of each IoT home appliance in detail and generate an optimal operation interface. For example, the generation unit can analyze the functions of each home appliance in detail and set common operation items. The generation unit can also generate a customizable operation interface according to the function of each home appliance. Furthermore, the generation unit can abstract the functions of each home appliance and provide an intuitive operation interface. This allows the generation unit to generate an optimal operation interface according to the function of each IoT home appliance. The operation interface is generated based on, for example, a touchscreen, voice commands, or remote control. For example, the generation unit can provide an intuitive operation interface using a touchscreen. The generation unit can also provide an operation interface using voice commands. Furthermore, the generation unit can provide an operation interface using a remote control. This allows the generation unit to generate an optimal operation interface according to the function of each IoT home appliance and provide an interface that is easy for the user to use. For example, the generation unit can analyze the functions of a refrigerator in detail and generate an operation interface for temperature adjustment. This allows the generation unit to generate an optimal operation interface according to the functions of each IoT home appliance, providing an interface that is easy for users to use.

[0044] The generation unit can provide an optimal interface by referring to the user's past operation history when generating the operation interface. For example, the generation unit can provide an optimal interface by referring to the user's past operation history when generating the operation interface. For example, the generation unit can propose an optimal operation interface based on the user's past operation history. The generation unit can also analyze the user's past operation patterns and provide a customizable interface. Furthermore, the generation unit can generate an intuitive operation interface by referring to the user's past operation history. As a result, the generation unit can provide an optimal operation interface based on the user's past operation history. The reference to the past operation history is based on, for example, an operation log, use frequency, or operation pattern. For example, the generation unit collects and analyzes the user's operation log to identify an optimal operation interface. The generation unit can also analyze use frequency and prioritize displaying frequently used operation items. Furthermore, the generation unit can analyze operation patterns and generate an interface that the user can intuitively operate. As a result, the generation unit can provide an optimal operation interface based on the user's past operation history and provide an interface that is easy for the user to use. For example, the generation unit can optimize the temperature adjustment operation interface by referring to the operation history of a refrigerator. This allows the generation unit to provide an optimal operation interface based on the user's past operation history, and to provide an interface that is easy for the user to use.

[0045] The generation unit can generate an operation interface by taking into account the geographical distribution of IoT home appliances. For example, the generation unit generates the operation interface by taking into account the geographical distribution of IoT home appliances. For example, the generation unit prioritizes generating operation interfaces for geographically close home appliances. The generation unit can also thin out the generation of operation interfaces for geographically distant home appliances. Furthermore, the generation unit can dynamically adjust the generation priority of operation interfaces based on the geographical distribution. This allows the generation unit to generate an optimal operation interface based on the geographical distribution. The geographical distribution can be taken into account, for example, based on the installation locations of devices and their distribution by region. For example, the generation unit can identify the installation locations of each IoT home appliance and evaluate the geographical distribution based on this. The generation unit can also analyze the distribution of devices by region and determine the generation priority of operation interfaces based on this. This allows the generation unit to generate an optimal operation interface based on the geographical distribution and efficiently provide the interface. For example, the generation unit can identify the locations where refrigerators are installed and determine the generation priority of operation interfaces based on this. This allows the generation unit to generate an optimal operation interface based on the geographical distribution and efficiently provide the interface.

[0046] The generation unit can improve the accuracy of the interface by referring to related design guidelines when generating the operation interface. For example, the generation unit can improve the accuracy of the interface by referring to related design guidelines when generating the operation interface. For example, the generation unit improves the accuracy of the operation interface based on the related design guidelines. The generation unit can also provide an intuitive operation interface by referring to the design guidelines. Furthermore, the generation unit can maintain consistency of the operation interface by utilizing the design guidelines. In this way, the generation unit can improve the accuracy of the interface by referring to the related design guidelines. The reference to the design guidelines is performed based on, for example, UI / UX guidelines or brand guidelines. For example, the generation unit collects UI / UX guidelines and improves the accuracy of the operation interface based on the UI / UX guidelines. The generation unit can also provide an intuitive operation interface by referring to the brand guidelines. Furthermore, the generation unit can maintain consistency of the operation interface by utilizing the design guidelines. In this way, the generation unit can improve the accuracy of the interface by referring to the related design guidelines and efficiently provide the interface. For example, the generation unit can improve the accuracy of the interface by referring to the UI / UX guidelines when generating the operation interface of a refrigerator. This allows the generation unit to improve the accuracy of the interface by referring to the relevant design guidelines, and to provide the interface efficiently.

[0047] The encryption unit can analyze the communication frequency of each IoT home appliance and select the optimal encryption algorithm. The encryption unit, for example, analyzes the communication frequency of each IoT home appliance and selects the optimal encryption algorithm. For example, the encryption unit applies an advanced encryption algorithm to home appliances with high communication frequencies. The encryption unit can also apply a simple encryption algorithm to home appliances with low communication frequencies. The encryption unit can also dynamically select the encryption algorithm depending on the communication frequency. This allows the encryption unit to select the optimal encryption algorithm depending on the communication frequency. The encryption algorithm is selected based on, for example, the Advanced Encryption Standard (AES), Rivest-Shamir-Adleman (RSA), or Transport Layer Security (TLS). For example, the encryption unit applies AES to home appliances with high communication frequencies and RSA to home appliances with low communication frequencies. The encryption unit can also use TLS to ensure the security of data communication over the Internet. This allows the encryption unit to select the optimal encryption algorithm depending on the communication frequency and efficiently ensure data security. For example, the encryption unit can apply AES when the refrigerator communicates frequently, and RSA when the air conditioner communicates infrequently. This allows the encryption unit to select the optimal encryption algorithm depending on the communication frequency, efficiently ensuring data security.

[0048] The encryption unit can select an optimal encryption method taking into account the geographical distribution of IoT home appliances when encrypting data. For example, the encryption unit selects an optimal encryption method taking into account the geographical distribution of IoT home appliances when encrypting data. For example, the encryption unit applies an encryption method suitable for a local network to geographically close home appliances. The encryption unit can also apply an encryption method suitable for communication via the Internet to geographically distant home appliances. The encryption unit can also dynamically select an encryption method based on the geographical distribution. This allows the encryption unit to select an optimal encryption method based on the geographical distribution. The encryption method selection is based on, for example, symmetric key encryption, public key encryption, or hybrid encryption. For example, the encryption unit applies symmetric key encryption to geographically close home appliances and public key encryption to geographically distant home appliances. The encryption unit can also use hybrid encryption to efficiently ensure data security. This allows the encryption unit to select an optimal encryption method based on the geographical distribution and efficiently ensure data security. For example, the encryption unit can apply symmetric key encryption to a refrigerator that is geographically close and public key encryption to an air conditioner that is geographically distant. This allows the encryption unit to select the most appropriate encryption method based on geographical distribution and efficiently ensure data security.

[0049] The authentication unit can analyze the security level of each IoT home appliance and select the optimal authentication method. The authentication unit, for example, analyzes the security level of each IoT home appliance and selects the optimal authentication method. For example, the authentication unit applies a strong authentication method to home appliances with a high security level. The authentication unit can also apply a simple authentication method to home appliances with a low security level. Furthermore, the authentication unit can dynamically select the authentication method according to the security level. This allows the authentication unit to select the optimal authentication method according to the security level. The security level analysis is performed based on, for example, vulnerability assessment and risk assessment. For example, the authentication unit evaluates the vulnerabilities of each IoT home appliance and identifies the security level based on this. The authentication unit can also perform risk assessment to evaluate the security risk of each IoT home appliance. This allows the authentication unit to select the optimal authentication method according to the security level and efficiently ensure security. For example, the authentication unit can apply a strong authentication method if the security level of a refrigerator is high, and apply a simple authentication method if the security level of an air conditioner is low. This allows the authentication unit to select the optimal authentication method according to the security level and efficiently ensure security.

[0050] The authentication unit can select an optimal authentication method during authentication, taking into account the geographical distribution of IoT home appliances. For example, the authentication unit selects an optimal authentication method during authentication, taking into account the geographical distribution of IoT home appliances. For example, the authentication unit applies an authentication method suitable for a local network to geographically close home appliances. The authentication unit can also apply an authentication method suitable for communication via the Internet to geographically distant home appliances. The authentication unit can also dynamically select an authentication method based on the geographical distribution. This allows the authentication unit to select an optimal authentication method based on the geographical distribution. The authentication method selection is based on, for example, password authentication, biometric authentication, or token authentication. For example, the authentication unit applies password authentication to geographically close home appliances and biometric authentication to geographically distant home appliances. The authentication unit can also use token authentication to perform authentication efficiently. This allows the authentication unit to select an optimal authentication method based on the geographical distribution and perform authentication efficiently. For example, the authentication unit can apply password authentication when a refrigerator is geographically close and biometric authentication when an air conditioner is geographically distant. This allows the authentication unit to select the most appropriate authentication method based on geographical distribution and perform authentication efficiently.

[0051] The registration unit can analyze the registration frequency of each IoT home appliance and select the optimal registration method. The registration unit, for example, analyzes the registration frequency of each IoT home appliance and selects the optimal registration method. For example, the registration unit applies a simple registration method to home appliances with a high registration frequency. The registration unit can also apply a detailed registration method to home appliances with a low registration frequency. Furthermore, the registration unit can dynamically select the registration method according to the registration frequency. This allows the registration unit to select the optimal registration method according to the registration frequency. The registration frequency analysis is performed based on, for example, the frequency of device addition and the user's operation history. For example, the registration unit measures the addition frequency of each IoT home appliance and identifies the registration method based on this. The registration unit can also analyze the user's operation history and evaluate the registration frequency. This allows the registration unit to select the optimal registration method according to the registration frequency and perform the registration work efficiently. For example, the registration unit can apply the simple registration method if the refrigerator is registered frequently, and the detailed registration method if the air conditioner is registered infrequently. This allows the registration unit to select the optimal registration method according to the registration frequency and perform the registration work efficiently.

[0052] The registration unit can select an optimal registration method taking into account the geographical distribution of IoT home appliances during registration. For example, the registration unit selects the optimal registration method taking into account the geographical distribution of IoT home appliances during registration. For example, the registration unit applies a registration method suitable for a local network to geographically close home appliances. The registration unit can also apply a registration method suitable for communication via the Internet to geographically distant home appliances. The registration unit can also dynamically select a registration method based on the geographical distribution. This allows the registration unit to select the optimal registration method based on the geographical distribution. The selection of the registration method is performed based on, for example, two-dimensional code scanning, Bluetooth connection, or manual input. For example, the registration unit applies two-dimensional code scanning to geographically close home appliances and Bluetooth connection to geographically distant home appliances. The registration unit can also use manual input to efficiently perform registration work. This allows the registration unit to select an optimal registration method based on the geographical distribution and efficiently perform registration work. For example, the registration unit can apply two-dimensional code scanning to a refrigerator that is geographically close and Bluetooth connection to an air conditioner that is geographically distant. This allows the registration unit to select the most appropriate registration method based on the geographical distribution and perform registration work efficiently.

[0053] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0054] IoT home appliance operation systems can learn a user's behavioral patterns and suggest optimal operation timing. For example, the system can learn a user's habit of turning on the air conditioner every morning and suggest automatically turning on the air conditioner at a specific time in the morning. Also, if a user has a habit of adjusting the lights at night, the system can suggest adjusting the lights at that time. Furthermore, the system can learn a user's habit of turning off certain home appliances when they leave the house and suggest automatically turning off the appliances when they leave the house. This allows users to receive suggestions for optimal operation timing based on their behavioral patterns, improving convenience.

[0055] IoT home appliance operation systems can monitor a user's health status and suggest home appliance operation based on that status. For example, the system can monitor a user's heart rate and body temperature and suggest adjusting the air conditioner temperature if an abnormality is detected. The system can also monitor a user's sleep status and suggest adjusting the brightness of lights or playing music to provide a comfortable sleeping environment. Furthermore, the system can monitor the user's exercise level and suggest home appliance operation to provide a relaxing environment after exercise. This allows users to receive suggestions for optimal home appliance operation based on their health status, making health management easier.

[0056] IoT home appliance operation systems can automate the operation of home appliances by taking into account the user's schedule. For example, the system can obtain the user's calendar information and automate the operation of air conditioners and lights based on meetings or outings. The system can also suggest turning home appliances on or off at specific times based on the user's schedule. Furthermore, the system can provide reminders to optimize the operation of home appliances according to the user's schedule. This allows users to receive suggestions for optimal home appliance operation based on their schedule, making their daily lives more efficient.

[0057] The IoT home appliance operation system can make suggestions to optimize energy consumption. For example, the system can monitor the energy consumption of each home appliance and suggest energy-efficient operation methods. The system can also analyze the user's energy consumption patterns and make suggestions to reduce energy consumption during peak hours. Furthermore, to promote the use of renewable energy, the system can suggest how to operate home appliances depending on the usage status of solar and wind power generation. This allows users to receive suggestions to optimize their energy consumption and achieve an environmentally friendly lifestyle.

[0058] IoT home appliance operation systems can customize the operation of home appliances based on user preferences. For example, the system can learn the user's preferred temperature and lighting brightness and suggest optimal settings. The system can also learn the user's music preferences and suggest music to play at specific times of the day. Furthermore, the system can learn the user's cooking preferences and suggest optimal settings for cooking appliances. This allows users to live a comfortable life by receiving suggestions for optimal home appliance operation based on their preferences.

[0059] The processing flow of the first embodiment will be briefly explained below.

[0060] Step 1: The acquisition unit acquires information about each IoT home appliance. For example, the acquisition unit can acquire information about refrigerators, air conditioners, lighting, etc. entered by the user. The acquisition unit can acquire information about each IoT home appliance by scanning a two-dimensional code or by using Bluetooth. For example, the acquisition unit can acquire information about the IoT home appliance by scanning a two-dimensional code with a camera. The acquisition unit can also acquire information by pairing with the IoT home appliance using Bluetooth. Step 2: The analysis unit analyzes the protocol of each IoT home appliance based on the information acquired by the acquisition unit. For example, the analysis unit analyzes the communication protocol of a refrigerator and identifies the temperature setting method. The analysis unit can also analyze the communication protocol of an air conditioner and identify the airflow adjustment method. Furthermore, the analysis unit can analyze the communication protocol of a lighting device and identify the brightness adjustment method. Step 3: The generation unit generates a unified operation interface based on the protocol analyzed by the analysis unit. For example, the generation unit enables operations such as setting the refrigerator temperature, adjusting the air conditioner's airflow, and adjusting the lighting brightness to be performed using a unified interface. The generation unit, for example, generates a GUI (graphical user interface) to enable users to operate intuitively. The generation unit can also generate an interface that is operated using voice commands. For example, the generation unit enables the user to set the refrigerator temperature by giving a voice command such as "Set the refrigerator temperature to 5 degrees."

[0061] (Example 2) An IoT home appliance control system according to an embodiment of the present invention remotely controls IoT home appliances from different manufacturers using a single app. This system begins when a user installs the app and registers each IoT home appliance in the app. The app then analyzes the protocols of each IoT home appliance and generates a unified operation interface. This allows the user to remotely control IoT home appliances from different manufacturers using a single app. For example, a user installs the app and registers IoT home appliances, such as a refrigerator, air conditioner, and lighting fixtures, in the app. The user then enters information about each IoT home appliance and saves it in the app. The app then analyzes the communication protocols of each IoT home appliance and generates a unified operation interface. For example, the user can perform operations such as setting the refrigerator temperature, adjusting the air conditioner's airflow, and adjusting the lighting brightness using a unified interface. Based on the generated operation interface, the user can remotely control IoT home appliances from different manufacturers using a single app. For example, the user can use the app to set the refrigerator temperature, adjust the air conditioner's airflow, and adjust the lighting brightness. In this way, the user can easily control IoT home appliances from different manufacturers using a single app. This mechanism allows the user to remotely control IoT home appliances from different manufacturers using a single app. This simplifies operation and improves user convenience. For example, users can control all IoT home appliances with one app, eliminating the need to use multiple apps. The unified operation interface also makes operation intuitive and simple. For example, a single app can be used to set the refrigerator temperature, adjust the air conditioner's fan speed, and adjust the brightness of the lights. This allows the IoT home appliance operation system to remotely control IoT home appliances from different manufacturers with a single app. This eliminates the need for users to use multiple apps, making operation simple and intuitive.

[0062] An IoT home appliance operation system according to an embodiment includes an acquisition unit, an analysis unit, and a generation unit. The acquisition unit acquires information about each IoT home appliance. For example, the acquisition unit can acquire information about a refrigerator, air conditioner, lighting, etc. entered by a user. The acquisition unit can acquire information about each IoT home appliance by scanning a two-dimensional code (e.g., a QR code) or using Bluetooth. For example, the acquisition unit can acquire information about the IoT home appliance by scanning a two-dimensional code with a camera. The acquisition unit can also acquire information by pairing with an IoT home appliance using Bluetooth. The analysis unit analyzes the protocol of each IoT home appliance based on the information acquired by the acquisition unit. For example, the analysis unit can analyze the communication protocol of a refrigerator to identify a temperature setting method. The analysis unit can also analyze the communication protocol of an air conditioner to identify a fan speed adjustment method. The analysis unit can also analyze the communication protocol of a lighting device to identify a brightness adjustment method. The generation unit generates a unified operation interface based on the protocols analyzed by the analysis unit. For example, the generation unit enables operations such as setting the refrigerator temperature, adjusting the airflow of an air conditioner, and adjusting the brightness of lights to be performed using a unified interface. The generation unit, for example, generates a GUI (Graphical User Interface) to enable users to operate intuitively. The generation unit can also generate an interface that is operated using voice commands. For example, the generation unit enables a user to set the refrigerator temperature by giving a voice command such as "Set the refrigerator temperature to 5 degrees." This allows the IoT home appliance operation system according to the embodiment to remotely control IoT home appliances from different manufacturers using a single app. This eliminates the need for users to use multiple apps, making operation simple and intuitive.

[0063] The IoT home appliance operation system includes an encryption unit that encrypts communications with each IoT home appliance. The encryption unit encrypts communications with each IoT home appliance. For example, the encryption unit can encrypt communications using the Advanced Encryption Standard (AES). AES is an advanced encryption standard that is widely used to ensure data security. The encryption unit can also encrypt communications using Rivest-Shamir-Adleman (RSA). RSA is a public key cryptography method used to encrypt and decrypt data. The encryption unit can also encrypt communications using Transport Layer Security (TLS). TLS is a protocol used to ensure the security of data communications over the Internet. This allows the encryption unit to encrypt communications with each IoT home appliance and ensure data security. For example, the encryption unit can encrypt communications with a refrigerator using AES, communications with an air conditioner using RSA, and communications with lighting using TLS. This allows the IoT home appliance operation system to encrypt communications with each IoT home appliance and ensure data security.

[0064] The IoT home appliance operation system includes an authentication unit that performs authentication with each IoT home appliance. The authentication unit performs authentication with each IoT home appliance. For example, the authentication unit can perform authentication with each IoT home appliance using password authentication. Password authentication is a method of authentication using a password set by the user. The authentication unit can also perform authentication with each IoT home appliance using biometric authentication. Biometric authentication is a method of authentication using biometric information such as fingerprint authentication or facial recognition. The authentication unit can also perform authentication with each IoT home appliance using token authentication. Token authentication is a method of authentication using a physical token or digital token held by the user. In this way, the authentication unit can perform authentication with each IoT home appliance and ensure security. For example, the authentication unit can perform authentication with a refrigerator using password authentication, authentication with an air conditioner using biometric authentication, and authentication with lighting using token authentication. In this way, the IoT home appliance operation system can perform authentication with each IoT home appliance and ensure security.

[0065] The IoT home appliance operation system includes a registration unit that supports the user's steps when registering an IoT home appliance. The registration unit supports the user's steps when registering an IoT home appliance. For example, the registration unit may provide a guided wizard to allow the user to register an IoT home appliance step by step. The guided wizard may guide the user through the registration process by showing the user the next steps to take. The registration unit may also display help messages to provide support if the user encounters difficulty during the registration process. The help messages may allow the user to refer to frequently asked questions and troubleshooting information. The registration unit may also provide tutorials to allow the user to learn the registration process. The tutorials may explain the registration process to the user and allow the user to learn by actually operating the appliance. In this way, the registration unit supports the user's steps when registering an IoT home appliance and simplifies the registration process. For example, the registration unit may support the refrigerator registration process with a guided wizard, the air conditioner registration process with help messages, and the lighting registration process with a tutorial. In this way, the IoT home appliance operation system allows the user to easily register IoT home appliances.

[0066] The acquisition unit can acquire information about each IoT home appliance by scanning a two-dimensional code or by using Bluetooth. The acquisition unit acquires information about each IoT home appliance by, for example, scanning a two-dimensional code. For example, the acquisition unit may use a camera to scan a two-dimensional code to acquire information about the IoT home appliance. The two-dimensional code includes identification information and setting information about the IoT home appliance, and the acquisition unit acquires information by reading the code. The acquisition unit can also acquire information about each IoT home appliance by using Bluetooth. For example, the acquisition unit may use Bluetooth to pair with the IoT home appliance and acquire information. Using Bluetooth allows information to be acquired wirelessly. This allows the acquisition unit to acquire information about each IoT home appliance by scanning a two-dimensional code or by using Bluetooth. For example, the acquisition unit can scan the two-dimensional code on a refrigerator to acquire information, pair with an air conditioner via Bluetooth to acquire information, and scan the two-dimensional code on a light to acquire information. This allows the acquisition unit to acquire information about IoT home appliances in a variety of ways.

[0067] The generation unit can abstract the functions of each IoT home appliance and set common operation items. For example, the generation unit abstracts the functions of each IoT home appliance and sets common operation items. For example, the generation unit abstracts operations such as refrigerator temperature setting, air conditioner airflow adjustment, and lighting brightness adjustment and sets them as common operation items. This allows users to operate IoT home appliances from different manufacturers with a unified operation interface. For example, the generation unit abstracts the refrigerator temperature setting as "temperature adjustment," the air conditioner airflow adjustment as "airflow adjustment," and the lighting brightness adjustment as "brightness adjustment." This allows users to operate IoT home appliances from different manufacturers using common operation items such as "temperature adjustment," "airflow adjustment," and "brightness adjustment." Furthermore, the generation unit can categorize the functions of each IoT home appliance when setting common operation items. For example, the generation unit categorizes home appliances such as refrigerators, air conditioners, and lighting into categories such as "temperature adjustment," "airflow adjustment," and "brightness adjustment." This allows users to select and operate operation items for each category. This allows the generation unit to abstract the functions of each IoT home appliance and set common operation items, making it possible to operate IoT home appliances from different manufacturers using a unified operation interface.

[0068] The acquisition unit can estimate a user's emotions and adjust the timing of information acquisition of the IoT home appliance based on the estimated user emotions. For example, the acquisition unit can estimate a user's emotions and adjust the timing of information acquisition based on the estimated emotions. For example, if the user is feeling stressed, the acquisition unit can reduce the frequency of information acquisition and minimize notifications. Furthermore, if the user is relaxed, the acquisition unit can increase the frequency of information acquisition and provide more detailed information. Furthermore, if the user is in a hurry, the acquisition unit can prioritize acquiring only important information and notify it quickly. This allows the acquisition unit to adjust the timing of information acquisition according to the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. For example, the acquisition unit can capture a user's facial expression with a camera and input it into the generation AI to estimate emotions. The acquisition unit can also record the user's voice and input it into the generation AI to estimate emotions. This allows the acquisition unit to adjust the timing of information acquisition in accordance with the user's emotions and provide information at the optimal timing for the user.

[0069] The acquisition unit can analyze the use frequency of each IoT home appliance and select the optimal information acquisition method. For example, the acquisition unit can analyze the use frequency of each IoT home appliance and select the optimal information acquisition method. For example, the acquisition unit can prioritize acquisition of information from frequently used home appliances. The acquisition unit can also reduce the acquisition of information from infrequently used home appliances. Furthermore, the acquisition unit can dynamically adjust the timing of information acquisition according to the use frequency. This allows the acquisition unit to analyze the use frequency of each IoT home appliance and select the optimal information acquisition method. The analysis of the use frequency is performed, for example, using log data analysis or statistical methods. For example, the acquisition unit can collect the use history of each IoT home appliance as log data and analyze it to identify the use frequency. The acquisition unit can also evaluate the use frequency of each IoT home appliance using statistical methods. This allows the acquisition unit to optimize the information acquisition method according to the use frequency and acquire information efficiently. For example, the acquisition unit can prioritize acquisition of information from the refrigerator if the refrigerator is used frequently, and reduce acquisition of information from the air conditioner if the air conditioner is used infrequently. This allows the acquisition unit to optimize the information acquisition method according to the frequency of use, and acquire information efficiently.

[0070] The acquisition unit can filter information about IoT home appliances based on the user's current living situation and usage pattern when acquiring the information. For example, the acquisition unit can filter information about IoT home appliances based on the user's current living situation and usage pattern when acquiring the information. For example, when the user is at home, the acquisition unit acquires detailed information about the status of the home appliance. Furthermore, when the user is out, the acquisition unit can acquire only important information. Furthermore, the acquisition unit can acquire only necessary information based on the user's usage pattern. This enables the acquisition unit to acquire information according to the user's living situation and usage pattern. The acquisition of living situations is performed based on, for example, daily behavior patterns and schedule information. For example, the acquisition unit acquires the user's schedule information and determines whether the user is at home or out. The acquisition unit can also analyze the user's behavior patterns to identify the user's current living situation. The acquisition of usage patterns is performed based on, for example, device usage history and operation logs. For example, the acquisition unit collects usage history of each IoT home appliance and analyzes it to identify the usage pattern. The acquisition unit can also analyze operation logs to identify the user's usage pattern. This enables the acquisition unit to acquire information according to the user's living situation and usage pattern, and enables the user to efficiently acquire information necessary for the user.

[0071] The acquisition unit can estimate the user's emotions and determine the priority of information to be acquired from the IoT home appliance based on the estimated user emotions. The acquisition unit, for example, estimates the user's emotions and determines the priority of information based on the estimated emotions. For example, when the user is feeling stressed, the acquisition unit prioritizes acquiring only important information. Furthermore, when the user is relaxed, the acquisition unit can prioritize acquiring detailed information. Furthermore, when the user is in a hurry, the acquisition unit can prioritize information that can be acquired quickly. This allows the acquisition unit to determine the priority of information according to the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. For example, the acquisition unit can capture the user's facial expression with a camera and input it into the generation AI to estimate the emotion. The acquisition unit can also record the user's voice and input it into the generation AI to estimate the emotion. This allows the acquisition unit to determine the priority of information according to the user's emotions and prioritize acquiring information that is important to the user.

[0072] When acquiring information about IoT home appliances, the acquisition unit can prioritize acquiring highly relevant information by taking into account the user's geographical location information. For example, when acquiring information about IoT home appliances, the acquisition unit prioritizes acquiring highly relevant information by taking into account the user's geographical location information. For example, when the user is at home, the acquisition unit prioritizes acquiring information about home appliances. Furthermore, when the user is at work, the acquisition unit can prioritize acquiring information about workplace appliances. Furthermore, when the user is traveling, the acquisition unit can prioritize acquiring information about appliances at the user's travel destination. This allows the acquisition unit to acquire highly relevant information based on the user's geographical location information. The acquisition of geographical location information is performed based on, for example, GPS data or Wi-Fi location information. For example, the acquisition unit acquires GPS data from the user's smartphone to identify the user's current location. Furthermore, the acquisition unit can also identify the user's current location using Wi-Fi location information. This allows the acquisition unit to acquire highly relevant information based on the user's geographical location information and prioritize providing information necessary to the user.

[0073] The acquisition unit can analyze the user's social media activity and acquire related information when acquiring information about an IoT home appliance. For example, when acquiring information about an IoT home appliance, the acquisition unit analyzes the user's social media activity and acquires related information. For example, if the user mentions a specific home appliance on social media, the acquisition unit preferentially acquires information about the home appliance. The acquisition unit can also acquire information about home appliances of interest from the user's social media activity. Furthermore, the acquisition unit can acquire information about related home appliances based on the content of the user's social media posts. This allows the acquisition unit to acquire related information based on the user's social media activity. The analysis of social media activity is performed based on, for example, the content of posts, like history, follower information, etc. For example, the acquisition unit collects post content from the user's social media account and analyzes it to identify information about related home appliances. The acquisition unit can also analyze the user's like history to identify information about home appliances of interest. Furthermore, the acquisition unit can analyze follower information to identify information about home appliances in which the user is interested. This allows the acquisition unit to acquire related information based on the user's social media activity and provide the user with necessary information.

[0074] The analysis unit can estimate the user's emotions and adjust the protocol analysis method based on the estimated user emotions. For example, the analysis unit can estimate the user's emotions and adjust the protocol analysis method based on the estimated emotions. For example, if the user is feeling stressed, the analysis unit can reduce the frequency of analysis and analyze only important protocols. The analysis unit can also perform detailed protocol analysis when the user is relaxed. Furthermore, if the user is in a hurry, the analysis unit can prioritize protocols that can be analyzed quickly. This allows the analysis unit to adjust the protocol analysis method according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. For example, the analysis unit can capture the user's facial expressions with a camera and input them into the generation AI to estimate emotions. The analysis unit can also record the user's voice and input them into the generation AI to estimate emotions. This allows the analysis unit to adjust the protocol analysis method according to the user's feelings and analyze the protocol in a method that is optimal for the user.

[0075] The analysis unit can analyze the complexity of the protocol of each IoT home appliance and select the optimal analysis algorithm. For example, the analysis unit analyzes the complexity of the protocol of each IoT home appliance and selects the optimal analysis algorithm. For example, the analysis unit applies an advanced analysis algorithm to home appliances with complex protocols. The analysis unit can also apply a simple analysis algorithm to home appliances with simple protocols. Furthermore, the analysis unit can dynamically select an analysis algorithm depending on the complexity of the protocol. This allows the analysis unit to select the optimal analysis algorithm depending on the complexity of the protocol. Protocol complexity analysis is performed based on, for example, the length of the message or the number of fields. For example, the analysis unit measures the length of the protocol message of each IoT home appliance and evaluates the complexity based on this. The analysis unit can also count the number of fields in the protocol message and evaluate the complexity based on this. This allows the analysis unit to select the optimal analysis algorithm depending on the complexity of the protocol and analyze the protocol efficiently. For example, the analysis unit can apply an advanced analysis algorithm if the protocol of a refrigerator is complex, and apply a simple analysis algorithm if the protocol of an air conditioner is simple. This allows the analysis unit to select the optimal analysis algorithm depending on the complexity of the protocol, and to analyze the protocol efficiently.

[0076] The analysis unit can improve the analysis accuracy by referring to past analysis data when analyzing a protocol. The analysis unit can improve the analysis accuracy by referring to past analysis data, for example, when analyzing a protocol. For example, the analysis unit learns protocol patterns based on the past analysis data and improves the analysis accuracy. The analysis unit can also refer to the past analysis data to efficiently analyze home appliances with similar protocols. Furthermore, the analysis unit can utilize the past analysis data to improve the accuracy of the analysis algorithm. In this way, the analysis unit can improve the analysis accuracy by referring to the past analysis data. The reference to the past analysis data is performed based on, for example, past log data and a history of analysis results. For example, the analysis unit collects past log data and analyzes it to identify protocol patterns. The analysis unit can also refer to a history of past analysis results to efficiently analyze home appliances with similar protocols. In this way, the analysis unit can improve the analysis accuracy by referring to the past analysis data and efficiently analyze protocols. For example, the analysis unit can refer to past analysis data of a refrigerator and learn protocol patterns to improve the analysis accuracy. In this way, the analysis unit can improve the analysis accuracy by referring to the past analysis data and efficiently analyze protocols.

[0077] The analysis unit can estimate the user's emotions and adjust the display method of the analysis results based on the estimated user emotions. For example, the analysis unit can estimate the user's emotions and adjust the display method of the analysis results based on the estimated emotions. For example, if the user is nervous, the analysis unit can provide a simple, highly visible display method. If the user is relaxed, the analysis unit can provide a display method that includes detailed information. Furthermore, if the user is in a hurry, the analysis unit can provide a display method that focuses on the main points. This allows the analysis unit to adjust the display method of the analysis results according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. For example, the analysis unit can capture the user's facial expressions with a camera and input them into the generation AI to estimate emotions. The analysis unit can also record the user's voice and input them into the generation AI to estimate emotions. This allows the analysis unit to adjust the display method of the analysis results in accordance with the user's emotions, and provide the analysis results in a manner that is optimal for the user.

[0078] The analysis unit can take into account the geographical distribution of IoT home appliances when analyzing protocols. For example, the analysis unit can perform protocol analysis while taking into account the geographical distribution of IoT home appliances. For example, the analysis unit prioritizes analysis of protocols of geographically close home appliances. The analysis unit can also thin out protocol analysis of geographically distant home appliances. Furthermore, the analysis unit can dynamically adjust the analysis priority based on the geographical distribution. This allows the analysis unit to perform protocol analysis based on the geographical distribution, thereby enabling efficient analysis. The geographical distribution can be taken into account, for example, based on the installation location of devices and their distribution by region. For example, the analysis unit can identify the installation location of each IoT home appliance and evaluate the geographical distribution based on this. The analysis unit can also analyze the distribution of devices by region and determine the analysis priority based on this. This allows the analysis unit to perform protocol analysis based on the geographical distribution and efficiently analyze protocols. For example, the analysis unit can identify the location of a refrigerator and determine the protocol analysis priority based on this. This allows the analysis unit to perform protocol analysis based on the geographical distribution and efficiently analyze protocols.

[0079] The analysis unit can improve the accuracy of the analysis by referring to related technical literature during protocol analysis. For example, the analysis unit improves the accuracy of the analysis by referring to related technical literature during protocol analysis. For example, the analysis unit learns protocol patterns based on related technical literature and improves the analysis accuracy. The analysis unit can also efficiently analyze home appliances with similar protocols by referring to technical literature. Furthermore, the analysis unit can utilize technical literature to improve the accuracy of the analysis algorithm. As a result, the analysis unit can improve the accuracy of the analysis by referring to related technical literature. Referencing technical literature is performed based on, for example, patent documents, academic papers, and technical reports. For example, the analysis unit collects patent documents and analyzes them to identify protocol patterns. The analysis unit can also efficiently analyze home appliances with similar protocols by referring to academic papers. Furthermore, the analysis unit can utilize technical reports to improve the accuracy of the analysis algorithm. As a result, the analysis unit can improve the accuracy of the analysis by referring to related technical literature and efficiently analyze protocols. For example, the analysis unit can improve the accuracy of the analysis by referring to related patent documents when analyzing a refrigerator protocol. This allows the analysis unit to improve the accuracy of analysis by referring to related technical literature, and to analyze protocols efficiently.

[0080] The generation unit can estimate the user's emotions and adjust the design of the operation interface based on the estimated user emotions. For example, the generation unit can estimate the user's emotions and adjust the design of the operation interface based on the estimated emotions. For example, if the user is nervous, the generation unit can provide an interface with subdued colors to reduce visual stress. Furthermore, if the user is having fun, the generation unit can provide an interface with bright colors to make input work more enjoyable. Furthermore, if the user is tired, the generation unit can provide a simple, highly visible interface to make input work easier. This allows the generation unit to adjust the design of the operation interface according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. For example, the generation unit can capture the user's facial expressions with a camera and input them into the generation AI to estimate emotions. The generation unit can also record the user's voice and input them into the generation AI to estimate emotions. This allows the generation unit to adjust the design of the operation interface in accordance with the user's emotions and provide the user with an optimal design.

[0081] The generation unit can analyze the functions of each IoT home appliance in detail and generate an optimal operation interface. For example, the generation unit can analyze the functions of each IoT home appliance in detail and generate an optimal operation interface. For example, the generation unit can analyze the functions of each home appliance in detail and set common operation items. The generation unit can also generate a customizable operation interface according to the function of each home appliance. Furthermore, the generation unit can abstract the functions of each home appliance and provide an intuitive operation interface. This allows the generation unit to generate an optimal operation interface according to the function of each IoT home appliance. The operation interface is generated based on, for example, a touchscreen, voice commands, or remote control. For example, the generation unit can provide an intuitive operation interface using a touchscreen. The generation unit can also provide an operation interface using voice commands. Furthermore, the generation unit can provide an operation interface using a remote control. This allows the generation unit to generate an optimal operation interface according to the function of each IoT home appliance and provide an interface that is easy for the user to use. For example, the generation unit can analyze the functions of a refrigerator in detail and generate an operation interface for temperature adjustment. This allows the generation unit to generate an optimal operation interface according to the functions of each IoT home appliance, providing an interface that is easy for users to use.

[0082] The generation unit can provide an optimal interface by referring to the user's past operation history when generating the operation interface. For example, the generation unit can provide an optimal interface by referring to the user's past operation history when generating the operation interface. For example, the generation unit can propose an optimal operation interface based on the user's past operation history. The generation unit can also analyze the user's past operation patterns and provide a customizable interface. Furthermore, the generation unit can generate an intuitive operation interface by referring to the user's past operation history. As a result, the generation unit can provide an optimal operation interface based on the user's past operation history. The reference to the past operation history is based on, for example, an operation log, use frequency, or operation pattern. For example, the generation unit collects and analyzes the user's operation log to identify an optimal operation interface. The generation unit can also analyze use frequency and prioritize displaying frequently used operation items. Furthermore, the generation unit can analyze operation patterns and generate an interface that the user can intuitively operate. As a result, the generation unit can provide an optimal operation interface based on the user's past operation history and provide an interface that is easy for the user to use. For example, the generation unit can optimize the temperature adjustment operation interface by referring to the operation history of a refrigerator. This allows the generation unit to provide an optimal operation interface based on the user's past operation history, and to provide an interface that is easy for the user to use.

[0083] The generation unit can estimate the user's emotion and adjust the display method of the operation interface based on the estimated user emotion. For example, the generation unit can estimate the user's emotion and adjust the display method of the operation interface based on the estimated emotion. For example, if the user is nervous, the generation unit can provide a simple, highly visible display method. If the user is relaxed, the generation unit can also provide a display method including detailed information. Furthermore, if the user is in a hurry, the generation unit can also provide a display method that focuses on the main points. This allows the generation unit to adjust the display method of the operation interface according to the user's emotion. Emotion estimation is realized using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. For example, the generation unit can capture the user's facial expression with a camera and input it into the generation AI to estimate the emotion. The generation unit can also record the user's voice and input it into the generation AI to estimate the emotion. This allows the generation unit to adjust the display method of the operation interface in accordance with the user's emotions, and provide the most suitable display method for the user.

[0084] The generation unit can generate an operation interface by taking into account the geographical distribution of IoT home appliances. For example, the generation unit generates the operation interface by taking into account the geographical distribution of IoT home appliances. For example, the generation unit prioritizes generating operation interfaces for geographically close home appliances. The generation unit can also thin out the generation of operation interfaces for geographically distant home appliances. Furthermore, the generation unit can dynamically adjust the generation priority of operation interfaces based on the geographical distribution. This allows the generation unit to generate an optimal operation interface based on the geographical distribution. The geographical distribution can be taken into account, for example, based on the installation locations of devices and their distribution by region. For example, the generation unit can identify the installation locations of each IoT home appliance and evaluate the geographical distribution based on this. The generation unit can also analyze the distribution of devices by region and determine the generation priority of operation interfaces based on this. This allows the generation unit to generate an optimal operation interface based on the geographical distribution and efficiently provide the interface. For example, the generation unit can identify the locations where refrigerators are installed and determine the generation priority of operation interfaces based on this. This allows the generation unit to generate an optimal operation interface based on the geographical distribution and efficiently provide the interface.

[0085] The generation unit can improve the accuracy of the interface by referring to related design guidelines when generating the operation interface. For example, the generation unit can improve the accuracy of the interface by referring to related design guidelines when generating the operation interface. For example, the generation unit improves the accuracy of the operation interface based on the related design guidelines. The generation unit can also provide an intuitive operation interface by referring to the design guidelines. Furthermore, the generation unit can maintain consistency of the operation interface by utilizing the design guidelines. In this way, the generation unit can improve the accuracy of the interface by referring to the related design guidelines. The reference to the design guidelines is performed based on, for example, UI / UX guidelines or brand guidelines. For example, the generation unit collects UI / UX guidelines and improves the accuracy of the operation interface based on the UI / UX guidelines. The generation unit can also provide an intuitive operation interface by referring to the brand guidelines. Furthermore, the generation unit can maintain consistency of the operation interface by utilizing the design guidelines. In this way, the generation unit can improve the accuracy of the interface by referring to the related design guidelines and efficiently provide the interface. For example, the generation unit can improve the accuracy of the interface by referring to the UI / UX guidelines when generating the operation interface of a refrigerator. This allows the generation unit to improve the accuracy of the interface by referring to the relevant design guidelines, and to provide the interface efficiently.

[0086] The encryption unit can estimate the user's emotions and adjust the encryption strength based on the estimated user emotions. For example, the encryption unit can estimate the user's emotions and adjust the encryption strength based on the estimated emotions. For example, if the user is feeling stressed, the encryption unit can increase the encryption strength to enhance security. The encryption unit can also moderately adjust the encryption strength when the user is relaxed. Furthermore, if the user is in a hurry, the encryption unit can select a method that allows for quick encryption. This allows the encryption unit to adjust the encryption strength according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to these examples. For example, the encryption unit can capture the user's facial expression with a camera and input it into the generation AI to estimate the emotion. The encryption unit can also record the user's voice and input it into the generation AI to estimate the emotion. This allows the encryption unit to adjust the encryption strength according to the user's emotions and provide optimal security for the user.

[0087] The encryption unit can analyze the communication frequency of each IoT home appliance and select the optimal encryption algorithm. The encryption unit, for example, analyzes the communication frequency of each IoT home appliance and selects the optimal encryption algorithm. For example, the encryption unit applies an advanced encryption algorithm to home appliances with high communication frequencies. The encryption unit can also apply a simple encryption algorithm to home appliances with low communication frequencies. The encryption unit can also dynamically select the encryption algorithm depending on the communication frequency. This allows the encryption unit to select the optimal encryption algorithm depending on the communication frequency. The encryption algorithm is selected based on, for example, the Advanced Encryption Standard (AES), Rivest-Shamir-Adleman (RSA), or Transport Layer Security (TLS). For example, the encryption unit applies AES to home appliances with high communication frequencies and RSA to home appliances with low communication frequencies. The encryption unit can also use TLS to ensure the security of data communication over the Internet. This allows the encryption unit to select the optimal encryption algorithm depending on the communication frequency and efficiently ensure data security. For example, the encryption unit can apply AES when the refrigerator communicates frequently, and RSA when the air conditioner communicates infrequently. This allows the encryption unit to select the optimal encryption algorithm depending on the communication frequency, efficiently ensuring data security.

[0088] The encryption unit can estimate a user's emotions and determine encryption priorities based on the estimated user emotions. For example, the encryption unit can estimate a user's emotions and determine encryption priorities based on the estimated emotions. For example, if a user is feeling stressed, the encryption unit can prioritize encryption of important communications. Furthermore, if a user is relaxed, the encryption unit can equally encrypt all communications. Furthermore, if a user is in a hurry, the encryption unit can prioritize communications that can be encrypted quickly. This allows the encryption unit to determine encryption priorities based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. For example, the encryption unit can capture a user's facial expression with a camera and input it into the generation AI to estimate emotions. The encryption unit can also record the user's voice and input it into the generation AI to estimate emotions. This allows the encryption unit to determine encryption priorities based on the user's emotions and provide optimal security for the user.

[0089] The encryption unit can select an optimal encryption method taking into account the geographical distribution of IoT home appliances when encrypting data. For example, the encryption unit selects an optimal encryption method taking into account the geographical distribution of IoT home appliances when encrypting data. For example, the encryption unit applies an encryption method suitable for a local network to geographically close home appliances. The encryption unit can also apply an encryption method suitable for communication via the Internet to geographically distant home appliances. The encryption unit can also dynamically select an encryption method based on the geographical distribution. This allows the encryption unit to select an optimal encryption method based on the geographical distribution. The encryption method selection is based on, for example, symmetric key encryption, public key encryption, or hybrid encryption. For example, the encryption unit applies symmetric key encryption to geographically close home appliances and public key encryption to geographically distant home appliances. The encryption unit can also use hybrid encryption to efficiently ensure data security. This allows the encryption unit to select an optimal encryption method based on the geographical distribution and efficiently ensure data security. For example, the encryption unit can apply symmetric key encryption to a refrigerator that is geographically close and public key encryption to an air conditioner that is geographically distant. This allows the encryption unit to select the most appropriate encryption method based on geographical distribution and efficiently ensure data security.

[0090] The authentication unit can estimate the user's emotions and adjust the authentication method based on the estimated user emotions. For example, the authentication unit estimates the user's emotions and adjusts the authentication method based on the estimated emotions. For example, if the user is stressed, the authentication unit can provide a simple authentication method to speed up the authentication procedure. Furthermore, if the user is relaxed, the authentication unit can provide a detailed authentication method to enhance security. Furthermore, if the user is in a hurry, the authentication unit can select a method that allows for quick authentication. This allows the authentication unit to adjust the authentication method according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generation AI. The generation AI can be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to these examples. For example, the authentication unit can capture the user's facial expressions with a camera and input them into the generation AI to estimate emotions. Furthermore, the authentication unit can record the user's voice and input them into the generation AI to estimate emotions. This allows the authentication unit to adjust the authentication method according to the user's emotions and provide the optimal authentication procedure for the user.

[0091] The authentication unit can analyze the security level of each IoT home appliance and select the optimal authentication method. The authentication unit, for example, analyzes the security level of each IoT home appliance and selects the optimal authentication method. For example, the authentication unit applies a strong authentication method to home appliances with a high security level. The authentication unit can also apply a simple authentication method to home appliances with a low security level. Furthermore, the authentication unit can dynamically select the authentication method according to the security level. This allows the authentication unit to select the optimal authentication method according to the security level. The security level analysis is performed based on, for example, vulnerability assessment and risk assessment. For example, the authentication unit evaluates the vulnerabilities of each IoT home appliance and identifies the security level based on this. The authentication unit can also perform risk assessment to evaluate the security risk of each IoT home appliance. This allows the authentication unit to select the optimal authentication method according to the security level and efficiently ensure security. For example, the authentication unit can apply a strong authentication method if the security level of a refrigerator is high, and apply a simple authentication method if the security level of an air conditioner is low. This allows the authentication unit to select the optimal authentication method according to the security level and efficiently ensure security.

[0092] The authentication unit can estimate the user's emotions and determine the authentication priority based on the estimated user emotions. The authentication unit, for example, estimates the user's emotions and determines the authentication priority based on the estimated emotions. For example, if the user is feeling stressed, the authentication unit can prioritize important authentications. Also, if the user is relaxed, the authentication unit can perform all authentications equally. Furthermore, if the user is in a hurry, the authentication unit can prioritize items that can be authenticated quickly. This allows the authentication unit to determine the authentication priority based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to these examples. For example, the authentication unit can capture the user's facial expression with a camera and input it into the generation AI to estimate the emotion. Also, the authentication unit can record the user's voice and input it into the generation AI to estimate the emotion. This allows the authentication unit to determine the authentication priority based on the user's emotions and provide the optimal authentication procedure for the user.

[0093] The authentication unit can select an optimal authentication method during authentication, taking into account the geographical distribution of IoT home appliances. For example, the authentication unit selects an optimal authentication method during authentication, taking into account the geographical distribution of IoT home appliances. For example, the authentication unit applies an authentication method suitable for a local network to geographically close home appliances. The authentication unit can also apply an authentication method suitable for communication via the Internet to geographically distant home appliances. The authentication unit can also dynamically select an authentication method based on the geographical distribution. This allows the authentication unit to select an optimal authentication method based on the geographical distribution. The authentication method selection is based on, for example, password authentication, biometric authentication, or token authentication. For example, the authentication unit applies password authentication to geographically close home appliances and biometric authentication to geographically distant home appliances. The authentication unit can also use token authentication to perform authentication efficiently. This allows the authentication unit to select an optimal authentication method based on the geographical distribution and perform authentication efficiently. For example, the authentication unit can apply password authentication when a refrigerator is geographically close and biometric authentication when an air conditioner is geographically distant. This allows the authentication unit to select the most appropriate authentication method based on geographical distribution and perform authentication efficiently.

[0094] The registration unit can estimate the user's emotions and adjust the registration procedure based on the estimated user emotions. For example, the registration unit can estimate the user's emotions and adjust the registration procedure based on the estimated emotions. For example, if the user is stressed, the registration unit can provide a simple registration procedure to quickly complete the registration process. Alternatively, if the user is relaxed, the registration unit can provide a detailed registration procedure to carefully complete the registration process. Furthermore, if the user is in a hurry, the registration unit can select a procedure that allows for quick registration. This allows the registration unit to adjust the registration procedure according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to these examples. For example, the registration unit can capture the user's facial expression with a camera and input it into the generation AI to estimate the emotion. Alternatively, the registration unit can record the user's voice and input it into the generation AI to estimate the emotion. This allows the registration unit to adjust the registration procedure according to the user's emotions and provide the optimal registration procedure for the user.

[0095] The registration unit can analyze the registration frequency of each IoT home appliance and select the optimal registration method. The registration unit, for example, analyzes the registration frequency of each IoT home appliance and selects the optimal registration method. For example, the registration unit applies a simple registration method to home appliances with a high registration frequency. The registration unit can also apply a detailed registration method to home appliances with a low registration frequency. Furthermore, the registration unit can dynamically select the registration method according to the registration frequency. This allows the registration unit to select the optimal registration method according to the registration frequency. The registration frequency analysis is performed based on, for example, the frequency of device addition and the user's operation history. For example, the registration unit measures the addition frequency of each IoT home appliance and identifies the registration method based on this. The registration unit can also analyze the user's operation history and evaluate the registration frequency. This allows the registration unit to select the optimal registration method according to the registration frequency and perform the registration work efficiently. For example, the registration unit can apply the simple registration method if the refrigerator is registered frequently, and the detailed registration method if the air conditioner is registered infrequently. This allows the registration unit to select the optimal registration method according to the registration frequency and perform the registration work efficiently.

[0096] The registration unit can estimate the user's emotions and determine the registration priority based on the estimated user emotions. The registration unit, for example, estimates the user's emotions and determines the registration priority based on the estimated emotions. For example, when the user is feeling stressed, the registration unit prioritizes the registration of important home appliances. Furthermore, when the user is relaxed, the registration unit can equally register all home appliances. Furthermore, when the user is in a hurry, the registration unit can prioritize home appliances that can be registered quickly. This allows the registration unit to determine the registration priority according to the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. For example, the registration unit can capture the user's facial expression with a camera and input it into the generation AI to estimate the emotion. The registration unit can also record the user's voice and input it into the generation AI to estimate the emotion. This allows the registration unit to determine the registration priority based on the user's emotions and provide an optimal registration procedure for the user.

[0097] The registration unit can select an optimal registration method taking into account the geographical distribution of IoT home appliances during registration. For example, the registration unit selects the optimal registration method taking into account the geographical distribution of IoT home appliances during registration. For example, the registration unit applies a registration method suitable for a local network to geographically close home appliances. The registration unit can also apply a registration method suitable for communication via the Internet to geographically distant home appliances. The registration unit can also dynamically select a registration method based on the geographical distribution. This allows the registration unit to select the optimal registration method based on the geographical distribution. The selection of the registration method is performed based on, for example, two-dimensional code scanning, Bluetooth connection, or manual input. For example, the registration unit applies two-dimensional code scanning to geographically close home appliances and Bluetooth connection to geographically distant home appliances. The registration unit can also use manual input to efficiently perform registration work. This allows the registration unit to select an optimal registration method based on the geographical distribution and efficiently perform registration work. For example, the registration unit can apply two-dimensional code scanning to a refrigerator that is geographically close and Bluetooth connection to an air conditioner that is geographically distant. This allows the registration unit to select the most appropriate registration method based on the geographical distribution and perform registration work efficiently. === Hard Collateral 1-1 === Each of the multiple elements, including the acquisition unit, analysis unit, generation unit, encryption unit, authentication unit, and registration unit, described above, is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the acquisition unit acquires information about each IoT home appliance using the camera 42 or Bluetooth of the smart device 14. The analysis unit analyzes the protocol of each IoT home appliance using the specific processing unit 290 of the data processing device 12. The generation unit generates a unified operation interface using the control unit 46A of the smart device 14. The encryption unit encrypts communication with each IoT home appliance using the specific processing unit 290 of the data processing device 12. The authentication unit authenticates each IoT home appliance using the control unit 46A of the smart device 14. The registration unit supports the procedure when a user registers an IoT home appliance using the control unit 46A of the smart device 14. === Hard Collateral 1-2 === Each of the multiple elements including the above-mentioned acquisition unit, analysis unit, generation unit, encryption unit, authentication unit, and registration unit is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the acquisition unit acquires information about each IoT home appliance using the camera 42 or Bluetooth of the smart glasses 214. The analysis unit analyzes the protocol of each IoT home appliance using the specific processing unit 290 of the data processing device 12. The generation unit generates a unified operation interface using the control unit 46A of the smart glasses 214. The encryption unit encrypts communication with each IoT home appliance using the specific processing unit 290 of the data processing device 12. The authentication unit performs authentication with each IoT home appliance using the control unit 46A of the smart glasses 214. The registration unit supports the procedure when a user registers an IoT home appliance using the control unit 46A of the smart glasses 214. === Hard Collateral 1-3 === Each of the multiple elements including the acquisition unit, analysis unit, generation unit, encryption unit, authentication unit, and registration unit described above is realized, for example, by at least one of the headset type terminal 314 and the data processing device 12. For example, the acquisition unit acquires information about each IoT home appliance using the camera 42 or Bluetooth of the headset type terminal 314. The analysis unit analyzes the protocol of each IoT home appliance using the specific processing unit 290 of the data processing device 12. The generation unit generates a unified operation interface using the control unit 46A of the headset type terminal 314. The encryption unit encrypts communication with each IoT home appliance using the specific processing unit 290 of the data processing device 12. The authentication unit authenticates each IoT home appliance using the control unit 46A of the headset type terminal 314. The registration unit supports the procedure when a user registers an IoT home appliance using the control unit 46A of the headset type terminal 314. === Hard Collateral 1-4 === Each of the multiple elements including the above-mentioned acquisition unit, analysis unit, generation unit, encryption unit, authentication unit, and registration unit is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the acquisition unit acquires information about each IoT home appliance using the camera 42 of the robot 414 or Bluetooth. The analysis unit analyzes the protocol of each IoT home appliance using the specific processing unit 290 of the data processing device 12. The generation unit generates a unified operation interface using the control unit 46A of the robot 414. The encryption unit encrypts communication with each IoT home appliance using the specific processing unit 290 of the data processing device 12. The authentication unit authenticates each IoT home appliance using the control unit 46A of the robot 414. The registration unit supports the procedure when a user registers an IoT home appliance using the control unit 46A of the robot 414.

[0098] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0099] IoT home appliance operation systems can learn a user's behavioral patterns and suggest optimal operation timing. For example, the system can learn a user's habit of turning on the air conditioner every morning and suggest automatically turning on the air conditioner at a specific time in the morning. Also, if a user has a habit of adjusting the lights at night, the system can suggest adjusting the lights at that time. Furthermore, the system can learn a user's habit of turning off certain home appliances when they leave the house and suggest automatically turning off the appliances when they leave the house. This allows users to receive suggestions for optimal operation timing based on their behavioral patterns, improving convenience.

[0100] IoT home appliance operation systems can monitor a user's health status and suggest home appliance operation based on that status. For example, the system can monitor a user's heart rate and body temperature and suggest adjusting the air conditioner temperature if an abnormality is detected. The system can also monitor a user's sleep status and suggest adjusting the brightness of lights or playing music to provide a comfortable sleeping environment. Furthermore, the system can monitor the user's exercise level and suggest home appliance operation to provide a relaxing environment after exercise. This allows users to receive suggestions for optimal home appliance operation based on their health status, making health management easier.

[0101] IoT home appliance operation systems can automate the operation of home appliances by taking into account the user's schedule. For example, the system can obtain the user's calendar information and automate the operation of air conditioners and lights based on meetings or outings. The system can also suggest turning home appliances on or off at specific times based on the user's schedule. Furthermore, the system can provide reminders to optimize the operation of home appliances according to the user's schedule. This allows users to receive suggestions for optimal home appliance operation based on their schedule, making their daily lives more efficient.

[0102] The IoT home appliance operation system can make suggestions to optimize energy consumption. For example, the system can monitor the energy consumption of each home appliance and suggest energy-efficient operation methods. The system can also analyze the user's energy consumption patterns and make suggestions to reduce energy consumption during peak hours. Furthermore, to promote the use of renewable energy, the system can suggest how to operate home appliances depending on the usage status of solar and wind power generation. This allows users to receive suggestions to optimize their energy consumption and achieve an environmentally friendly lifestyle.

[0103] IoT home appliance operation systems can customize the operation of home appliances based on user preferences. For example, the system can learn the user's preferred temperature and lighting brightness and suggest optimal settings. The system can also learn the user's music preferences and suggest music to play at specific times of the day. Furthermore, the system can learn the user's cooking preferences and suggest optimal settings for cooking appliances. This allows users to live a comfortable life by receiving suggestions for optimal home appliance operation based on their preferences.

[0104] An IoT home appliance operation system can estimate a user's emotions and adjust the operation of home appliances based on the estimated emotions. For example, if a user is feeling stressed, the system can adjust the brightness of the lights or the music playback to provide a relaxing environment. If a user is tired, the system can also adjust the temperature of the air conditioner or the brightness of the lights to provide a comfortable sleeping environment. Furthermore, if a user is happy, the system can adjust the music playback or the color of the lights to create a fun atmosphere. This allows users to live a comfortable life by receiving suggestions for optimal home appliance operation based on their emotions.

[0105] The IoT home appliance operation system can estimate a user's emotions and adjust the content and timing of notifications based on the estimated emotions. For example, if a user is feeling stressed, the system can prioritize displaying only important notifications and suppressing other notifications. The system can also provide detailed notifications when the user is relaxed. Furthermore, if the user is in a hurry, the system can prioritize displaying notifications that require a quick response. This allows users to receive notifications that are optimal for their emotions, reducing stress and enabling efficient information management.

[0106] The IoT home appliance operation system can estimate the user's emotions and adjust the operation mode of the appliances based on the estimated emotions. For example, if the user is relaxed, the system can disable the energy-saving mode of the air conditioner to provide a comfortable temperature. If the user is concentrating, the system can also adjust the brightness of the lights to provide an environment suitable for work. Furthermore, if the user is tired, the system can switch the appliances to a mode that minimizes their operating noise. This allows the user to live a comfortable life by receiving suggestions for the optimal operation mode of the appliances according to their emotions.

[0107] The IoT home appliance operation system can infer a user's emotions and analyze the usage history of home appliances based on the inferred emotions. For example, if the user is feeling stressed, the system can analyze the usage history of home appliances that help relieve stress and suggest the optimal operation method. If the user is feeling relaxed, the system can analyze the usage history of home appliances that have a relaxing effect and suggest the optimal operation method. Furthermore, if the user is tired, the system can analyze the usage history of home appliances that help recover from fatigue and suggest the optimal operation method. In this way, users can receive suggestions for the optimal way to use home appliances based on their emotions, allowing them to live a comfortable life.

[0108] The IoT home appliance operation system can estimate a user's emotions and adjust the maintenance schedule of the appliance based on the estimated emotions. For example, if the user is feeling stressed, the system can refrain from sending a maintenance notification and reschedule it later. The system can also provide a detailed maintenance schedule if the user is relaxed. Furthermore, if the user is in a hurry, the system can prioritize notifications of simple maintenance procedures. This allows the user to receive an optimal maintenance schedule based on their emotions, allowing them to maintain their appliances while reducing stress.

[0109] The processing flow of the second embodiment will be briefly explained below.

[0110] Step 1: The acquisition unit acquires information about each IoT home appliance. For example, the acquisition unit can acquire information about refrigerators, air conditioners, lighting, etc. entered by the user. The acquisition unit can acquire information about each IoT home appliance by scanning a two-dimensional code or by using Bluetooth. For example, the acquisition unit can acquire information about the IoT home appliance by scanning a two-dimensional code with a camera. The acquisition unit can also acquire information by pairing with the IoT home appliance using Bluetooth. Step 2: The analysis unit analyzes the protocol of each IoT home appliance based on the information acquired by the acquisition unit. For example, the analysis unit analyzes the communication protocol of a refrigerator and identifies the temperature setting method. The analysis unit can also analyze the communication protocol of an air conditioner and identify the airflow adjustment method. Furthermore, the analysis unit can analyze the communication protocol of a lighting device and identify the brightness adjustment method. Step 3: The generation unit generates a unified operation interface based on the protocol analyzed by the analysis unit. For example, the generation unit enables operations such as setting the refrigerator temperature, adjusting the air conditioner's airflow, and adjusting the lighting brightness to be performed using a unified interface. The generation unit, for example, generates a GUI (graphical user interface) to enable users to operate intuitively. The generation unit can also generate an interface that is operated using voice commands. For example, the generation unit enables the user to set the refrigerator temperature by giving a voice command such as "Set the refrigerator temperature to 5 degrees."

[0111] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0112] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> Examples of the generative AI include a neural network (NN) and a neural network (NN). The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, image data, etc. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and may perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-mentioned parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. The processing performed by an AI including the generative AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI including the generative AI.

[0113] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0114] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0115] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0116] 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0117] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0118] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0119] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0120] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0121] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0122] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0123] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0124] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0125] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0126] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0127] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0128] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt including an instruction, as well as inference data such as audio data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as audio data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation models 58 include AIs other than the generation AI. Examples of AIs other than the generation AI include, but are not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes. These AIs can perform various types of processing, but are not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-described parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. Processing performed by an AI, including the generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI, including the generation AI.

[0129] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0130] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0131] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0132] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0133] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0134] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0135] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0136] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0137] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0138] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0139] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0140] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0141] In the headset type terminal 314, the identification process is performed by the processor 46. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0142] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0143] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0144] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt including an instruction, as well as inference data such as audio data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as audio data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation models 58 include AIs other than the generation AI. Examples of AIs other than the generation AI include, but are not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes. These AIs can perform various types of processing, but are not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-described parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. Processing performed by an AI, including the generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI, including the generation AI.

[0145] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0146] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0147] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0148] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0149] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0150] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0151] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0152] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0153] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0154] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0155] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0156] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0157] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0158] In the robot 414, the processor 46 performs the identification process. The storage 50 stores the identification program 60. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as the control unit 46A in accordance with the identification program 60 executed on the RAM 48. The robot 414 also has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform the same process as the identification processing unit 290 using these models.

[0159] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0160] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0161] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt including an instruction, as well as inference data such as audio data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as audio data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation models 58 include AIs other than the generation AI. Examples of AIs other than the generation AI include, but are not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes. These AIs can perform various types of processing, but are not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-described parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. Processing performed by an AI, including the generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI, including the generation AI.

[0162] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0163] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0164] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0165] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0166] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0167] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0168] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0169] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0170] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0171] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0172] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0173] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0174] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0175] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0176] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0177] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0178] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0179] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0180] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0181] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0182] [Explanation of symbols]

[0183] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. an acquisition unit that acquires information about each IoT home appliance; an analysis unit that analyzes a protocol of each IoT home appliance based on the information acquired by the acquisition unit; a generation unit that generates a unified operation interface based on the protocol analyzed by the analysis unit. A system characterized by:

2. Equipped with an encryption unit that encrypts communications with each IoT home appliance 2. The system of claim 1.

3. Equipped with an authentication unit that authenticates each IoT home appliance 2. The system of claim 1.

4. It has a registration section that supports the procedure when users register IoT home appliances.

2. The system of claim 1.

5. The acquisition unit Obtain information about each IoT appliance by scanning a 2D code or using Bluetooth 2. The system of claim 1.

6. The generation unit Abstract the functions of each IoT appliance and set common operation items 2. The system of claim 1.

7. The acquisition unit Estimates user emotions and adjusts the timing of information acquisition from IoT appliances based on the estimated user emotions.

2. The system of claim 1.

8. The acquisition unit Analyze the frequency of use of each IoT appliance and select the appropriate method of obtaining information 2. The system of claim 1.

Images