System

Abstract

To provide a system that properly informs users of solutions to malfunctions or problems with home appliances and achieves efficient operation through cooperation between home appliances.SOLUTION: A system according to an embodiment includes an analysis unit, a fault proposal unit, a voice unit, a detection unit, a power-off proposal unit, and a talk generation unit. The analysis unit analyzes an operation log. The fault proposal unit proposes a solution based on the information analyzed by the analysis unit. The voice unit conveys the solution proposed by the fault proposal unit to a user by voice. The detection unit detects that the user has gone out. The power-off proposal unit makes a proposal to turn off power of a home appliance when the detection unit detects that the user has gone out. The talk generation unit generates a talk for the user by having the home appliances communicate with each other.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional technology, it was difficult to properly communicate solutions to problems or malfunctions in home appliances to users, and there were also issues with efficient operation through coordination between home appliances.

[0005] The system according to the embodiment aims to properly inform users of solutions to malfunctions or problems with home appliances and to realize efficient operation through cooperation between home appliances. [Means for solving the problem]

[0006] The system according to the embodiment includes an analysis unit, a fault proposal unit, a voice unit, a detection unit, a power-off proposal unit, and a talk generation unit. The analysis unit analyzes the operation log. The fault proposal unit proposes a solution based on the information analyzed by the analysis unit. The voice unit conveys the solution proposed by the fault proposal unit to the user by voice. The detection unit detects that the user has gone out. The power-off proposal unit proposes to turn off the power of the home appliance when the detection unit detects that the user has gone out. The talk generation unit generates talk for the user by allowing the home appliances to communicate with each other. Effect of the Invention

[0007] The system according to the embodiment can properly inform users of solutions to malfunctions or problems with home appliances, and can achieve efficient operation through cooperation between home appliances. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The home appliance dialogue system according to the embodiment of the present invention is a system in which a home appliance judges its operating state based on its own operation log and communicates with a user using machine learning and voice conversion. In this home appliance dialogue system, the home appliance analyzes its own operation log, detects malfunctions and defects, proposes solutions based on the detected information, and communicates this to the user by voice. For example, if a refrigerator determines that there is a problem with the cooling function, it speaks to the user, saying, "There is a problem with the cooling function. Please clean the filter." In addition, by home appliance cooperation, other home appliances detect that the user has gone out and suggest that the user should turn off their own power. For example, an air conditioner detects that the user has gone out and suggests, "We have detected that you have gone out. Would you like to turn off the air conditioner?" In addition, home appliances talk to each other using a messenger app and generate a talk for the user. For example, a washing machine and a dryer send a message to the user saying, "The washing is done. Would you like to start drying?" This system allows home appliances to interact with the user as if they had a will, propose solutions to malfunctions and defects, and suggest turning off the power when going out. In addition, by home appliances talking to each other, information is smoothly provided to the user. As a result, the home appliance dialogue system can provide quick and appropriate information to the user.

[0029] The home appliance dialogue system according to the embodiment includes an analysis unit, a fault suggestion unit, a voice unit, a detection unit, a power off suggestion unit, and a talk generation unit. The analysis unit analyzes an operation log of the home appliance. The operation log includes, for example, time, events, error codes, etc., but is not limited to such examples. The analysis unit analyzes the operation log using, for example, data mining technology to determine the operation state of the home appliance. The analysis unit can also analyze the operation log using statistical analysis technology. For example, the analysis unit detects an abnormal pattern from the operation log and identifies a sign of a fault or malfunction. The fault suggestion unit proposes a solution based on the information analyzed by the analysis unit. The solution includes, for example, a repair procedure and a setting change, but is not limited to such examples. For example, the fault suggestion unit proposes an appropriate repair procedure for an abnormality detected by the analysis unit. The fault suggestion unit can also propose a method of solving the problem by changing the settings. For example, the fault suggestion unit suggests cleaning the filter when there is a problem with the cooling function. The voice unit conveys the solution proposed by the fault suggestion unit to the user by voice. The voice unit outputs the solution by voice using, for example, voice synthesis technology. The voice unit can also adjust the tone and speed of the voice to provide a voice that is easy for the user to hear. For example, the voice unit provides a voice that is easy to understand even when the user is stressed by speaking slowly in a calm tone. The detection unit detects the user's going out. The detection unit detects the user's going out using, for example, location information. The detection unit can also detect the user's going out using a motion sensor. For example, the detection unit acquires location information of the user's smartphone and detects the user's going out. The power off suggestion unit suggests turning off the power of the home appliance when the detection unit detects the user's going out. The power off suggestion unit suggests turning off the power at an appropriate timing for the user by adjusting, for example, the notification method and the timing of the suggestion. For example, the power off suggestion unit suggests turning off the air conditioner immediately after going out. The talk generation unit generates a talk for the user by communication between the home appliances. The talk generation unit generates a talk using, for example, natural language generation technology. The talk generation unit can also provide a talk that is easy for the user to understand by adjusting the content and format of the talk.For example, the conversation generating unit transmits a message to the user from the washing machine and the dryer saying, "Washing is done. Do you want to start drying?". This allows the home appliance dialogue system according to the embodiment to provide the user with prompt and appropriate information.

[0030] The analysis unit analyzes the operation log of the home appliance. The operation log includes, for example, time, events, error codes, etc., but is not limited to such examples. The analysis unit analyzes the operation log using, for example, a data mining technique to determine the operation state of the home appliance. Specifically, as the data mining technique, a method such as clustering, association rule mining, or decision tree is used. As a result, the analysis unit can extract patterns and trends from the operation log and distinguish between normal and abnormal operations. The analysis unit can also analyze the operation log using a statistical analysis technique. For example, the analysis unit can analyze the fluctuations of the operation log and detect abnormal patterns using time series analysis or regression analysis. Furthermore, the analysis unit can also build an anomaly detection model from the operation log using a machine learning algorithm. For example, the analysis unit can learn the feature amount of the operation log using a support vector machine or random forest, and identify signs of failure or malfunction with high accuracy. As a result, the analysis unit can monitor the operation state of the home appliance in real time and respond quickly when an abnormality occurs.

[0031] The fault suggestion unit proposes a solution based on the information analyzed by the analysis unit. The solutions include, for example, repair procedures and setting changes, but are not limited to such examples. For example, the fault suggestion unit proposes an appropriate repair procedure for the abnormality detected by the analysis unit. Specifically, the fault suggestion unit refers to the manual of the home appliance and past repair history, and provides the user with an optimal repair procedure. The fault suggestion unit can also propose a method of solving the problem by changing the settings. For example, the fault suggestion unit suggests cleaning the filter when there is a problem with the cooling function. Furthermore, the fault suggestion unit can adjust the difficulty and detail of the solution according to the user's skill level and situation. For example, a simple procedure is proposed for a novice user, and a detailed procedure is provided for an experienced user. In addition, the fault suggestion unit provides a list of necessary tools and parts when proposing a solution, and supports the user to perform repairs smoothly. As a result, the fault suggestion unit can provide the user with a quick and appropriate solution, and effectively resolve the breakdown or malfunction of the home appliance.

[0032] The voice unit communicates the solution proposed by the fault suggestion unit to the user by voice. The voice unit outputs the solution by voice, for example, using a voice synthesis technology. Specifically, as the voice synthesis technology, a text-to-speech (TTS) technology is used, and the text information of the solution is converted into a natural voice. The voice unit can also adjust the tone and speed of the voice to provide a voice that is easy for the user to hear. For example, the voice unit provides a voice that is easy to understand even when the user is stressed by speaking slowly in a calm tone. Furthermore, the voice unit can dynamically change the content and expression of the voice depending on the user's reaction and situation. For example, if the user does not understand the solution, the voice unit will explain it again or explain it in a different way. The voice unit also has a multilingual function and can provide solutions in multiple languages ​​depending on the user's language setting. As a result, the voice unit can provide solutions to the user in an easy-to-understand and familiar form, and support the resolution of breakdowns and malfunctions of home appliances.

[0033] The detection unit detects the user's going out. The detection unit detects the user's going out using, for example, location information. Specifically, the detection unit uses the GPS function of the user's smartphone to acquire the current location, and determines that the user has gone out when the user goes out of a certain range. The detection unit can also detect the user's going out using a motion sensor. For example, a motion sensor installed at the entrance of the house detects the user's movement and determines that the user has gone out. Furthermore, the detection unit can integrate multiple sensor information to perform more accurate detection of going out. For example, the detection unit combines the location information of the smartphone and the information of the motion sensor at the entrance to detect the user's going out with high accuracy. The detection unit can also learn the user's life pattern and predict the user's going out. For example, the detection unit grasps the tendency to go out at a specific time period or day of the week based on the past history of going out, and predicts the user's going out. As a result, the detection unit can accurately detect the user's going out and appropriately trigger a suggestion to turn off home appliances and other functions.

[0034] The power-off suggestion unit suggests turning off the power of the home appliance when the detection unit detects that the user has gone out. The power-off suggestion unit suggests turning off the power at a timing appropriate for the user by, for example, adjusting the notification method and the timing of the suggestion. Specifically, the power-off suggestion unit suggests turning off the power to the user by using the notification function of the smartphone. For example, it suggests turning off the air conditioner immediately after going out. In addition, the power-off suggestion unit can customize the suggestion content according to the user's schedule and the purpose of going out. For example, in the case of going out for a short time, it suggests changing the temperature setting of the air conditioner, and in the case of going out for a long time, it suggests turning off the power completely. Furthermore, the power-off suggestion unit can learn the user's past behavior history and calculate the optimal suggestion timing. For example, it makes a suggestion at a timing that is most acceptable to the user based on the history of power-off operations when going out in the past. In addition, the power-off suggestion unit can also suggest turning off the power of multiple home appliances at once. For example, it suggests turning off the power of the air conditioner, TV, lights, etc. at once when going out. In this way, the power-off suggestion unit can improve the user's convenience and also contribute to saving energy.

[0035] The talk generation unit generates talks for the user by communicating with the home appliances. The talk generation unit generates talks using, for example, natural language generation technology. Specifically, as natural language generation technology, template-based generation and machine learning-based generation are used. In template-based generation, talks are generated based on a template prepared in advance, and in machine learning-based generation, talks are generated using a neural network. The talk generation unit can also adjust the content and format of the talk to provide a talk that is easy for the user to understand. For example, the talk generation unit sends a message to the user from the washing machine and dryer saying, "Washing is done. Would you like to start drying?" Furthermore, the talk generation unit can learn the user's preferences and past response history and generate personalized talks. For example, if the user has responded "Yes" to the message "Do you want to start drying?" in the past, the unit will suggest that the dryer automatically start drying from the next time onwards. The talk generation unit can also generate talks in cooperation with multiple home appliances. For example, the refrigerator suggests, "There is little milk left. Would you like to order some?", and when the user responds "Yes", the smart speaker notifies the user that "The order has been completed." This enables the talk generation unit to strengthen the cooperation between home appliances and provide more convenient and efficient information to users.

[0036] The talk generation unit can generate talk for a user by communication between home appliances. For example, the talk generation unit generates talk for a user by communication between home appliances using a communication protocol such as Wi-Fi or Bluetooth (registered trademark). For example, the talk generation unit causes a refrigerator and an air conditioner to communicate with each other, and the refrigerator suggests to the user, "There is a problem with the cooling function. Would you like to turn off the air conditioner?". The talk generation unit can also generate talk using natural language generation technology. For example, the talk generation unit causes a washing machine and a dryer to send a message to the user, "Washing is done. Would you like to start drying?". As a result, home appliances communicate with each other to generate talk for a user, and information is smoothly provided to the user.

[0037] The analysis unit can analyze the operation log. For example, the analysis unit detects an abnormal pattern from the operation log and identifies signs of a failure or malfunction. The operation log includes, for example, time, events, error codes, etc., but is not limited to these examples. For example, the analysis unit analyzes the operation log using data mining technology to determine the operation state of the home appliance. The analysis unit can also analyze the operation log using statistical analysis technology. For example, the analysis unit detects an abnormal pattern from the operation log and identifies signs of a failure or malfunction. In this way, the operation state of the home appliance can be determined by analyzing the operation log.

[0038] The fault proposal unit can propose a solution based on the information analyzed by the analysis unit. The fault proposal unit, for example, proposes an appropriate repair procedure for the abnormality detected by the analysis unit. The solution includes, for example, a repair procedure and a setting change, but is not limited to such examples. The fault proposal unit, for example, proposes an appropriate repair procedure for the abnormality detected by the analysis unit. The fault proposal unit can also propose a method of solving the problem by changing the settings. For example, the fault proposal unit suggests cleaning the filter when there is a problem with the cooling function. In this way, a solution is proposed based on the analyzed information, so that the failure or malfunction can be quickly solved.

[0039] The voice unit can communicate the solution proposed by the fault suggestion unit to the user by voice. The voice unit outputs the solution by voice, for example, using voice synthesis technology. Specific methods and standards for communicating by voice include, for example, tone and speed of the voice, but are not limited to such examples. The voice unit can also provide a voice that is easy for the user to hear, for example, by adjusting the tone and speed of the voice. For example, the voice unit provides a voice that is easy for the user to understand by speaking slowly in a calm tone, even when the user is stressed. This allows the user to respond quickly by communicating the solution by voice.

[0040] The detection unit can detect the user going out. The detection unit detects the user going out by using, for example, location information. Specific methods and standards for detecting the user going out include, for example, location information, a motion sensor, and the like, but are not limited to such examples. The detection unit, for example, acquires location information of the user's smartphone and detects the user going out. The detection unit can also detect the user going out by using a motion sensor. For example, the detection unit analyzes the user's motion pattern and detects the user going out. In this way, by detecting the user going out, a suggestion to turn off the power can be made at an appropriate time.

[0041] The power-off suggestion unit can suggest turning off the power of the home appliance when the detection unit detects that the user has gone out. The power-off suggestion unit, for example, adjusts the notification method and the timing of the suggestion to suggest turning off the power at a timing appropriate for the user. Specific methods and standards for suggesting turning off the power include, for example, the notification method and the timing of the suggestion, but are not limited to such examples. The power-off suggestion unit, for example, suggests turning off the air conditioner immediately after going out. This makes it possible to save energy by suggesting turning off the power when going out.

[0042] The analysis unit can determine the priority of analysis based on the frequency of use of the home appliance when analyzing the operation log. For example, the analysis unit determines the priority of analysis based on the frequency of use of the home appliance when analyzing the operation log. Specific measurement methods and standards of the frequency of use include, for example, the number of times of use and the duration of use, but are not limited to such examples. For example, the analysis unit prioritizes the analysis of the operation log of a home appliance that is used frequently, and provides important information quickly. In addition, the analysis of the operation log of a home appliance that is used less frequently can be postponed. Furthermore, the level of detail of the analysis can be adjusted according to the frequency of use, and an efficient analysis can be performed. In this way, important information can be provided quickly by determining the priority of analysis based on the frequency of use.

[0043] The analysis unit can apply different analysis algorithms depending on the type of home appliance when analyzing the operation log. For example, the analysis unit applies different analysis algorithms depending on the type of home appliance when analyzing the operation log. Specific classification methods and standards for the types of home appliances include, for example, refrigerators, washing machines, and air conditioners, but are not limited to such examples. Specific types and implementation methods of the analysis algorithms include, for example, machine learning algorithms and statistical analysis algorithms, but are not limited to such examples. For example, the analysis unit applies an analysis algorithm specialized in cooling performance to the operation log of a refrigerator. In addition, an analysis algorithm specialized in cleaning performance can also be applied to the operation log of a washing machine. Furthermore, an analysis algorithm specialized in heating and cooling performance can also be applied to the operation log of an air conditioner. Thereby, by applying an analysis algorithm according to the type of home appliance, the accuracy of the analysis is improved.

[0044] The analysis unit can perform the analysis while considering the installation location information of the home appliance when analyzing the operation log. For example, the analysis unit performs the analysis while considering the installation location information of the home appliance when analyzing the operation log. Specific types and acquisition methods of the installation location information include, for example, GPS information, room layout information, and the like, but are not limited to such examples. For example, the analysis unit can consider the usage conditions specific to the kitchen when analyzing the operation log of a home appliance installed in the kitchen. In addition, the analysis unit can also consider the usage conditions specific to the living room when analyzing the operation log of a home appliance installed in the living room. Furthermore, the analysis unit can also consider the usage conditions specific to the bathroom when analyzing the operation log of a home appliance installed in the bathroom. As a result, by considering the installation location information, more accurate analysis is possible.

[0045] The analysis unit can improve the accuracy of the analysis when analyzing the operation log by referring to the past maintenance history of the home appliance. For example, the analysis unit improves the accuracy of the analysis when analyzing the operation log by referring to the past maintenance history of the home appliance. Specific contents and acquisition methods of the maintenance history include, for example, repair history and inspection history, but are not limited to such examples. For example, the analysis unit refers to the past maintenance history to detect early signs of failure. In addition, it is also possible to predict deterioration of a specific part based on the maintenance history. Furthermore, it is also possible to improve the reliability of the analysis result by taking the maintenance history into consideration. As a result, the accuracy of the analysis is improved by referring to the past maintenance history.

[0046] The fault suggestion unit can adjust the level of detail of the proposal based on the importance of the home appliance when proposing a fault. For example, the fault suggestion unit adjusts the level of detail of the proposal based on the importance of the home appliance when proposing a fault. Specific criteria and methods for determining the importance of the home appliance include, for example, the frequency of use and the impact of the fault, but are not limited to such examples. For example, the fault suggestion unit provides detailed information for a fault proposal of a home appliance with high importance. Also, the fault suggestion unit can provide concise information for a fault proposal of a home appliance with low importance. Furthermore, the priority of the proposal can be adjusted according to the importance. As a result, important information can be provided preferentially by adjusting the level of detail of the proposal based on the importance of the home appliance.

[0047] The fault suggestion unit can apply different proposal algorithms depending on the category of the home appliance when proposing a fault. For example, the fault suggestion unit applies different proposal algorithms depending on the category of the home appliance when proposing a fault. Specific classification methods and standards for the categories of home appliances include, for example, kitchen appliances, living room appliances, etc., but are not limited to such examples. Specific types and implementation methods of the proposal algorithm include, for example, rule-based, machine learning-based, etc., but are not limited to such examples. For example, the fault suggestion unit applies an algorithm specialized in cooling performance to a fault suggestion for a refrigerator. Also, an algorithm specialized in cleaning performance can be applied to a fault suggestion for a washing machine. Furthermore, an algorithm specialized in heating and cooling performance can be applied to a fault suggestion for an air conditioner. As a result, the accuracy of the proposal is improved by applying a proposal algorithm according to the category of the home appliance.

[0048] The fault suggestion unit can make a proposal taking into consideration installation location information of the home appliance when making a fault suggestion. For example, the fault suggestion unit makes a proposal taking into consideration installation location information of the home appliance when making a fault suggestion. Specific types and acquisition methods of the installation location information include, for example, GPS information, room layout information, and the like, but are not limited to such examples. For example, the fault suggestion unit may make a fault suggestion for a home appliance installed in a kitchen by taking into consideration a usage situation specific to the kitchen. Furthermore, a fault suggestion for a home appliance installed in a living room may also take into consideration a usage situation specific to the living room. Furthermore, a fault suggestion for a home appliance installed in a bathroom may also take into consideration a usage situation specific to the bathroom. As a result, more accurate suggestions can be made by taking into consideration the installation location information.

[0049] The fault suggestion unit can improve the accuracy of the proposal by referring to the past fault history of the home appliance when proposing a fault. The fault suggestion unit, for example, improves the accuracy of the proposal by referring to the past fault history of the home appliance when proposing a fault. Specific contents and acquisition methods of the fault history include, for example, repair history, error code, etc., but are not limited to such examples. The fault suggestion unit, for example, refers to the past fault history to detect early signs of a fault. In addition, it is also possible to predict deterioration of a specific part based on the fault history. Furthermore, it is also possible to improve the reliability of the proposal by taking the fault history into consideration. As a result, the accuracy of the proposal is improved by referring to the past fault history.

[0050] The audio unit can customize the content of the audio based on the usage status of the home appliance when outputting the audio. For example, the audio unit customizes the content of the audio based on the usage status of the home appliance when outputting the audio. Specific content and acquisition method of the usage status include, for example, the number of times of use and the usage time, but are not limited to such examples. For example, the audio unit provides information on the cooling function based on the usage status of the refrigerator. Also, it can provide information on the cleaning function based on the usage status of the washing machine. Furthermore, it can provide information on the heating and cooling function based on the usage status of the air conditioner. In this way, by customizing the content of the audio based on the usage status of the home appliance, it is possible to provide information that is useful to the user.

[0051] The voice unit can apply different voice algorithms depending on the type of home appliance when outputting voice. For example, the voice unit applies different voice algorithms depending on the type of home appliance when outputting voice. Specific classification methods and standards for the types of home appliances include, for example, refrigerators, washing machines, and air conditioners, but are not limited to such examples. Specific types and implementation methods of voice algorithms include, for example, voice synthesis algorithms and voice recognition algorithms, but are not limited to such examples. For example, the voice unit applies an algorithm specialized in cooling performance to the voice output of a refrigerator. Also, an algorithm specialized in cleaning performance can be applied to the voice output of a washing machine. Furthermore, an algorithm specialized in heating and cooling performance can be applied to the voice output of an air conditioner. As a result, the accuracy of the voice output is improved by applying a voice algorithm according to the type of home appliance.

[0052] The audio unit can adjust the content of the audio in consideration of the installation location information of the home appliance when outputting the audio. For example, the audio unit adjusts the content of the audio in consideration of the installation location information of the home appliance when outputting the audio. Specific types and acquisition methods of the installation location information include, for example, GPS information, room layout information, and the like, but are not limited to such examples. For example, the audio unit can consider the usage situation specific to the kitchen when outputting the audio of an appliance installed in the kitchen. Also, the audio output of an appliance installed in the living room can also consider the usage situation specific to the living room. Furthermore, the audio output of an appliance installed in the bathroom can also consider the usage situation specific to the bathroom. As a result, more appropriate audio output is possible by considering the installation location information.

[0053] The audio unit can improve the accuracy of the audio by referring to the past audio history of the home appliance when outputting the audio. The audio unit can improve the accuracy of the audio by referring to the past audio history of the home appliance when outputting the audio. Specific contents and acquisition methods of the audio history include, for example, past audio output contents, audio recognition results, etc., but are not limited to such examples. The audio unit can provide audio that is easy for the user to understand by referring to the past audio history, for example. In addition, specific information can be emphasized based on the audio history. Furthermore, the reliability of the audio output can be improved by taking the audio history into consideration. As a result, the accuracy of the audio output is improved by referring to the past audio history.

[0054] The detection unit can improve the accuracy of detection when detecting an outing by referring to the user's past outing history. The detection unit, for example, improves the accuracy of detection when detecting an outing by referring to the user's past outing history. Specific contents and acquisition methods of the outing history include, for example, past outing times and outing destinations, but are not limited to such examples. The detection unit, for example, improves accuracy by referring to the past outing history and learning outing patterns. In addition, it is also possible to predict outings during specific time periods based on the outing history. Furthermore, it is also possible to improve the reliability of outing detection by taking the outing history into consideration. As a result, the accuracy of outing detection is improved by referring to the past outing history.

[0055] The detection unit can perform detection while taking into consideration the schedule information of the user when detecting an outing. For example, the detection unit performs detection while taking into consideration the schedule information of the user when detecting an outing. Specific contents and acquisition methods of the schedule information include, for example, calendar information, a timetable, and the like, but are not limited to such examples. For example, the detection unit refers to the calendar information of the user and detects an outing based on a plan. In addition, it is also possible to predict an outing related to a specific event from the schedule information. Furthermore, it is also possible to improve the accuracy of outing detection based on the schedule information. As a result, more accurate outing detection is possible by taking into consideration the schedule information.

[0056] The detection unit can perform detection while taking into consideration the geographical location information of the user when detecting going out. For example, the detection unit performs detection while taking into consideration the geographical location information of the user when detecting going out. Specific contents and acquisition methods of the geographical location information include, for example, GPS information, location information services, etc., but are not limited to such examples. For example, the detection unit acquires the user's current location in real time and detects going out. In addition, it is also possible to predict going out at a specific location based on the geographical location information. Furthermore, it is also possible to improve the accuracy of going out detection by taking into consideration the geographical location information. As a result, more accurate detection of going out is possible by taking into consideration the geographical location information.

[0057] The detection unit can improve the accuracy of detection when detecting going out by analyzing the social media activity of the user. For example, the detection unit improves the accuracy of detection when detecting going out by analyzing the social media activity of the user. Specific contents and acquisition methods of the social media activity include, for example, posting contents, activity frequency, etc., but are not limited to such examples. The detection unit, for example, analyzes the social media posts of the user and learns the pattern of going out to improve the accuracy. In addition, it is also possible to predict going out related to a specific event based on the social media activity. Furthermore, it is also possible to improve the reliability of the outing detection by taking into account the social media activity. Thereby, the accuracy of the outing detection is improved by analyzing the social media activity.

[0058] The power-off suggestion unit can adjust the level of detail of the proposal based on the usage status of the home appliance when proposing to turn off the power. For example, the power-off suggestion unit adjusts the level of detail of the proposal based on the usage status of the home appliance when proposing to turn off the power. Specific contents and acquisition methods of the usage status include, for example, the number of times of use, the usage time, etc., but are not limited to such examples. For example, the power-off suggestion unit provides detailed information for a power-off suggestion for a home appliance that is used frequently. Also, the power-off suggestion unit can provide concise information for a power-off suggestion for a home appliance that is used infrequently. Furthermore, the priority of the proposal can be adjusted according to the usage status. As a result, important information can be provided preferentially by adjusting the level of detail of the proposal based on the usage status of the home appliance.

[0059] The power-off suggestion unit can apply different proposal algorithms depending on the type of home appliance when proposing to turn off the power. For example, the power-off suggestion unit applies different proposal algorithms depending on the type of home appliance when proposing to turn off the power. Specific classification methods and standards for the types of home appliances include, for example, refrigerators, washing machines, and air conditioners, but are not limited to such examples. Specific types and implementation methods of the proposal algorithm include, for example, rule-based, machine learning-based, and the like, but are not limited to such examples. For example, the power-off suggestion unit applies an algorithm specialized in cooling performance to the proposal to turn off the refrigerator. Also, an algorithm specialized in cleaning performance can be applied to the proposal to turn off the washing machine. Furthermore, an algorithm specialized in heating and cooling performance can be applied to the proposal to turn off the air conditioner. Thereby, by applying a proposal algorithm according to the type of home appliance, the accuracy of the proposal is improved.

[0060] The power-off suggestion unit can make a proposal taking into consideration installation location information of the home appliance when proposing to turn off the power. For example, the power-off suggestion unit makes a proposal taking into consideration installation location information of the home appliance when proposing to turn off the power. Specific types and acquisition methods of the installation location information include, for example, GPS information, room layout information, and the like, but are not limited to such examples. For example, the power-off suggestion unit may take into consideration a usage situation specific to the kitchen when proposing to turn off a home appliance installed in a kitchen. Also, the power-off suggestion unit may take into consideration a usage situation specific to the living room when proposing to turn off a home appliance installed in a bathroom. In this way, more accurate suggestions can be made by taking into consideration the installation location information.

[0061] The power-off suggestion unit can improve the accuracy of the suggestion by referring to the past power-off history of the home appliance when proposing to turn off the power. For example, the power-off suggestion unit improves the accuracy of the suggestion by referring to the past power-off history of the home appliance when proposing to turn off the power. Specific contents and acquisition methods of the power-off history include, for example, past power-off times and power-off frequencies, but are not limited to such examples. The power-off suggestion unit improves accuracy by, for example, referring to the past power-off history and learning power-off patterns. In addition, it is also possible to predict power-off in a specific time period based on the power-off history. Furthermore, it is also possible to improve the reliability of the suggestion by taking the power-off history into consideration. As a result, the accuracy of the suggestion is improved by referring to the past power-off history.

[0062] The talk generation unit can customize the content of the talk based on the usage status of the home appliance when generating the talk. For example, the talk generation unit customizes the content of the talk based on the usage status of the home appliance when generating the talk. Specific content and acquisition method of the usage status include, for example, the number of times of use and the usage time, but are not limited to such examples. For example, the talk generation unit generates talk regarding a cooling function based on the usage status of a refrigerator. Also, it is possible to generate talk regarding a cleaning function based on the usage status of a washing machine. Furthermore, it is possible to generate talk regarding a heating and cooling function based on the usage status of an air conditioner. In this way, by customizing the content of the talk based on the usage status of the home appliance, it is possible to provide useful information to the user.

[0063] The talk generation unit can apply different talk algorithms depending on the type of home appliance when generating talk. For example, the talk generation unit applies different talk algorithms depending on the type of home appliance when generating talk. Specific classification methods and standards for the types of home appliances include, for example, refrigerators, washing machines, and air conditioners, but are not limited to such examples. Specific types and implementation methods of talk algorithms include, for example, natural language generation algorithms and machine learning algorithms, but are not limited to such examples. For example, the talk generation unit applies an algorithm specialized in cooling performance to talk generation for a refrigerator. Also, an algorithm specialized in cleaning performance can be applied to talk generation for a washing machine. Furthermore, an algorithm specialized in heating and cooling performance can be applied to talk generation for an air conditioner. As a result, the accuracy of the talk is improved by applying a talk algorithm according to the type of home appliance.

[0064] The talk generation unit can adjust the content of the talk in consideration of the installation location information of the home appliance when generating the talk. For example, the talk generation unit adjusts the content of the talk in consideration of the installation location information of the home appliance when generating the talk. Specific types and acquisition methods of the installation location information include, for example, GPS information, room layout information, and the like, but are not limited to such examples. For example, the talk generation unit generates talk for a home appliance installed in a kitchen by considering a usage situation specific to the kitchen. Also, the talk generation for a home appliance installed in a living room can also consider a usage situation specific to the living room. Furthermore, the talk generation for a home appliance installed in a bathroom can also consider a usage situation specific to the bathroom. In this way, more appropriate talk is generated by considering the installation location information.

[0065] The talk generation unit can improve the accuracy of the talk by referring to the past talk history of the home appliance when generating the talk. The talk generation unit, for example, improves the accuracy of the talk by referring to the past talk history of the home appliance when generating the talk. Specific contents and acquisition methods of the talk history include, for example, past talk contents, talk frequency, etc., but are not limited to such examples. The talk generation unit, for example, refers to the past talk history to generate a talk that is easy for the user to understand. In addition, specific information can be emphasized based on the talk history. Furthermore, the reliability of the talk generation can be improved by taking the talk history into consideration. As a result, the accuracy of the talk is improved by referring to the past talk history.

[0066] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0067] The home appliance dialogue system may further include a health monitoring unit that monitors the user's health condition. The health monitoring unit, for example, measures the user's heart rate and body temperature with a sensor, and notifies the user if an abnormality is detected. For example, if the heart rate is abnormally high, the unit notifies the user, "Your heart rate is high. Please take a break." Also, if the body temperature is high, the unit may suggest, "Your body temperature is high. Please drink fluids." Furthermore, the health monitoring unit may accumulate the user's health data and support long-term health management. This makes it possible to monitor the user's health condition in real time and provide appropriate advice.

[0068] The home appliance dialogue system can further include an energy management unit that optimizes energy consumption. The energy management unit, for example, monitors the energy consumption of each home appliance in real time and suggests energy-efficient usage methods. For example, it suggests reducing energy consumption by optimizing the temperature setting of an air conditioner. The energy management unit can also adjust the usage schedule of the home appliances to reduce energy consumption during peak hours. Furthermore, the energy management unit can analyze past energy consumption data and support long-term energy efficiency improvements. This optimizes energy consumption, contributing to cost reduction and environmental protection.

[0069] The home appliance dialogue system may further include a customization unit that customizes the operation of the home appliance based on the user's preferences. The customization unit, for example, learns the user's past usage history and settings and proposes an operation that matches the user's preferences. For example, the temperature setting of an air conditioner may be automatically adjusted to match the user's preferences. The customization unit may also collect user feedback and improve the accuracy of the operation. Furthermore, the customization unit may also propose new functions and services based on the user's preferences. This enables customization to match the user's preferences, providing a more comfortable usage experience.

[0070] The home appliance dialogue system can further include a maintenance unit that supports the maintenance of home appliances. The maintenance unit, for example, analyzes the usage status and operation logs of the home appliance and notifies the user of the need for regular maintenance. For example, it notifies the user when it is time to clean the filter or replace parts. The maintenance unit can also provide maintenance procedures by audio or video to support the user in easily performing maintenance. Furthermore, the maintenance unit can manage past maintenance history and create future maintenance plans. This can extend the lifespan of the home appliance and prevent breakdowns.

[0071] The process flow of the first embodiment will be briefly described below.

[0072] Step 1: The analysis unit analyzes the operation log of the home appliance. The operation log includes, for example, time, events, error codes, etc., but is not limited to these examples. The analysis unit analyzes the operation log using, for example, data mining technology or statistical analysis technology, and determines the operation status of the home appliance. For example, the analysis unit detects abnormal patterns in the operation log and identifies signs of failure or malfunction. Step 2: The fault suggestion unit proposes solutions based on the information analyzed by the analysis unit. Solutions include, for example, repair procedures and configuration changes. For example, the analysis unit proposes a method of solving the problem by appropriate repair procedures and configuration changes for the anomaly detected by the analysis unit. Step 3: The voice unit communicates the solution proposed by the fault suggestion unit to the user by voice. The voice unit outputs the solution by voice using, for example, voice synthesis technology, and adjusts the tone and speed of the voice to provide a voice that is easy for the user to hear. Step 4: The detection unit detects that the user has gone out. The detection unit detects that the user has gone out, for example, by using location information or a motion sensor. For example, the detection unit obtains location information of the user's smartphone and detects that the user has gone out. Step 5: The power-off suggestion unit suggests turning off the home appliance when the detection unit detects that the user has left the house. The power-off suggestion unit suggests turning off the home appliance at a timing appropriate for the user, for example, by adjusting the notification method or the timing of the suggestion. For example, the power-off suggestion unit suggests turning off the air conditioner immediately after leaving the house. Step 6: The conversation generation unit generates conversations for the user through communication between home appliances. The conversation generation unit generates conversations using, for example, natural language generation technology, and adjusts the content and format of the conversation to provide conversations that are easy for the user to understand. For example, the washing machine and dryer send a message to the user saying, "Washing is done. Do you want to start drying?"

[0073] (Example 2) The home appliance dialogue system according to the embodiment of the present invention is a system in which a home appliance judges its operating state based on its own operation log and communicates with a user using machine learning and voice conversion. In this home appliance dialogue system, the home appliance analyzes its own operation log, detects malfunctions and defects, proposes solutions based on the detected information, and communicates this to the user by voice. For example, if a refrigerator determines that there is a problem with the cooling function, it speaks to the user, saying, "There is a problem with the cooling function. Please clean the filter." In addition, by home appliance cooperation, other home appliances detect that the user has gone out and suggest that the user should turn off their own power. For example, an air conditioner detects that the user has gone out and suggests, "We have detected that you have gone out. Would you like to turn off the air conditioner?" In addition, home appliances talk to each other using a messenger app and generate a talk for the user. For example, a washing machine and a dryer send a message to the user saying, "The washing is done. Would you like to start drying?" This system allows home appliances to interact with the user as if they had a will, propose solutions to malfunctions and defects, and suggest turning off the power when going out. In addition, by home appliances talking to each other, information is smoothly provided to the user. As a result, the home appliance dialogue system can provide quick and appropriate information to the user.

[0074] The home appliance dialogue system according to the embodiment includes an analysis unit, a fault suggestion unit, a voice unit, a detection unit, a power off suggestion unit, and a talk generation unit. The analysis unit analyzes an operation log of the home appliance. The operation log includes, for example, time, events, error codes, etc., but is not limited to such examples. The analysis unit analyzes the operation log using, for example, data mining technology to determine the operation state of the home appliance. The analysis unit can also analyze the operation log using statistical analysis technology. For example, the analysis unit detects an abnormal pattern from the operation log and identifies a sign of a fault or malfunction. The fault suggestion unit proposes a solution based on the information analyzed by the analysis unit. The solution includes, for example, a repair procedure and a setting change, but is not limited to such examples. For example, the fault suggestion unit proposes an appropriate repair procedure for an abnormality detected by the analysis unit. The fault suggestion unit can also propose a method of solving the problem by changing the settings. For example, the fault suggestion unit suggests cleaning the filter when there is a problem with the cooling function. The voice unit conveys the solution proposed by the fault suggestion unit to the user by voice. The voice unit outputs the solution by voice using, for example, voice synthesis technology. The voice unit can also adjust the tone and speed of the voice to provide a voice that is easy for the user to hear. For example, the voice unit provides a voice that is easy to understand even when the user is stressed by speaking slowly in a calm tone. The detection unit detects the user's going out. The detection unit detects the user's going out using, for example, location information. The detection unit can also detect the user's going out using a motion sensor. For example, the detection unit acquires location information of the user's smartphone and detects the user's going out. The power off suggestion unit suggests turning off the power of the home appliance when the detection unit detects the user's going out. The power off suggestion unit suggests turning off the power at an appropriate timing for the user by adjusting, for example, the notification method and the timing of the suggestion. For example, the power off suggestion unit suggests turning off the air conditioner immediately after going out. The talk generation unit generates a talk for the user by communication between the home appliances. The talk generation unit generates a talk using, for example, natural language generation technology. The talk generation unit can also provide a talk that is easy for the user to understand by adjusting the content and format of the talk.For example, the conversation generating unit transmits a message to the user from the washing machine and the dryer saying, "Washing is done. Do you want to start drying?". This allows the home appliance dialogue system according to the embodiment to provide the user with prompt and appropriate information.

[0075] The analysis unit analyzes the operation log of the home appliance. The operation log includes, for example, time, events, error codes, etc., but is not limited to such examples. The analysis unit analyzes the operation log using, for example, a data mining technique to determine the operation state of the home appliance. Specifically, as the data mining technique, a method such as clustering, association rule mining, or decision tree is used. As a result, the analysis unit can extract patterns and trends from the operation log and distinguish between normal and abnormal operations. The analysis unit can also analyze the operation log using a statistical analysis technique. For example, the analysis unit can analyze the fluctuations of the operation log and detect abnormal patterns using time series analysis or regression analysis. Furthermore, the analysis unit can also build an anomaly detection model from the operation log using a machine learning algorithm. For example, the analysis unit can learn the feature amount of the operation log using a support vector machine or random forest, and identify signs of failure or malfunction with high accuracy. As a result, the analysis unit can monitor the operation state of the home appliance in real time and respond quickly when an abnormality occurs.

[0076] The fault suggestion unit proposes a solution based on the information analyzed by the analysis unit. The solutions include, for example, repair procedures and setting changes, but are not limited to such examples. For example, the fault suggestion unit proposes an appropriate repair procedure for the abnormality detected by the analysis unit. Specifically, the fault suggestion unit refers to the manual of the home appliance and past repair history, and provides the user with an optimal repair procedure. The fault suggestion unit can also propose a method of solving the problem by changing the settings. For example, the fault suggestion unit suggests cleaning the filter when there is a problem with the cooling function. Furthermore, the fault suggestion unit can adjust the difficulty and detail of the solution according to the user's skill level and situation. For example, a simple procedure is proposed for a novice user, and a detailed procedure is provided for an experienced user. In addition, the fault suggestion unit provides a list of necessary tools and parts when proposing a solution, and supports the user to perform repairs smoothly. As a result, the fault suggestion unit can provide the user with a quick and appropriate solution, and effectively resolve the breakdown or malfunction of the home appliance.

[0077] The voice unit communicates the solution proposed by the fault suggestion unit to the user by voice. The voice unit outputs the solution by voice, for example, using a voice synthesis technology. Specifically, as the voice synthesis technology, a text-to-speech (TTS) technology is used, and the text information of the solution is converted into a natural voice. The voice unit can also adjust the tone and speed of the voice to provide a voice that is easy for the user to hear. For example, the voice unit provides a voice that is easy to understand even when the user is stressed by speaking slowly in a calm tone. Furthermore, the voice unit can dynamically change the content and expression of the voice depending on the user's reaction and situation. For example, if the user does not understand the solution, the voice unit will explain it again or explain it in a different way. The voice unit also has a multilingual function and can provide solutions in multiple languages ​​depending on the user's language setting. As a result, the voice unit can provide solutions to the user in an easy-to-understand and familiar form, and support the resolution of breakdowns and malfunctions of home appliances.

[0078] The detection unit detects the user's going out. The detection unit detects the user's going out using, for example, location information. Specifically, the detection unit uses the GPS function of the user's smartphone to acquire the current location, and determines that the user has gone out when the user goes out of a certain range. The detection unit can also detect the user's going out using a motion sensor. For example, a motion sensor installed at the entrance of the house detects the user's movement and determines that the user has gone out. Furthermore, the detection unit can integrate multiple sensor information to perform more accurate detection of going out. For example, the detection unit combines the location information of the smartphone and the information of the motion sensor at the entrance to detect the user's going out with high accuracy. The detection unit can also learn the user's life pattern and predict the user's going out. For example, the detection unit grasps the tendency to go out at a specific time period or day of the week based on the past history of going out, and predicts the user's going out. As a result, the detection unit can accurately detect the user's going out and appropriately trigger a suggestion to turn off home appliances and other functions.

[0079] The power-off suggestion unit suggests turning off the power of the home appliance when the detection unit detects that the user has gone out. The power-off suggestion unit suggests turning off the power at a timing appropriate for the user by, for example, adjusting the notification method and the timing of the suggestion. Specifically, the power-off suggestion unit suggests turning off the power to the user by using the notification function of the smartphone. For example, it suggests turning off the air conditioner immediately after going out. In addition, the power-off suggestion unit can customize the suggestion content according to the user's schedule and the purpose of going out. For example, in the case of going out for a short time, it suggests changing the temperature setting of the air conditioner, and in the case of going out for a long time, it suggests turning off the power completely. Furthermore, the power-off suggestion unit can learn the user's past behavior history and calculate the optimal suggestion timing. For example, it makes a suggestion at a timing that is most acceptable to the user based on the history of power-off operations when going out in the past. In addition, the power-off suggestion unit can also suggest turning off the power of multiple home appliances at once. For example, it suggests turning off the power of the air conditioner, TV, lights, etc. at once when going out. In this way, the power-off suggestion unit can improve the user's convenience and also contribute to saving energy.

[0080] The talk generation unit generates talks for the user by communicating with the home appliances. The talk generation unit generates talks using, for example, natural language generation technology. Specifically, as natural language generation technology, template-based generation and machine learning-based generation are used. In template-based generation, talks are generated based on a template prepared in advance, and in machine learning-based generation, talks are generated using a neural network. The talk generation unit can also adjust the content and format of the talk to provide a talk that is easy for the user to understand. For example, the talk generation unit sends a message to the user from the washing machine and dryer saying, "Washing is done. Would you like to start drying?" Furthermore, the talk generation unit can learn the user's preferences and past response history and generate personalized talks. For example, if the user has responded "Yes" to the message "Do you want to start drying?" in the past, the unit will suggest that the dryer automatically start drying from the next time onwards. The talk generation unit can also generate talks in cooperation with multiple home appliances. For example, the refrigerator suggests, "There is little milk left. Would you like to order some?", and when the user responds "Yes", the smart speaker notifies the user that "The order has been completed." This enables the talk generation unit to strengthen the cooperation between home appliances and provide more convenient and efficient information to users.

[0081] The talk generation unit can generate talk for a user by having home appliances communicate with each other. For example, the talk generation unit allows home appliances to communicate with each other using a communication protocol such as Wi-Fi or Bluetooth, and generates talk for a user. For example, the talk generation unit allows a refrigerator and an air conditioner to communicate, and the refrigerator suggests to the user, "There is a problem with the cooling function. Would you like to turn off the air conditioner?". The talk generation unit can also generate talk using natural language generation technology. For example, the talk generation unit allows a washing machine and a dryer to send a message to the user, "Washing is done. Would you like to start drying?". In this way, home appliances communicate with each other to generate talk for a user, and information can be smoothly provided to the user.

[0082] The analysis unit can analyze the operation log. For example, the analysis unit detects an abnormal pattern from the operation log and identifies signs of a failure or malfunction. The operation log includes, for example, time, events, error codes, etc., but is not limited to these examples. For example, the analysis unit analyzes the operation log using data mining technology to determine the operation state of the home appliance. The analysis unit can also analyze the operation log using statistical analysis technology. For example, the analysis unit detects an abnormal pattern from the operation log and identifies signs of a failure or malfunction. In this way, the operation state of the home appliance can be determined by analyzing the operation log.

[0083] The fault proposal unit can propose a solution based on the information analyzed by the analysis unit. The fault proposal unit, for example, proposes an appropriate repair procedure for the abnormality detected by the analysis unit. The solution includes, for example, a repair procedure and a setting change, but is not limited to such examples. The fault proposal unit, for example, proposes an appropriate repair procedure for the abnormality detected by the analysis unit. The fault proposal unit can also propose a method of solving the problem by changing the settings. For example, the fault proposal unit suggests cleaning the filter when there is a problem with the cooling function. In this way, a solution is proposed based on the analyzed information, so that the failure or malfunction can be quickly solved.

[0084] The voice unit can communicate the solution proposed by the fault suggestion unit to the user by voice. The voice unit outputs the solution by voice, for example, using voice synthesis technology. Specific methods and standards for communicating by voice include, for example, tone and speed of the voice, but are not limited to such examples. The voice unit can also provide a voice that is easy for the user to hear, for example, by adjusting the tone and speed of the voice. For example, the voice unit provides a voice that is easy for the user to understand by speaking slowly in a calm tone, even when the user is stressed. This allows the user to respond quickly by communicating the solution by voice.

[0085] The detection unit can detect the user going out. The detection unit detects the user going out by using, for example, location information. Specific methods and standards for detecting the user going out include, for example, location information, a motion sensor, and the like, but are not limited to such examples. The detection unit, for example, acquires location information of the user's smartphone and detects the user going out. The detection unit can also detect the user going out by using a motion sensor. For example, the detection unit analyzes the user's motion pattern and detects the user going out. In this way, by detecting the user going out, a suggestion to turn off the power can be made at an appropriate time.

[0086] The power-off suggestion unit can suggest turning off the power of the home appliance when the detection unit detects that the user has gone out. The power-off suggestion unit, for example, adjusts the notification method and the timing of the suggestion to suggest turning off the power at a timing appropriate for the user. Specific methods and standards for suggesting turning off the power include, for example, the notification method and the timing of the suggestion, but are not limited to such examples. The power-off suggestion unit, for example, suggests turning off the air conditioner immediately after going out. This makes it possible to save energy by suggesting turning off the power when going out.

[0087] The analysis unit can estimate the user's emotion and adjust the analysis method of the operation log based on the estimated user's emotion. The analysis unit, for example, estimates the user's emotion and adjusts the analysis method of the operation log based on the estimated user's emotion. Specific methods and standards for estimating emotion include, for example, facial expression recognition, voice analysis, and the like, but are not limited to such examples. For example, when the user is stressed, the analysis unit performs a quick and concise analysis and provides the result immediately. Also, when the user is relaxed, the analysis unit can perform a detailed analysis and provide a comprehensive result. Furthermore, when the user is excited, the analysis result can be provided in a visually attractive format. In this way, by adjusting the analysis method according to the user's emotion, more appropriate analysis results can be provided. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0088] The analysis unit can determine the priority of analysis based on the frequency of use of the home appliance when analyzing the operation log. For example, the analysis unit determines the priority of analysis based on the frequency of use of the home appliance when analyzing the operation log. Specific measurement methods and standards of the frequency of use include, for example, the number of times of use and the duration of use, but are not limited to such examples. For example, the analysis unit prioritizes the analysis of the operation log of a home appliance that is used frequently, and provides important information quickly. In addition, the analysis of the operation log of a home appliance that is used less frequently can be postponed. Furthermore, the level of detail of the analysis can be adjusted according to the frequency of use, and an efficient analysis can be performed. In this way, important information can be provided quickly by determining the priority of analysis based on the frequency of use.

[0089] The analysis unit can apply different analysis algorithms depending on the type of home appliance when analyzing the operation log. For example, the analysis unit applies different analysis algorithms depending on the type of home appliance when analyzing the operation log. Specific classification methods and standards for the types of home appliances include, for example, refrigerators, washing machines, and air conditioners, but are not limited to such examples. Specific types and implementation methods of the analysis algorithms include, for example, machine learning algorithms and statistical analysis algorithms, but are not limited to such examples. For example, the analysis unit applies an analysis algorithm specialized in cooling performance to the operation log of a refrigerator. In addition, an analysis algorithm specialized in cleaning performance can also be applied to the operation log of a washing machine. Furthermore, an analysis algorithm specialized in heating and cooling performance can also be applied to the operation log of an air conditioner. Thereby, by applying an analysis algorithm according to the type of home appliance, the accuracy of the analysis is improved.

[0090] The analysis unit can estimate the user's emotion and adjust the display method of the analysis result based on the estimated user's emotion. The analysis unit, for example, estimates the user's emotion and adjusts the display method of the analysis result based on the estimated user's emotion. Specific methods and standards for estimating emotion include, for example, facial expression recognition, voice analysis, and the like, but are not limited to such examples. For example, the analysis unit displays the analysis result concisely when the user is stressed. In addition, when the user is relaxed, it can also display detailed analysis results. Furthermore, when the user is excited, it can also display the analysis result in a visually attractive format. Thereby, by adjusting the display method of the analysis result according to the user's emotion, it is possible to display it in a way that is easy for the user to understand. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0091] The analysis unit can perform the analysis while considering the installation location information of the home appliance when analyzing the operation log. For example, the analysis unit performs the analysis while considering the installation location information of the home appliance when analyzing the operation log. Specific types and acquisition methods of the installation location information include, for example, GPS information, room layout information, and the like, but are not limited to such examples. For example, the analysis unit can consider the usage conditions specific to the kitchen when analyzing the operation log of a home appliance installed in the kitchen. In addition, the analysis unit can also consider the usage conditions specific to the living room when analyzing the operation log of a home appliance installed in the living room. Furthermore, the analysis unit can also consider the usage conditions specific to the bathroom when analyzing the operation log of a home appliance installed in the bathroom. As a result, by considering the installation location information, more accurate analysis is possible.

[0092] The analysis unit can improve the accuracy of the analysis when analyzing the operation log by referring to the past maintenance history of the home appliance. For example, the analysis unit improves the accuracy of the analysis when analyzing the operation log by referring to the past maintenance history of the home appliance. Specific contents and acquisition methods of the maintenance history include, for example, repair history and inspection history, but are not limited to such examples. For example, the analysis unit refers to the past maintenance history to detect early signs of failure. In addition, it is also possible to predict deterioration of a specific part based on the maintenance history. Furthermore, it is also possible to improve the reliability of the analysis result by taking the maintenance history into consideration. As a result, the accuracy of the analysis is improved by referring to the past maintenance history.

[0093] The fault suggestion unit can estimate the user's emotion and adjust the way of expressing the proposal based on the estimated user's emotion. The fault suggestion unit, for example, estimates the user's emotion and adjusts the way of expressing the proposal based on the estimated user's emotion. Specific methods and standards for estimating the emotion include, for example, facial expression recognition, voice analysis, and the like, but are not limited to such examples. For example, when the user is stressed, the fault suggestion unit makes a concise and easy-to-understand proposal. In addition, when the user is relaxed, the fault suggestion unit can also make a detailed proposal. Furthermore, when the user is excited, the proposal can also be made in a visually attractive format. This allows for more appropriate proposals by adjusting the way of expressing the proposal according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0094] The fault suggestion unit can adjust the level of detail of the proposal based on the importance of the home appliance when proposing a fault. For example, the fault suggestion unit adjusts the level of detail of the proposal based on the importance of the home appliance when proposing a fault. Specific criteria and methods for determining the importance of the home appliance include, for example, the frequency of use and the impact of the fault, but are not limited to such examples. For example, the fault suggestion unit provides detailed information for a fault proposal of a home appliance with high importance. Also, the fault suggestion unit can provide concise information for a fault proposal of a home appliance with low importance. Furthermore, the priority of the proposal can be adjusted according to the importance. As a result, important information can be provided preferentially by adjusting the level of detail of the proposal based on the importance of the home appliance.

[0095] The fault suggestion unit can apply different proposal algorithms depending on the category of the home appliance when proposing a fault. For example, the fault suggestion unit applies different proposal algorithms depending on the category of the home appliance when proposing a fault. Specific classification methods and standards for the categories of home appliances include, for example, kitchen appliances, living room appliances, etc., but are not limited to such examples. Specific types and implementation methods of the proposal algorithm include, for example, rule-based, machine learning-based, etc., but are not limited to such examples. For example, the fault suggestion unit applies an algorithm specialized in cooling performance to a fault suggestion for a refrigerator. Also, an algorithm specialized in cleaning performance can be applied to a fault suggestion for a washing machine. Furthermore, an algorithm specialized in heating and cooling performance can be applied to a fault suggestion for an air conditioner. As a result, the accuracy of the proposal is improved by applying a proposal algorithm according to the category of the home appliance.

[0096] The fault suggestion unit can estimate the user's emotion and determine the priority of the suggestions based on the estimated user's emotion. The fault suggestion unit, for example, estimates the user's emotion and determines the priority of the suggestions based on the estimated user's emotion. Specific methods and standards for estimating the emotion include, for example, facial expression recognition, voice analysis, and the like, but are not limited to such examples. For example, the fault suggestion unit prioritizes important suggestions when the user is stressed. In addition, detailed suggestions can be made when the user is relaxed. Furthermore, suggestions can be made in a visually attractive format when the user is excited. In this way, important suggestions can be prioritized by determining the priority of the suggestions according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI can be, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0097] The fault suggestion unit can make a proposal taking into consideration installation location information of the home appliance when making a fault suggestion. For example, the fault suggestion unit makes a proposal taking into consideration installation location information of the home appliance when making a fault suggestion. Specific types and acquisition methods of the installation location information include, for example, GPS information, room layout information, and the like, but are not limited to such examples. For example, the fault suggestion unit may make a fault suggestion for a home appliance installed in a kitchen by taking into consideration a usage situation specific to the kitchen. Furthermore, a fault suggestion for a home appliance installed in a living room may also take into consideration a usage situation specific to the living room. Furthermore, a fault suggestion for a home appliance installed in a bathroom may also take into consideration a usage situation specific to the bathroom. As a result, more accurate suggestions can be made by taking into consideration the installation location information.

[0098] The fault suggestion unit can improve the accuracy of the proposal by referring to the past fault history of the home appliance when proposing a fault. The fault suggestion unit, for example, improves the accuracy of the proposal by referring to the past fault history of the home appliance when proposing a fault. Specific contents and acquisition methods of the fault history include, for example, repair history, error code, etc., but are not limited to such examples. The fault suggestion unit, for example, refers to the past fault history to detect early signs of a fault. In addition, it is also possible to predict deterioration of a specific part based on the fault history. Furthermore, it is also possible to improve the reliability of the proposal by taking the fault history into consideration. As a result, the accuracy of the proposal is improved by referring to the past fault history.

[0099] The voice unit can estimate the user's emotion and adjust the tone and speed of the voice based on the estimated user's emotion. The voice unit, for example, estimates the user's emotion and adjusts the tone and speed of the voice based on the estimated user's emotion. Specific methods and standards for estimating emotion include, for example, facial expression recognition, voice analysis, and the like, but are not limited to such examples. For example, when the user is stressed, the voice unit can speak slowly in a calm tone. Also, when the user is relaxed, the voice unit can speak in a bright tone. Furthermore, when the user is excited, the voice unit can speak in an energetic tone. This allows for more appropriate voice output by adjusting the tone and speed of the voice according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0100] The audio unit can customize the content of the audio based on the usage status of the home appliance when outputting the audio. For example, the audio unit customizes the content of the audio based on the usage status of the home appliance when outputting the audio. Specific content and acquisition method of the usage status include, for example, the number of times of use and the usage time, but are not limited to such examples. For example, the audio unit provides information on the cooling function based on the usage status of the refrigerator. Also, it can provide information on the cleaning function based on the usage status of the washing machine. Furthermore, it can provide information on the heating and cooling function based on the usage status of the air conditioner. In this way, by customizing the content of the audio based on the usage status of the home appliance, it is possible to provide information that is useful to the user.

[0101] The voice unit can apply different voice algorithms depending on the type of home appliance when outputting voice. For example, the voice unit applies different voice algorithms depending on the type of home appliance when outputting voice. Specific classification methods and standards for the types of home appliances include, for example, refrigerators, washing machines, and air conditioners, but are not limited to such examples. Specific types and implementation methods of voice algorithms include, for example, voice synthesis algorithms and voice recognition algorithms, but are not limited to such examples. For example, the voice unit applies an algorithm specialized in cooling performance to the voice output of a refrigerator. Also, an algorithm specialized in cleaning performance can be applied to the voice output of a washing machine. Furthermore, an algorithm specialized in heating and cooling performance can be applied to the voice output of an air conditioner. As a result, the accuracy of the voice output is improved by applying a voice algorithm according to the type of home appliance.

[0102] The voice unit can estimate the user's emotion and determine the priority of the voice based on the estimated user's emotion. The voice unit, for example, estimates the user's emotion and determines the priority of the voice based on the estimated user's emotion. Specific methods and standards for estimating emotion include, for example, facial expression recognition, voice analysis, and the like, but are not limited to such examples. For example, when the user is stressed, the voice unit can preferentially convey important information by voice. In addition, when the user is relaxed, detailed information can also be conveyed by voice. Furthermore, when the user is excited, the voice can also be conveyed in a visually attractive format. In this way, important information can be preferentially conveyed by determining the priority of the voice according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI can be, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0103] The audio unit can adjust the content of the audio in consideration of the installation location information of the home appliance when outputting the audio. For example, the audio unit adjusts the content of the audio in consideration of the installation location information of the home appliance when outputting the audio. Specific types and acquisition methods of the installation location information include, for example, GPS information, room layout information, and the like, but are not limited to such examples. For example, the audio unit can consider the usage situation specific to the kitchen when outputting the audio of an appliance installed in the kitchen. Also, the audio output of an appliance installed in the living room can also consider the usage situation specific to the living room. Furthermore, the audio output of an appliance installed in the bathroom can also consider the usage situation specific to the bathroom. As a result, more appropriate audio output is possible by considering the installation location information.

[0104] The audio unit can improve the accuracy of the audio by referring to the past audio history of the home appliance when outputting the audio. The audio unit can improve the accuracy of the audio by referring to the past audio history of the home appliance when outputting the audio. Specific contents and acquisition methods of the audio history include, for example, past audio output contents, audio recognition results, etc., but are not limited to such examples. The audio unit can provide audio that is easy for the user to understand by referring to the past audio history, for example. In addition, specific information can be emphasized based on the audio history. Furthermore, the reliability of the audio output can be improved by taking the audio history into consideration. As a result, the accuracy of the audio output is improved by referring to the past audio history.

[0105] The detection unit can estimate the user's emotion and adjust the method of outing detection based on the estimated user's emotion. The detection unit, for example, estimates the user's emotion and adjusts the method of outing detection based on the estimated user's emotion. Specific methods and standards for estimating emotion include, for example, facial expression recognition, voice analysis, and the like, but are not limited to such examples. For example, the detection unit performs quick and concise outing detection when the user is stressed. In addition, detailed outing detection can also be performed when the user is relaxed. Furthermore, outing detection can be performed in a visually appealing format when the user is excited. This allows for more appropriate outing detection by adjusting the method of outing detection according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0106] The detection unit can improve the accuracy of detection when detecting an outing by referring to the user's past outing history. The detection unit, for example, improves the accuracy of detection when detecting an outing by referring to the user's past outing history. Specific contents and acquisition methods of the outing history include, for example, past outing times and outing destinations, but are not limited to such examples. The detection unit, for example, improves accuracy by referring to the past outing history and learning outing patterns. In addition, it is also possible to predict outings during specific time periods based on the outing history. Furthermore, it is also possible to improve the reliability of outing detection by taking the outing history into consideration. As a result, the accuracy of outing detection is improved by referring to the past outing history.

[0107] The detection unit can perform detection while taking into consideration the schedule information of the user when detecting an outing. For example, the detection unit performs detection while taking into consideration the schedule information of the user when detecting an outing. Specific contents and acquisition methods of the schedule information include, for example, calendar information, a timetable, and the like, but are not limited to such examples. For example, the detection unit refers to the calendar information of the user and detects an outing based on a plan. In addition, it is also possible to predict an outing related to a specific event from the schedule information. Furthermore, it is also possible to improve the accuracy of outing detection based on the schedule information. As a result, more accurate outing detection is possible by taking into consideration the schedule information.

[0108] The detection unit can estimate the user's emotion and determine the priority of outing detection based on the estimated user's emotion. The detection unit, for example, estimates the user's emotion and determines the priority of outing detection based on the estimated user's emotion. Specific methods and standards for estimating emotion include, for example, facial expression recognition, voice analysis, and the like, but are not limited to such examples. For example, when the user is stressed, the detection unit prioritizes important outing detection. In addition, when the user is relaxed, detailed outing detection can also be performed. Furthermore, when the user is excited, outing detection can be performed in a visually attractive format. In this way, by determining the priority of outing detection according to the user's emotion, important outing detection can be prioritized. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0109] The detection unit can perform detection while taking into consideration the geographical location information of the user when detecting going out. For example, the detection unit performs detection while taking into consideration the geographical location information of the user when detecting going out. Specific contents and acquisition methods of the geographical location information include, for example, GPS information, location information services, etc., but are not limited to such examples. For example, the detection unit acquires the user's current location in real time and detects going out. In addition, it is also possible to predict going out at a specific location based on the geographical location information. Furthermore, it is also possible to improve the accuracy of going out detection by taking into consideration the geographical location information. As a result, more accurate detection of going out is possible by taking into consideration the geographical location information.

[0110] The detection unit can improve the accuracy of detection when detecting going out by analyzing the social media activity of the user. For example, the detection unit improves the accuracy of detection when detecting going out by analyzing the social media activity of the user. Specific contents and acquisition methods of the social media activity include, for example, posting contents, activity frequency, etc., but are not limited to such examples. The detection unit, for example, analyzes the social media posts of the user and learns the pattern of going out to improve the accuracy. In addition, it is also possible to predict going out related to a specific event based on the social media activity. Furthermore, it is also possible to improve the reliability of the outing detection by taking into account the social media activity. Thereby, the accuracy of the outing detection is improved by analyzing the social media activity.

[0111] The power-off suggestion unit can estimate the user's emotion and adjust the way of expressing the power-off suggestion based on the estimated user's emotion. The power-off suggestion unit, for example, estimates the user's emotion and adjusts the way of expressing the power-off suggestion based on the estimated user's emotion. Specific methods and standards for estimating emotion include, for example, facial expression recognition, voice analysis, and the like, but are not limited to such examples. For example, when the user is stressed, the power-off suggestion unit makes a concise and easy-to-understand suggestion. In addition, when the user is relaxed, it can also make a detailed suggestion. Furthermore, when the user is excited, it can also make a suggestion in a visually attractive format. This allows for more appropriate suggestions by adjusting the way of expressing the suggestion according to the user's emotion. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0112] The power-off suggestion unit can adjust the level of detail of the proposal based on the usage status of the home appliance when proposing to turn off the power. For example, the power-off suggestion unit adjusts the level of detail of the proposal based on the usage status of the home appliance when proposing to turn off the power. Specific contents and acquisition methods of the usage status include, for example, the number of times of use, the usage time, etc., but are not limited to such examples. For example, the power-off suggestion unit provides detailed information for a power-off suggestion for a home appliance that is used frequently. Also, the power-off suggestion unit can provide concise information for a power-off suggestion for a home appliance that is used infrequently. Furthermore, the priority of the proposal can be adjusted according to the usage status. As a result, important information can be provided preferentially by adjusting the level of detail of the proposal based on the usage status of the home appliance.

[0113] The power-off suggestion unit can apply different proposal algorithms depending on the type of home appliance when proposing to turn off the power. For example, the power-off suggestion unit applies different proposal algorithms depending on the type of home appliance when proposing to turn off the power. Specific classification methods and standards for the types of home appliances include, for example, refrigerators, washing machines, and air conditioners, but are not limited to such examples. Specific types and implementation methods of the proposal algorithm include, for example, rule-based, machine learning-based, and the like, but are not limited to such examples. For example, the power-off suggestion unit applies an algorithm specialized in cooling performance to the proposal to turn off the refrigerator. Also, an algorithm specialized in cleaning performance can be applied to the proposal to turn off the washing machine. Furthermore, an algorithm specialized in heating and cooling performance can be applied to the proposal to turn off the air conditioner. Thereby, by applying a proposal algorithm according to the type of home appliance, the accuracy of the proposal is improved.

[0114] The power-off suggestion unit can estimate the user's emotion and determine the priority of power-off suggestions based on the estimated user's emotion. The power-off suggestion unit, for example, estimates the user's emotion and determines the priority of power-off suggestions based on the estimated user's emotion. Specific methods and standards for estimating emotion include, for example, facial expression recognition, voice analysis, and the like, but are not limited to such examples. For example, the power-off suggestion unit prioritizes important suggestions when the user is stressed. In addition, detailed suggestions can be made when the user is relaxed. Furthermore, suggestions can be made in a visually attractive format when the user is excited. In this way, important suggestions can be prioritized by determining the priority of suggestions according to the user's emotion. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI can be, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0115] The power-off suggestion unit can make a proposal taking into consideration installation location information of the home appliance when proposing to turn off the power. For example, the power-off suggestion unit makes a proposal taking into consideration installation location information of the home appliance when proposing to turn off the power. Specific types and acquisition methods of the installation location information include, for example, GPS information, room layout information, and the like, but are not limited to such examples. For example, the power-off suggestion unit may take into consideration a usage situation specific to the kitchen when proposing to turn off a home appliance installed in a kitchen. Also, the power-off suggestion unit may take into consideration a usage situation specific to the living room when proposing to turn off a home appliance installed in a bathroom. In this way, more accurate suggestions can be made by taking into consideration the installation location information.

[0116] The power-off suggestion unit can improve the accuracy of the suggestion by referring to the past power-off history of the home appliance when proposing to turn off the power. For example, the power-off suggestion unit improves the accuracy of the suggestion by referring to the past power-off history of the home appliance when proposing to turn off the power. Specific contents and acquisition methods of the power-off history include, for example, past power-off times and power-off frequencies, but are not limited to such examples. The power-off suggestion unit improves accuracy by, for example, referring to the past power-off history and learning power-off patterns. In addition, it is also possible to predict power-off in a specific time period based on the power-off history. Furthermore, it is also possible to improve the reliability of the suggestion by taking the power-off history into consideration. As a result, the accuracy of the suggestion is improved by referring to the past power-off history.

[0117] The talk generation unit can estimate the user's emotion and adjust the content of the talk based on the estimated user's emotion. The talk generation unit, for example, estimates the user's emotion and adjusts the content of the talk based on the estimated user's emotion. Specific methods and standards for estimating emotion include, for example, facial expression recognition, voice analysis, and the like, but are not limited to such examples. For example, when the user is stressed, the talk generation unit generates a concise and easy-to-understand talk. In addition, when the user is relaxed, detailed talk can also be generated. Furthermore, when the user is excited, talk can also be generated in a visually attractive format. In this way, by adjusting the content of the talk according to the user's emotion, a more appropriate talk is generated. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0118] The talk generation unit can customize the content of the talk based on the usage status of the home appliance when generating the talk. For example, the talk generation unit customizes the content of the talk based on the usage status of the home appliance when generating the talk. Specific content and acquisition method of the usage status include, for example, the number of times of use and the usage time, but are not limited to such examples. For example, the talk generation unit generates talk regarding a cooling function based on the usage status of a refrigerator. Also, it is possible to generate talk regarding a cleaning function based on the usage status of a washing machine. Furthermore, it is possible to generate talk regarding a heating and cooling function based on the usage status of an air conditioner. In this way, by customizing the content of the talk based on the usage status of the home appliance, it is possible to provide useful information to the user.

[0119] The talk generation unit can apply different talk algorithms depending on the type of home appliance when generating talk. For example, the talk generation unit applies different talk algorithms depending on the type of home appliance when generating talk. Specific classification methods and standards for the types of home appliances include, for example, refrigerators, washing machines, and air conditioners, but are not limited to such examples. Specific types and implementation methods of talk algorithms include, for example, natural language generation algorithms and machine learning algorithms, but are not limited to such examples. For example, the talk generation unit applies an algorithm specialized in cooling performance to talk generation for a refrigerator. Also, an algorithm specialized in cleaning performance can be applied to talk generation for a washing machine. Furthermore, an algorithm specialized in heating and cooling performance can be applied to talk generation for an air conditioner. As a result, the accuracy of the talk is improved by applying a talk algorithm according to the type of home appliance.

[0120] The talk generation unit can estimate the user's emotion and determine the priority of talks based on the estimated user's emotion. The talk generation unit, for example, estimates the user's emotion and determines the priority of talks based on the estimated user's emotion. Specific methods and standards for estimating emotion include, for example, facial expression recognition, voice analysis, and the like, but are not limited to such examples. For example, the talk generation unit generates important talks preferentially when the user is stressed. In addition, when the user is relaxed, detailed talks can also be generated. Furthermore, when the user is excited, talks can also be generated in a visually attractive format. In this way, important talks can be generated preferentially by determining the priority of talks according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples.

[0121] The talk generation unit can adjust the content of the talk in consideration of the installation location information of the home appliance when generating the talk. For example, the talk generation unit adjusts the content of the talk in consideration of the installation location information of the home appliance when generating the talk. Specific types and acquisition methods of the installation location information include, for example, GPS information, room layout information, and the like, but are not limited to such examples. For example, the talk generation unit generates talk for a home appliance installed in a kitchen by considering a usage situation specific to the kitchen. Also, the talk generation for a home appliance installed in a living room can also consider a usage situation specific to the living room. Furthermore, the talk generation for a home appliance installed in a bathroom can also consider a usage situation specific to the bathroom. In this way, more appropriate talk is generated by considering the installation location information.

[0122] The talk generation unit can improve the accuracy of the talk by referring to the past talk history of the home appliance when generating the talk. The talk generation unit, for example, improves the accuracy of the talk by referring to the past talk history of the home appliance when generating the talk. Specific contents and acquisition methods of the talk history include, for example, past talk contents, talk frequency, etc., but are not limited to such examples. The talk generation unit, for example, refers to the past talk history to generate a talk that is easy for the user to understand. In addition, specific information can be emphasized based on the talk history. Furthermore, the reliability of the talk generation can be improved by taking the talk history into consideration. As a result, the accuracy of the talk is improved by referring to the past talk history.

[0123] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0124] The home appliance dialogue system may further include a health monitoring unit that monitors the user's health condition. The health monitoring unit, for example, measures the user's heart rate and body temperature with a sensor, and notifies the user if an abnormality is detected. For example, if the heart rate is abnormally high, the unit notifies the user, "Your heart rate is high. Please take a break." Also, if the body temperature is high, the unit may suggest, "Your body temperature is high. Please drink fluids." Furthermore, the health monitoring unit may accumulate the user's health data and support long-term health management. This makes it possible to monitor the user's health condition in real time and provide appropriate advice.

[0125] The home appliance dialogue system can further include an energy management unit that optimizes energy consumption. The energy management unit, for example, monitors the energy consumption of each home appliance in real time and suggests energy-efficient usage methods. For example, it suggests reducing energy consumption by optimizing the temperature setting of an air conditioner. The energy management unit can also adjust the usage schedule of the home appliances to reduce energy consumption during peak hours. Furthermore, the energy management unit can analyze past energy consumption data and support long-term energy efficiency improvements. This optimizes energy consumption, contributing to cost reduction and environmental protection.

[0126] The home appliance dialogue system may further include a customization unit that customizes the operation of the home appliance based on the user's preferences. The customization unit, for example, learns the user's past usage history and settings and proposes an operation that matches the user's preferences. For example, the temperature setting of an air conditioner may be automatically adjusted to match the user's preferences. The customization unit may also collect user feedback and improve the accuracy of the operation. Furthermore, the customization unit may also propose new functions and services based on the user's preferences. This enables customization to match the user's preferences, providing a more comfortable usage experience.

[0127] The home appliance dialogue system can further include a maintenance unit that supports the maintenance of home appliances. The maintenance unit, for example, analyzes the usage status and operation logs of the home appliance and notifies the user of the need for regular maintenance. For example, it notifies the user when it is time to clean the filter or replace parts. The maintenance unit can also provide maintenance procedures by audio or video to support the user in easily performing maintenance. Furthermore, the maintenance unit can manage past maintenance history and create future maintenance plans. This can extend the lifespan of the home appliance and prevent breakdowns.

[0128] The home appliance dialogue system may further include an emotion adjustment unit that estimates the user's emotion and adjusts the operation of the home appliance based on the estimated user's emotion. The emotion adjustment unit, for example, analyzes the user's facial expression and voice to estimate the emotion. If the user is feeling stressed, the temperature of the air conditioner may be adjusted to a comfortable range. Also, if the user is relaxed, the brightness of the lighting may be adjusted to create a relaxing atmosphere. Furthermore, if the user is excited, music may be played to calm the user. This makes it possible to adjust the operation of the home appliance according to the user's emotion, and a more comfortable living environment can be provided.

[0129] The home appliance dialogue system may further include a notification adjustment unit that estimates the user's emotions and adjusts the content and timing of notifications based on the estimated user's emotions. The notification adjustment unit, for example, analyzes the user's facial expressions and voice to estimate the user's emotions. When the user is stressed, only important notifications are given priority. When the user is relaxed, detailed notifications can be given. Furthermore, when the user is excited, notifications can be given in a visually appealing format. This makes it possible to adjust notifications according to the user's emotions, and provide more appropriate information.

[0130] The home appliance dialogue system may further include a coordination adjustment unit that estimates the user's emotions and adjusts the coordinated operation of the home appliances based on the estimated user's emotions. The coordination adjustment unit, for example, analyzes the user's facial expressions and voice to estimate the emotions. If the user is stressed, the coordination adjustment unit simplifies the coordinated operation of the home appliances. If the user is relaxed, a scenario in which multiple home appliances work together can be suggested. Furthermore, if the user is excited, the coordination operation can be suggested in a visually attractive format. This enables the home appliances to work together in accordance with the user's emotions, providing a more comfortable user experience.

[0131] The home appliance dialogue system may further include an analysis customization unit that estimates the user's emotions and customizes the analysis results of the operation log of the home appliance based on the estimated user's emotions. The analysis customization unit, for example, analyzes the user's facial expressions and voice to estimate the emotions. If the user is stressed, a concise analysis result is provided. Also, if the user is relaxed, a detailed analysis result can be provided. Furthermore, if the user is excited, the analysis result can be provided in a visually attractive format. This makes it possible to customize the analysis result according to the user's emotions, and provide more appropriate information.

[0132] The home appliance dialogue system may further include a schedule adjustment unit that estimates the user's emotions and adjusts the operation schedule of the home appliance based on the estimated user's emotions. The schedule adjustment unit, for example, analyzes the user's facial expressions and voice to estimate the emotions. If the user is stressed, the operation schedule of the home appliance is simplified. Also, if the user is relaxed, a detailed schedule can be proposed. Furthermore, if the user is excited, a schedule can be proposed in a visually attractive format. This makes it possible to adjust the operation schedule according to the user's emotions, and provides a more comfortable usage experience.

[0133] The home appliance dialogue system may further include a mode adjustment unit that estimates the user's emotion and adjusts the operation mode of the home appliance based on the estimated user's emotion. The mode adjustment unit, for example, analyzes the user's facial expression and voice to estimate the emotion. If the user is stressed, the operation mode of the home appliance is switched to a relaxation mode. Also, if the user is relaxed, the home appliance may be operated in a normal mode. Furthermore, if the user is excited, the home appliance may be operated in an energetic mode. This makes it possible to adjust the operation mode according to the user's emotion, and provides a more comfortable usage experience.

[0134] The process flow of the second embodiment will be briefly described below.

[0135] Step 1: The analysis unit analyzes the operation log of the home appliance. The operation log includes, for example, time, events, error codes, etc., but is not limited to these examples. The analysis unit analyzes the operation log using, for example, data mining technology or statistical analysis technology, and determines the operation status of the home appliance. For example, the analysis unit detects abnormal patterns in the operation log and identifies signs of failure or malfunction. Step 2: The fault suggestion unit proposes solutions based on the information analyzed by the analysis unit. Solutions include, for example, repair procedures and configuration changes. For example, the analysis unit proposes a method of solving the problem by appropriate repair procedures and configuration changes for the anomaly detected by the analysis unit. Step 3: The voice unit communicates the solution proposed by the fault suggestion unit to the user by voice. The voice unit outputs the solution by voice using, for example, voice synthesis technology, and adjusts the tone and speed of the voice to provide a voice that is easy for the user to hear. Step 4: The detection unit detects that the user has gone out. The detection unit detects that the user has gone out, for example, by using location information or a motion sensor. For example, the detection unit obtains location information of the user's smartphone and detects that the user has gone out. Step 5: The power-off suggestion unit suggests turning off the home appliance when the detection unit detects that the user has left the house. The power-off suggestion unit suggests turning off the home appliance at a timing appropriate for the user, for example, by adjusting the notification method or the timing of the suggestion. For example, the power-off suggestion unit suggests turning off the air conditioner immediately after leaving the house. Step 6: The conversation generation unit generates conversations for the user through communication between home appliances. The conversation generation unit generates conversations using, for example, natural language generation technology, and adjusts the content and format of the conversation to provide conversations that are easy for the user to understand. For example, the washing machine and dryer send a message to the user saying, "Washing is done. Do you want to start drying?"

[0136] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0137] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0138] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0139] For example, the analysis unit, the fault suggestion unit, the voice unit, the detection unit, the power off suggestion unit, and the talk generation unit are realized by at least one of the smart device 14 and the data processing device 12. For example, the analysis unit is realized by the specific processing unit 290 of the data processing device 12, and analyzes the operation log of the home appliance. The fault suggestion unit proposes a solution based on the information analyzed by the analysis unit, and is realized by the specific processing unit 290 of the data processing device 12. The voice unit is realized by the control unit 46A of the smart device 14, and conveys the proposed solution to the user by voice. The detection unit detects the user's going out using the camera 42 or the microphone 38B of the smart device 14. The power off suggestion unit proposes turning off the power of the home appliance when the detection unit detects the user's going out, and is realized by the specific processing unit 290 of the data processing device 12. The talk generation unit is realized by the control unit 46A of the smart device 14, and the home appliances communicate with each other to generate talk for the user. The correspondence between each unit and the device or control unit is not limited to the above example, and various changes are possible.

[0140] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0141] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0142] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0143] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0144] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0145] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0146] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0147] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0148] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0149] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0150] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0151] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0152] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0153] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0154] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0155] For example, the analysis unit, the fault suggestion unit, the voice unit, the detection unit, the power off suggestion unit, and the talk generation unit are realized by at least one of the smart glasses 214 and the data processing device 12. For example, the analysis unit is realized by the specific processing unit 290 of the data processing device 12, and analyzes the operation log of the home appliance. The fault suggestion unit proposes a solution based on the information analyzed by the analysis unit, and is realized by the specific processing unit 290 of the data processing device 12. The voice unit is realized by the control unit 46A of the smart glasses 214, and conveys the proposed solution to the user by voice. The detection unit detects the user's going out using the camera 42 and the microphone 238 of the smart glasses 214. The power off suggestion unit proposes turning off the power of the home appliance when the detection unit detects the user's going out, and is realized by the specific processing unit 290 of the data processing device 12. The talk generation unit is realized by the control unit 46A of the smart glasses 214, and the home appliances communicate with each other to generate talk for the user. The correspondence between each unit and the device or control unit is not limited to the above example, and various changes are possible.

[0156] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0157] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0158] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0159] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0160] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0161] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0162] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0163] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0164] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0165] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0166] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0167] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0168] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0169] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0170] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0171] For example, the analysis unit, the fault suggestion unit, the voice unit, the detection unit, the power off suggestion unit, and the talk generation unit are realized by at least one of the headset type terminal 314 and the data processing device 12. For example, the analysis unit is realized by the specific processing unit 290 of the data processing device 12, and analyzes the operation log of the home appliance. The fault suggestion unit proposes a solution based on the information analyzed by the analysis unit, and is realized by the specific processing unit 290 of the data processing device 12. The voice unit is realized by the control unit 46A of the headset type terminal 314, and conveys the proposed solution to the user by voice. The detection unit detects the user's going out using the camera 42 and the microphone 238 of the headset type terminal 314. The power off suggestion unit proposes turning off the power of the home appliance when the detection unit detects the user's going out, and is realized by the specific processing unit 290 of the data processing device 12. The talk generation unit is realized by the control unit 46A of the headset type terminal 314, and the home appliances communicate with each other to generate talk for the user. The correspondence between each unit and the device or control unit is not limited to the above example, and various changes are possible.

[0172] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0173] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0174] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0175] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0176] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0177] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0178] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0179] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0180] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0181] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0182] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0183] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0184] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0185] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0186] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0187] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0188] For example, the analysis unit, the fault suggestion unit, the voice unit, the detection unit, the power off suggestion unit, and the talk generation unit are realized by at least one of the robot 414 and the data processing device 12. For example, the analysis unit is realized by the specific processing unit 290 of the data processing device 12, and analyzes the operation log of the home appliance. The fault suggestion unit proposes a solution based on the information analyzed by the analysis unit, and is realized by the specific processing unit 290 of the data processing device 12. The voice unit is realized by the control unit 46A of the robot 414, and conveys the proposed solution to the user by voice. The detection unit detects the user's going out using the camera 42 and microphone 238 of the robot 414. The power off suggestion unit proposes turning off the power of the home appliance when the detection unit detects the user's going out, and is realized by the specific processing unit 290 of the data processing device 12. The talk generation unit is realized by the control unit 46A of the robot 414, and the home appliances communicate with each other to generate talk for the user. The correspondence between each unit and the device or control unit is not limited to the above example, and various changes are possible.

[0189] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0190] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0191] These emotions are distributed in the three o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0192] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0193] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0194] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0195] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0196] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0197] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0198] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0199] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0200] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0201] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0202] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0203] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0204] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0205] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0206] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0207] (Appendix 1) an analysis unit that analyzes an operation log; a fault suggestion unit that suggests a solution based on the information analyzed by the analysis unit; a voice unit for audibly transmitting the solution proposed by the fault suggestion unit to a user; A detection unit that detects when the user goes out; a power-off suggestion unit that suggests turning off the power of the home appliance when the detection unit detects that the person is going out; A talk generation unit that generates talks for the user by communicating with the home appliances. A system characterized by: (Appendix 2) The token generation unit is The home appliances communicate with each other to generate a token for the user. A system characterized by: (Appendix 3) The analysis unit is Analyzing the operation log 2. The system of claim 1, (Appendix 4) The fault suggestion unit is Proposing a solution based on the information analyzed by the analysis unit 2. The system of claim 1, (Appendix 5) The audio section includes: The solution proposed by the fault suggestion unit is communicated to the user by voice. 2. The system of claim 1, (Appendix 6) The detection unit is Detect when the user goes out 2. The system of claim 1, (Appendix 7) The power off suggestion unit is When the detection unit detects that the user is leaving the home, the device suggests turning off the power to the home appliance. 2. The system of claim 1, (Appendix 8) The analysis unit is Inferring user emotions and adjusting the analysis method of the action log based on the estimated user emotions A system characterized by: (Appendix 9) The analysis unit is When analyzing operation logs, the analysis priority is determined based on the frequency of use of home appliances. 2. The system of claim 1, (Appendix 10) The analysis unit is When analyzing operation logs, different analysis algorithms are applied depending on the type of home appliance. 2. The system of claim 1, (Appendix 11) The analysis unit is Infer user emotions and adjust the way analysis results are displayed based on the inferred user emotions A system characterized by: (Appendix 12) The analysis unit is When analyzing operation logs, the location information of the home appliances is taken into consideration. 2. The system of claim 1, (Appendix 13) The analysis unit is When analyzing operation logs, the accuracy of the analysis can be improved by referring to the past maintenance history of the home appliances. 2. The system of claim 1, (Appendix 14) The fault suggestion unit is Infer user emotions and adjust the presentation of suggestions based on the inferred user emotions A system characterized by: (Appendix 15) The fault suggestion unit is When making fault suggestions, adjust the level of detail of the suggestions based on the importance of the appliance 2. The system of claim 1, (Appendix 16) The fault suggestion unit is When making fault suggestions, different suggestion algorithms are applied depending on the category of the appliance. 2. The system of claim 1, (Appendix 17) The fault suggestion unit is Infer user sentiment and prioritize suggestions based on the inferred user sentiment A system characterized by: (Appendix 18) The fault suggestion unit is When making repair suggestions, the system takes into account the location of the appliance. 2. The system of claim 1, (Appendix 19) The fault suggestion unit is When making repair suggestions, the accuracy of the suggestions will be improved by referring to the appliance's past repair history. 2. The system of claim 1, (Appendix 20) The audio section includes: Infer user emotion and adjust speech tone and speed based on the estimated user emotion A system characterized by: (Appendix 21) The audio section includes: When outputting audio, customize the audio content based on the usage of the home appliance 2. The system of claim 1, (Appendix 22) The audio section includes: When outputting audio, different audio algorithms are applied depending on the type of home appliance. 2. The system of claim 1, (Appendix 23) The audio section includes: Estimating user emotions and prioritizing audio based on the estimated user emotions A system characterized by: (Appendix 24) The audio section includes: When outputting audio, the content of the audio is adjusted taking into account the location information of the home appliance. 2. The system of claim 1, (Appendix 25) The audio section includes: When outputting voice, the accuracy of the voice is improved by referencing the appliance's past voice history. 2. The system of claim 1, (Appendix 26) The detection unit is A method for detecting an out-of-home presence is provided, comprising: estimating a user's emotion; and adjusting a method for detecting an out-of-home presence based on the estimated user's emotion. A system characterized by: (Appendix 27) The detection unit is When detecting an outing, improve the accuracy of the detection by referring to the user's past outing history 2. The system of claim 1, (Appendix 28) The detection unit is When detecting an out-of-home visit, the system takes into account the user's schedule information. 2. The system of claim 1, (Appendix 29) The detection unit is A user's emotion is estimated, and a priority is determined for out-of-home detection based on the estimated user's emotion. A system characterized by: (Appendix 30) The detection unit is When detecting whether a user is out, the detection is performed taking into account the user's geographic location information. 2. The system of claim 1, (Appendix 31) The detection unit is When detecting when someone is out, analyze the user's social media activity to improve detection accuracy. 2. The system of claim 1, (Appendix 32) The power off suggestion unit is Estimate the user's emotions and adjust the way the power off suggestion is presented based on the estimated user's emotions. A system characterized by: (Appendix 33) The power off suggestion unit is When suggesting power off, adjust the level of detail of the suggestion based on the usage of the appliance 2. The system of claim 1, (Appendix 34) The power off suggestion unit is When suggesting power off, different suggestion algorithms are applied depending on the type of appliance. 2. The system of claim 1, (Appendix 35) The power off suggestion unit is Estimate user's emotions and prioritize power off suggestions based on the estimated user's emotions A system characterized by: (Appendix 36) The power off suggestion unit is When suggesting power off, make suggestions taking into account the location of home appliances 2. The system of claim 1, (Appendix 37) The power off suggestion unit is When suggesting power off, improve the accuracy of the suggestion by referring to the past power off history of the home appliance 2. The system of claim 1, (Appendix 38) The token generation unit is Estimate the user's emotions and adjust the content of the conversation based on the estimated user emotions A system characterized by: (Appendix 39) The token generation unit is When generating a chat, customize the chat content based on the usage of the home appliance 2. The system of claim 1, (Appendix 40) The token generation unit is When generating tokens, different token algorithms are applied depending on the type of home appliance. 2. The system of claim 1, (Appendix 41) The token generation unit is Estimate user emotions and prioritize talks based on the estimated user emotions A system characterized by: (Appendix 42) The token generation unit is When generating a conversation, the content of the conversation is adjusted taking into account the location information of the home appliances. 2. The system of claim 1, (Appendix 43) The token generation unit is When generating a chat, the accuracy of the chat is improved by referring to the appliance's past chat history. 2. The system of claim 1, [Explanation of symbols]

[0208] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. an analysis unit that analyzes an operation log; a fault suggestion unit that suggests a solution based on the information analyzed by the analysis unit; a voice unit for audibly transmitting the solution proposed by the fault suggestion unit to a user; A detection unit that detects when the user goes out; a power-off suggestion unit that suggests turning off the power of the home appliance when the detection unit detects that the person is going out; A talk generation unit that generates talks for the user by communicating with the home appliances. A system characterized by:

2. The analysis unit is The emotion of the user is estimated, and a method of analyzing the operation log is adjusted based on the estimated emotion of the user.

2. The system of claim 1 .

3. The analysis unit is When analyzing the operation log, the priority of the analysis is determined based on the frequency of use of the home appliance.

2. The system of claim 1 .

4. The analysis unit is When analyzing the operation log, different analysis algorithms are applied depending on the type of the home appliance.

2. The system of claim 1 .

Images