System

Abstract

To provide a system for learning an action pattern or a preference of a user to propose a predictive operation.SOLUTION: A system includes a reception unit, a control unit, a learning unit and a proposal unit. The reception unit receives an audio command of a user. The control unit operates an electric apparatus on the basis of the audio command received by the reception unit. The learning unit learns an action pattern or a preference of the user. The proposal unit proposes a predictive operation to the user on the basis of information learned by the learning unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technologies have limited ability to operate electrical devices based on user voice commands, and have a lack of predictive operation suggestions that take into account the user's behavioral patterns and preferences.

[0005] The system according to the embodiment aims to learn the behavioral patterns and preferences of the user and to suggest predictive operations. [Means for solving the problem]

[0006] The system according to the embodiment includes a reception unit, a control unit, a learning unit, and a suggestion unit. The reception unit receives a voice command from a user. The control unit operates an electric device based on the voice command received by the reception unit. The learning unit learns the user's behavioral patterns and preferences. The suggestion unit suggests to the user an operation predicted based on the information learned by the learning unit. Effect of the Invention

[0007] The system according to the embodiment can learn the user's behavioral patterns and preferences and suggest predictive operations. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The voice control system AI assistant for smart home according to the embodiment of the present invention is a system that accepts voice commands and supports the operation of home appliances, lighting, etc. This system learns the user's daily habits and preferences and suggests predictive operations. In addition, it provides information such as security alerts and energy usage status, and notifies the latest information such as weather and news by linking with the outside. For example, it accepts a voice command from the user. For example, it accepts a voice command such as "Turn on the living room light." This voice command is accepted by a reception unit equipped with a noise canceling function. Next, it operates an electric device based on the accepted voice command. For example, an operation to turn on the living room light is performed. This operation is executed by a control unit. In addition, it learns the user's daily habits and preferences. For example, if a user has a habit of turning on the living room light at 7 o'clock every morning, the learning unit learns that information using a machine learning algorithm. Based on the learned information, it suggests predictive operations. For example, when a user enters the living room, the suggestion unit suggests "Should I turn on the light?" This suggestion is made based on the learned information and the user's biometric information. It also provides a security alert. For example, when it detects that a door has been opened, the alert unit notifies the user. The energy usage status is monitored and power saving suggestions are made. For example, a monitoring unit equipped with an interface that visualizes the energy usage status in real time makes power saving suggestions to the user. The latest weather and news information is provided by linking with the outside. For example, the linking unit notifies the user of weather forecasts and news. In this way, the present invention is a voice control system AI assistant for a smart home that accepts voice commands, supports the operation of home appliances and lighting, etc., learns the user's daily habits and preferences to suggest predictive operations, provides information such as security alerts and energy usage status, and notifies the user of the latest information such as weather and news by linking with the outside. As a result, the voice control system AI assistant for a smart home can accept the user's voice commands, support the operation of home appliances and lighting, learns the user's daily habits and preferences, and suggests predictive operations.It can also provide information such as security alerts and energy usage, and by connecting with external parties it can provide the latest information on weather, news, etc.

[0029] The voice control system AI assistant for a smart home according to the embodiment includes a reception unit, a control unit, a learning unit, and a suggestion unit. The reception unit receives a voice command from a user. The user's voice command includes, for example, "Turn on the living room light," but is not limited to such an example. The reception unit receives the voice command using, for example, a microphone with a noise canceling function. The reception unit can also analyze the voice command using a voice recognition technology and convert it into text data. For example, the reception unit receives the voice command using a microphone with a noise canceling function, and converts the voice command into text data using a voice recognition technology. The control unit operates an electric device based on the voice command received by the reception unit. The electric device includes, for example, a light, an air conditioner, a television, and the like, but is not limited to such an example. The control unit performs an operation to turn on the living room light based on the voice command, for example. The control unit can also adjust the temperature of the air conditioner based on the voice command. For example, the control unit sets the temperature of the air conditioner based on the voice command. The learning unit learns the user's daily habits and preferences. The learning unit learns the user's daily habits and preferences, for example, using a machine learning algorithm. For example, the learning unit learns the user's habit of turning on the lights in the living room at 7:00 every morning. The learning unit can also learn the user's favorite music and TV programs. For example, the learning unit learns the user's favorite music playlist. The suggestion unit suggests to the user a predicted operation based on the information learned by the learning unit. For example, the suggestion unit suggests "Would you like to turn on the lights?" when the user enters the living room. The suggestion unit can also suggest playing music that the user likes. For example, the suggestion unit suggests "Would you like to play your favorite music?" when the user enters the living room. As a result, the AI ​​assistant of the smart home voice control system according to the embodiment can accept the user's voice commands, operate electrical appliances, learn the user's daily habits and preferences, and suggest predictive operations.

[0030] The reception unit receives a voice command from the user. The user's voice command includes, but is not limited to, for example, "Turn on the living room lights." The reception unit receives the voice command using, for example, a microphone with a noise canceling function. The noise canceling function can reduce ambient noise and capture the user's voice clearly. This makes it possible to accurately receive voice commands even in noisy environments. The reception unit can also analyze the voice command using voice recognition technology and convert it into text data. A model using deep learning is used for the voice recognition technology, and the voice data is analyzed with high accuracy. For example, the reception unit receives a voice command using a microphone with a noise canceling function and converts the voice command into text data using voice recognition technology. The voice command converted into text data is transmitted to the control unit, which is the next processing step. Furthermore, the reception unit can support multiple languages ​​and dialects, and can meet the diverse needs of users. For example, by supporting regional dialects in addition to major languages ​​such as English, Japanese, and Spanish, it is expected that the device will be used by a wider range of users. This enables the reception unit to receive the user's voice commands accurately and quickly, improving the operability of the entire system.

[0031] The control unit operates the electric appliance based on the voice command received by the reception unit. The electric appliance includes, but is not limited to, for example, a light, an air conditioner, and a television. For example, the control unit performs an operation of turning on the light in the living room based on the voice command. Specifically, the control unit analyzes the voice command and generates an operation command for the corresponding electric appliance. For example, when a voice command such as "Turn on the light in the living room" is transmitted from the reception unit, the control unit generates an instruction to turn on the light and transmits it to the lighting system. The control unit can also adjust the temperature of the air conditioner based on the voice command. For example, when the control unit receives a voice command such as "Set the temperature of the air conditioner to 25 degrees," the control unit generates an instruction to change the set temperature of the air conditioner to 25 degrees and transmits it to the air conditioner. Furthermore, the control unit can operate multiple electric appliances simultaneously. For example, when a composite voice command such as "Turn on the light in the living room and set the air conditioner to 25 degrees" is received, the control unit generates appropriate operation commands for both the light and the air conditioner and executes them simultaneously. This allows the control unit to quickly and accurately operate the electric appliance based on the user's voice command, thereby improving the convenience of the smart home.

[0032] The learning unit learns the user's daily habits and preferences. The learning unit learns the user's daily habits and preferences, for example, using a machine learning algorithm. Specifically, the learning unit collects the user's voice commands and operation history as data, and identifies the user's behavioral patterns and preferences by analyzing these data. For example, the learning unit learns that the user has a habit of turning on the living room lights at 7:00 every morning. Based on this information, the learning unit can predict the time of day when the user enters the living room and the frequency of use of the lights. The learning unit can also learn the music and TV programs that the user likes. For example, the learning unit learns that the user frequently plays a specific music playlist, and preferentially suggests the playlist. Furthermore, the learning unit can improve the learning accuracy by incorporating the user's feedback. For example, if the user responds positively to a proposed operation, the operation is reflected in future suggestions. Conversely, if the user responds negatively, the operation is excluded from the suggestion list. This allows the learning unit to provide a more personalized service based on the user's habits and preferences.

[0033] The suggestion unit suggests to the user a predicted operation based on the information learned by the learning unit. For example, when the user enters the living room, the suggestion unit suggests "Shall I turn on the light?". Specifically, the suggestion unit predicts an operation that the user is likely to perform next based on the user's behavioral patterns and preference data provided by the learning unit. For example, when the user enters the living room, the suggestion unit determines from past data that there is a high possibility that the user will turn on the light, and suggests "Shall I turn on the light?". The suggestion unit can also suggest playing music that the user likes. For example, when the user enters the living room, the suggestion unit suggests "Shall I play your favorite music?". Furthermore, the suggestion unit can continuously improve the content of the suggestion based on user feedback. For example, if the user responds positively to the suggestion, the suggestion unit continues to make the suggestion in the future. Conversely, if the user responds negatively, the suggestion unit reviews the suggestion and makes a more appropriate suggestion. In this way, the suggestion unit can suggest an optimal operation according to the user's needs, and further improve the convenience of the smart home.

[0034] The smart home voice control system AI assistant includes an alert unit that provides a security alert. The security alert includes, for example, a detection of a door opening and notifying the user, but is not limited to such an example. The alert unit detects the opening and closing of a door using, for example, a door sensor, and notifies the user when an abnormality occurs. The alert unit can also detect an abnormality using a window opening and closing or a motion detection sensor. For example, the alert unit detects that a window is open and notifies the user. This makes it possible to ensure the safety of the user by providing a security alert. A part or all of the above-mentioned processing in the alert unit may be performed using, for example, AI, or may be performed without using AI. For example, the alert unit can input data from a door sensor to AI and cause AI to perform abnormality detection.

[0035] The smart home voice control system AI assistant includes a monitoring unit that monitors energy usage and makes suggestions for power saving. The energy usage includes, for example, but is not limited to, power consumption measurement and data collection. The monitoring unit measures power consumption using, for example, a power meter and collects data in real time. The monitoring unit also includes an interface that visualizes the energy usage and can make suggestions for power saving to the user. For example, the monitoring unit displays a graph of power consumption and suggests specific actions for power saving. This makes it possible to promote efficient use of energy by monitoring the energy usage and making suggestions for power saving. Some or all of the above-mentioned processing in the monitoring unit may be performed using, for example, AI, or may be performed without using AI. For example, the monitoring unit can input power consumption data to AI and have AI execute suggestions for power saving.

[0036] The AI ​​assistant voice control system for smart home includes a linking unit that provides weather information and news to the user in cooperation with the outside. Weather information includes, for example, weather forecast, temperature, and probability of precipitation, but is not limited to such examples. News includes, for example, the latest news and news of a specific category, but is not limited to such examples. The linking unit, for example, acquires weather information and news through the Internet and notifies the user. The linking unit can also provide information customized according to the user's preferences. For example, the linking unit learns news categories in which the user is interested and notifies the user of news related to the category preferentially. This allows the user to be provided with the latest information on weather and news by providing the latest information in cooperation with the outside. Some or all of the above-mentioned processing in the linking unit may be performed, for example, using AI, or may be performed without using AI. For example, the linking unit inputs data acquired from the Internet into the AI ​​and allows the AI ​​to select information to be notified to the user.

[0037] The reception unit may have a noise canceling function. The noise canceling function includes, but is not limited to, active noise canceling and passive noise canceling, for example. The reception unit may use, for example, active noise canceling technology to reduce ambient noise and improve the recognition accuracy of the voice command. The reception unit may also use passive noise canceling technology to physically block noise. For example, the reception unit may use a microphone with a noise canceling function to accurately recognize the voice command. As a result, the noise canceling function can be provided to improve the recognition accuracy of the voice command. A part or all of the above-mentioned processing in the reception unit may be performed, for example, using AI, or may be performed without using AI. For example, the reception unit may cause AI to adjust the noise canceling function.

[0038] The learning unit can learn the daily habits and preferences of the user using a machine learning algorithm. Machine learning algorithms include, but are not limited to, deep learning and support vector machines. The learning unit can learn the daily habits and preferences of the user using, for example, deep learning. The learning unit can also analyze the user's behavioral patterns using a support vector machine. For example, the learning unit can learn the user's habit of turning on the living room lights at 7 o'clock every morning using deep learning. The learning unit can also learn the music and TV programs that the user likes using a support vector machine. This makes it possible to accurately learn the user's daily habits and preferences by using a machine learning algorithm. Some or all of the above-mentioned processing in the learning unit may be performed using, for example, AI, or may be performed without using AI. For example, the learning unit can input the user's behavioral data into AI and have the AI ​​execute the analysis of the behavioral patterns.

[0039] The suggestion unit can predict an operation by the user based on the information learned by the learning unit and the biometric information of the user. The biometric information includes, for example, heart rate, body temperature, blood pressure, etc., but is not limited to such examples. The suggestion unit predicts an operation based on, for example, the information learned by the learning unit and the heart rate of the user. The suggestion unit can also predict an operation based on the body temperature and blood pressure of the user. For example, the suggestion unit suggests a relaxing environment when the user's heart rate is rising. The suggestion unit can also suggest lowering the temperature of the air conditioner when the user's body temperature is high. This makes it possible to make a more appropriate suggestion by predicting an operation based on the learned information and the biometric information. A part or all of the above-mentioned processing in the suggestion unit may be performed using, for example, AI, or may be performed without using AI. For example, the suggestion unit can input the biometric data of the user to AI and cause the AI ​​to execute the prediction of the operation.

[0040] The monitoring unit may include an interface that visualizes the energy usage status in real time. Real time includes, for example, the frequency of data updates and delay times, but is not limited to such examples. The monitoring unit updates data, for example, every second, and displays the energy usage status in real time. The monitoring unit may also display data with minimal delay time. For example, the monitoring unit displays the energy usage status as a graph in real time, making it easier for the user to understand the energy usage status. This makes it easier for the user to understand the energy usage status by visualizing the energy usage status in real time. Some or all of the above-mentioned processing in the monitoring unit may be performed, for example, using AI, or may be performed without using AI. For example, the monitoring unit may input the energy usage data to AI and cause AI to execute visualization in real time.

[0041] The suggestion unit can predict an operation by the user by further using the schedule information of the user. The schedule information includes, for example, data of a calendar app and a type of schedule, but is not limited to such examples. The suggestion unit predicts an operation by using, for example, data of a calendar app of the user. The suggestion unit can also predict an operation based on the type of schedule of the user. For example, if the user has a meeting scheduled, the suggestion unit can suggest adjusting the temperature of the air conditioner before the start of the meeting. Also, if the user has a plan to go out, the suggestion unit can suggest turning off the lights before going out. This makes it possible to predict an operation more accurately by using the schedule information. A part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the suggestion unit can input the schedule data of the user to the AI ​​and cause the AI ​​to execute the prediction of the operation.

[0042] The learning unit can integrate and learn sensor information in the user's residence. The sensor information includes, for example, a temperature sensor and a motion sensor, but is not limited to such examples. The learning unit, for example, uses data from a temperature sensor to learn the environment in the user's residence. The learning unit can also learn the user's movements using data from a motion sensor. For example, the learning unit learns the temperature at which the user feels comfortable based on data from a temperature sensor. The learning unit can also learn the user's movement patterns based on data from a motion sensor. This improves the accuracy of learning by integrating the sensor information. A part or all of the above-mentioned processing in the learning unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the learning unit can input data from a sensor to an AI and cause the AI ​​to learn the environment and movement patterns.

[0043] The suggestion unit can suggest an operation to the user based on the health information of the user. The health information includes, for example, fitness tracker data and medical records, but is not limited to such examples. The suggestion unit can suggest an operation using, for example, fitness tracker data. The suggestion unit can also suggest an operation based on medical records. For example, the suggestion unit can suggest an environment in which the user can relax after exercise based on the fitness tracker data of the user. The suggestion unit can also suggest a health-conscious operation based on the medical records of the user. As a result, by making a suggestion based on the health information, an operation that takes the user's health into consideration is possible. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the suggestion unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the suggestion unit can input the user's health data to the AI ​​and cause the AI ​​to execute the operation suggestion.

[0044] When receiving a voice command, the reception unit can select the optimal reception method by referring to the user's past command history. For example, the reception unit preferentially recognizes commands that the user has frequently used in the past. The reception unit can also learn specific phrases and expressions that the user has used in the past and preferentially accept similar commands. For example, the reception unit predicts commands to be used in a specific time period from the user's past command history and adjusts the reception sensitivity. In this way, the optimal reception method can be selected by referring to the past command history. A part or all of the above-mentioned processing in the reception unit may be performed, for example, using AI or may be performed without using AI. For example, the reception unit can input the user's command history data to AI and cause the AI ​​to select the optimal reception method.

[0045] The reception unit can optimize the effect of noise canceling by analyzing the user's current environmental sound when receiving a voice command. For example, when the user is in a noisy environment, the reception unit strengthens noise canceling to accurately recognize the voice command. In addition, when the user is in a quiet environment, the reception unit can minimize noise canceling to perform natural voice recognition. For example, when the user is moving, the reception unit adjusts the effect of noise canceling in real time according to changes in the environmental sound. As a result, the recognition accuracy of the voice command is improved by analyzing the environmental sound and optimizing the effect of noise canceling. A part or all of the above-mentioned processing in the reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the reception unit can input environmental sound data to AI and cause AI to optimize the effect of noise canceling.

[0046] When receiving a voice command, the reception unit can preferentially receive commands with high relevance in consideration of the geographical location information of the user. For example, when the user is at home, the reception unit preferentially receives commands for operating home appliances and lighting. In addition, when the user is in an office, the reception unit can also preferentially receive work-related commands. For example, when the user is out, the reception unit preferentially receives commands for navigation and weather information. In this way, by taking the geographical location information into consideration, it is possible to preferentially receive commands with high relevance. Part or all of the above-mentioned processing in the reception unit may be performed, for example, using AI, or may be performed without using AI. For example, the reception unit can input the geographical location data of the user to AI and cause the AI ​​to execute the determination of the priority order of commands with high relevance.

[0047] The reception unit can analyze the user's social media activity and receive related commands when receiving a voice command. For example, when the user mentions a specific event on social media, the reception unit preferentially receives commands related to the event. In addition, when the user mentions a specific place on social media, the reception unit can also preferentially receive commands related to the place. For example, when the user mentions a specific product on social media, the reception unit preferentially receives commands related to the product. In this way, by analyzing social media activity, related commands can be preferentially received. A part or all of the above-mentioned processing in the reception unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the reception unit can input the user's social media data into the AI ​​and cause the AI ​​to execute the reception of related commands.

[0048] The control unit can select the optimal operation method by referring to the user's past operation history when operating the electrical device. For example, the control unit preferentially selects an operation method that the user has frequently used in the past. The control unit can also predict an operation that the user will perform in a specific time period and select that operation method. For example, the control unit selects the most efficient operation method from the user's past operation history. In this way, the optimal operation method can be selected by referring to the past operation history. A part or all of the above-mentioned processing in the control unit may be performed using, for example, AI, or may be performed without using AI. For example, the control unit can input the user's operation history data to AI and cause the AI ​​to select the optimal operation method.

[0049] The control unit can adjust the timing of the operation based on the user's current living situation when operating the electrical appliance. For example, when the user is relaxed, the control unit adjusts the timing of the operation slowly. Also, when the user is in a hurry, the control unit can adjust the timing of the operation quickly. For example, when the user is performing a specific activity, the control unit adjusts the timing of the operation to match the activity. This allows for more appropriate operation by adjusting the timing of the operation based on the living situation. Some or all of the above-mentioned processing in the control unit may be performed using, for example, AI, or may be performed without using AI. For example, the control unit can input the user's living situation data to AI and cause AI to adjust the timing of the operation.

[0050] The control unit can operate highly relevant devices with priority in consideration of the geographical location information of the user when operating electrical appliances. For example, when the user is at home, the control unit operates home appliances and lighting with priority. In addition, when the user is in the office, the control unit can also operate work-related devices with priority. For example, when the user is out, the control unit operates security-related devices with priority. In this way, highly relevant devices can be operated with priority by considering the geographical location information. Part or all of the above-mentioned processing in the control unit may be performed using, for example, AI, or may be performed without using AI. For example, the control unit can input the geographical location data of the user to AI and cause the AI ​​to execute the determination of the priority order of highly relevant devices.

[0051] The control unit can analyze the social media activity of the user when operating the electrical appliance, and operate the related appliance. For example, when the user mentions a specific event on social media, the control unit operates appliances related to the event. In addition, when the user mentions a specific place on social media, the control unit can also operate appliances related to the place. For example, when the user mentions a specific product on social media, the control unit operates appliances related to the product. In this way, by analyzing the social media activity, the related appliances can be operated preferentially. A part or all of the above-mentioned processing in the control unit may be performed using, for example, AI, or may be performed without using AI. For example, the control unit can input the user's social media data to AI and cause the AI ​​to execute the operation of the related appliance.

[0052] The learning unit can optimize the learning algorithm by referring to past learning data during learning. For example, the learning unit selects the most effective algorithm from the past learning data and performs learning. The learning unit can also analyze the past learning data and optimize the parameters of the algorithm. For example, the learning unit improves the learning method for new data based on the past learning data. In this way, the learning algorithm can be optimized by referring to the past learning data. A part or all of the above-mentioned processing in the learning unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the learning unit can input the past learning data to an AI and cause the AI ​​to optimize the learning algorithm.

[0053] The learning unit can analyze the user's life pattern during learning to improve the accuracy of learning. The learning unit, for example, analyzes the user's life pattern and selects the optimal learning timing. The learning unit can also weight the learning data based on the user's life pattern. For example, the learning unit adjusts the learning algorithm taking into account the user's life pattern. This improves the accuracy of learning by analyzing the life pattern. A part or all of the above-mentioned processing in the learning unit may be performed using, for example, AI, or may be performed without using AI. For example, the learning unit can input the user's life pattern data to AI and cause the AI ​​to improve the accuracy of learning.

[0054] The learning unit can weight the learning data in consideration of the geographical location information of the user during learning. For example, when the user is at home, the learning unit learns by emphasizing data related to behavior at home. Also, when the user is in the office, the learning unit can also learn by emphasizing work-related data. For example, when the user is out, the learning unit learns by emphasizing data related to behavior outside the office. This makes it possible to weight the learning data by taking the geographical location information into consideration. A part or all of the above-mentioned processing in the learning unit may be performed, for example, using AI, or may be performed without using AI. For example, the learning unit can input the geographical location data of the user to AI and cause the AI ​​to perform weighting of the learning data.

[0055] During learning, the learning unit can analyze the user's social media activities to complement the learning data. For example, the learning unit reflects the interests and concerns mentioned by the user on social media in the learning data. The learning unit can also analyze articles and posts shared by the user on social media to learn related data. For example, the learning unit extracts specific trends and patterns from the user's social media activities to complement the learning data. In this way, the learning data can be complemented by analyzing the social media activities. Some or all of the above-mentioned processing in the learning unit may be performed, for example, using AI or may be performed without using AI. For example, the learning unit can input the user's social media data into AI and cause the AI ​​to complement the learning data.

[0056] At the time of proposal, the suggestion unit can select the optimal proposal method by referring to the user's past operation history. For example, the suggestion unit preferentially selects a proposal method that the user has accepted in the past. The suggestion unit can also select a proposal method suitable for a specific time period from the user's past operation history. For example, the suggestion unit analyzes the user's past operation history and selects the most effective proposal method. In this way, the optimal proposal method can be selected by referring to the past operation history. A part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI, or may be performed without using AI. For example, the suggestion unit can input the user's operation history data to AI and cause the AI ​​to select the optimal proposal method.

[0057] The suggestion unit can adjust the timing of the suggestion based on the user's current living situation at the time of suggestion. For example, when the user is relaxed, the suggestion unit slowly adjusts the timing of the suggestion. Also, when the user is in a hurry, the suggestion unit can quickly adjust the timing of the suggestion. For example, when the user is performing a specific activity, the suggestion unit adjusts the timing of the suggestion to match the activity. This allows for more appropriate suggestions by adjusting the timing of the suggestion based on the living situation. Some or all of the above-mentioned processing in the suggestion unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the suggestion unit can input the user's living situation data into the AI ​​and cause the AI ​​to adjust the timing of the suggestion.

[0058] When making a proposal, the suggestion unit can give priority to a proposal with high relevance by taking into account the geographical location information of the user. For example, when the user is at home, the suggestion unit gives priority to a proposal related to activities at home. Also, when the user is in the office, the suggestion unit can give priority to a proposal related to work. For example, when the user is out, the suggestion unit gives priority to a proposal related to activities at the outing. In this way, by taking into account the geographical location information, a proposal with high relevance can be given priority. Part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI, or may be performed without using AI. For example, the suggestion unit can input the geographical location data of the user to AI and cause AI to execute the determination of the priority order of the proposal with high relevance.

[0059] At the time of making a proposal, the suggestion unit can analyze the user's social media activity and make a related proposal. For example, when the user mentions a specific event on social media, the suggestion unit makes a proposal related to the event. In addition, when the user mentions a specific place on social media, the suggestion unit can also make a proposal related to the place. For example, when the user mentions a specific product on social media, the suggestion unit makes a proposal related to the product. In this way, by analyzing social media activity, related proposals can be made preferentially. A part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI, or may be performed without using AI. For example, the suggestion unit can input the user's social media data into AI and cause the AI ​​to execute the related proposal.

[0060] At the time of an alert, the alert unit can select the optimal notification method by referring to the user's past alert history. For example, the alert unit preferentially selects a notification method that the user has accepted in the past. The alert unit can also select a notification method suitable for a specific time period from the user's past alert history. For example, the alert unit analyzes the user's past alert history and selects the most effective notification method. In this way, the optimal notification method can be selected by referring to the past alert history. A part or all of the above-mentioned processing in the alert unit may be performed, for example, using AI, or may be performed without using AI. For example, the alert unit can input the user's alert history data into AI and have the AI ​​select the optimal notification method.

[0061] The alert unit can give priority to highly relevant alerts at the time of an alert, taking into account the geographical location information of the user. For example, when the user is at home, the alert unit gives priority to security-related alerts at home. Also, when the user is in the office, the alert unit can give priority to work-related alerts. For example, when the user is out, the alert unit gives priority to security-related alerts at the out-of-office location. In this way, by taking into account the geographical location information, it is possible to give priority to highly relevant alerts. Part or all of the above-mentioned processing in the alert unit may be performed, for example, using AI, or may be performed without using AI. For example, the alert unit can input the geographical location data of the user to AI and cause AI to execute the determination of the priority order of highly relevant alerts.

[0062] The monitoring unit can select the optimal display method by referring to the user's past usage history when monitoring the energy usage status. For example, the monitoring unit preferentially selects a display method that the user has used in the past. The monitoring unit can also select a display method suitable for a specific time period from the user's past usage history. For example, the monitoring unit analyzes the user's past usage history and selects the most effective display method. In this way, the optimal display method can be selected by referring to the past usage history. A part or all of the above-mentioned processing in the monitoring unit may be performed using, for example, AI, or may be performed without using AI. For example, the monitoring unit can input the user's usage history data to AI and cause the AI ​​to select the optimal display method.

[0063] The monitoring unit can display highly relevant information preferentially in consideration of the geographical location information of the user when monitoring the energy usage status. For example, when the user is at home, the monitoring unit preferentially displays the energy usage status at home. Also, when the user is in an office, the monitoring unit can preferentially display the energy usage status at the office. For example, when the user is out, the monitoring unit preferentially displays the energy usage status at the outside. In this way, highly relevant information can be displayed preferentially by considering the geographical location information. Part or all of the above-mentioned processing in the monitoring unit may be performed, for example, using AI, or may be performed without using AI. For example, the monitoring unit can input the geographical location data of the user to AI and cause AI to execute the display of highly relevant information.

[0064] The linking unit can select the optimal notification method by referring to the user's past notification history when notifying the user of weather or news. For example, the linking unit preferentially selects a notification method that the user has used in the past. The linking unit can also select a notification method suitable for a specific time period from the user's past notification history. For example, the linking unit analyzes the user's past notification history and selects the most effective notification method. In this way, the optimal notification method can be selected by referring to the past notification history. A part or all of the above-mentioned processing in the linking unit may be performed, for example, using AI, or may be performed without using AI. For example, the linking unit can input the user's notification history data into AI and have the AI ​​select the optimal notification method.

[0065] When notifying the user of the weather or news, the linking unit can give priority to notifying the user of highly relevant information by taking into account the user's geographical location information. For example, when the user is at home, the linking unit gives priority to notifying the user of the weather and news around the user's home. Also, when the user is in the office, the linking unit can give priority to notifying the user of the weather and news around the office. For example, when the user is out, the linking unit gives priority to notifying the user of the weather and news at the user's destination. In this way, by taking into account the geographical location information, it is possible to give priority to notifying the user of highly relevant information. Part or all of the above-mentioned processing in the linking unit may be performed, for example, using AI, or may be performed without using AI. For example, the linking unit can input the user's geographical location data into AI and cause AI to execute notification of highly relevant information.

[0066] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0067] The smart home voice control system AI assistant can analyze the user's behavioral patterns and automatically perform operations based on specific behaviors. For example, the learning unit can learn the user's habit of turning on the living room lights at 7:00 every morning, and automatically turn on the lights before the user enters the living room. The learning unit can also learn the user's habit of turning off the air conditioner at 10:00 every night, and automatically turn off the air conditioner before the user enters the bedroom. Furthermore, the learning unit can learn the user's habit of playing specific music at a specific time, and automatically play music when that time comes. This can improve the user's convenience by automatically performing operations based on the user's behavioral patterns.

[0068] A smart home voice control system AI assistant can use the user's schedule information to automatically perform operations at specific times. For example, the suggestion unit can use data from the user's calendar app to adjust the temperature of the air conditioner before a meeting starts. In addition, if the user plans to go out, the suggestion unit can also suggest turning off the lights before going out. In addition, if the user plans to exercise, the suggestion unit can suggest an environment in which the user can relax after exercising. This makes it possible to predict operations more accurately by using schedule information.

[0069] The smart home voice control system AI assistant can use the geographic location information of the user to suggest operations at a specific location. For example, when the user is at home, the suggestion unit suggests operations at home. Also, when the user is in the office, the suggestion unit can suggest operations at the office. Furthermore, when the user is out, the suggestion unit can suggest operations outside the home. In this way, by using the geographic location information, more appropriate operations can be suggested.

[0070] The smart home voice control system AI assistant can analyze the user's social media activity and suggest related operations. For example, if the user mentions a specific event on social media, the suggestion unit can suggest operations related to the event. Also, if the user mentions a specific place on social media, the suggestion unit can suggest operations related to the place. Furthermore, if the user mentions a specific product on social media, the suggestion unit can suggest operations related to the product. In this way, by analyzing social media activity, related operations can be suggested.

[0071] The smart home voice control system AI assistant can suggest the optimal operation method by referring to the user's past operation history. For example, the suggestion unit preferentially suggests an operation method that the user has frequently used in the past. The suggestion unit can also suggest an operation method suitable for a specific time period from the user's past operation history. Furthermore, the suggestion unit can analyze the user's past operation history and suggest the most efficient operation method. As a result, the optimal operation method can be suggested by referring to the past operation history.

[0072] The process flow of the first embodiment will be briefly described below.

[0073] Step 1: The reception unit receives a voice command from a user. For example, the reception unit receives the voice command using a microphone equipped with a noise canceling function, and converts the voice command into text data using a voice recognition technology. Step 2: The control unit operates the electrical appliance based on the voice command received by the reception unit, for example, turning on the lights in the living room or setting the temperature of the air conditioner based on the voice command. Step 3: The learning unit learns the user's daily habits and preferences. For example, a machine learning algorithm is used to learn that the user has a habit of turning on the living room lights at 7am every morning, and their preferred music playlist. Step 4: The suggestion unit suggests predicted actions to the user based on the information learned by the learning unit. For example, when the user enters the living room, the suggestion unit suggests "Should I turn on the lights?" or "Should I play my favorite music?"

[0074] (Example 2) The voice control system AI assistant for smart home according to the embodiment of the present invention is a system that accepts voice commands and supports the operation of home appliances, lighting, etc. This system learns the user's daily habits and preferences and suggests predictive operations. In addition, it provides information such as security alerts and energy usage status, and notifies the latest information such as weather and news by linking with the outside. For example, it accepts a voice command from the user. For example, it accepts a voice command such as "Turn on the living room light." This voice command is accepted by a reception unit equipped with a noise canceling function. Next, it operates an electric device based on the accepted voice command. For example, an operation to turn on the living room light is performed. This operation is executed by a control unit. In addition, it learns the user's daily habits and preferences. For example, if a user has a habit of turning on the living room light at 7 o'clock every morning, the learning unit learns that information using a machine learning algorithm. Based on the learned information, it suggests predictive operations. For example, when a user enters the living room, the suggestion unit suggests "Should I turn on the light?" This suggestion is made based on the learned information and the user's biometric information. It also provides a security alert. For example, when it detects that a door has been opened, the alert unit notifies the user. The energy usage status is monitored and power saving suggestions are made. For example, a monitoring unit equipped with an interface that visualizes the energy usage status in real time makes power saving suggestions to the user. The latest weather and news information is provided by linking with the outside. For example, the linking unit notifies the user of weather forecasts and news. In this way, the present invention is a voice control system AI assistant for a smart home that accepts voice commands, supports the operation of home appliances and lighting, etc., learns the user's daily habits and preferences to suggest predictive operations, provides information such as security alerts and energy usage status, and notifies the user of the latest information such as weather and news by linking with the outside. As a result, the voice control system AI assistant for a smart home can accept the user's voice commands, support the operation of home appliances and lighting, learns the user's daily habits and preferences, and suggests predictive operations.It can also provide information such as security alerts and energy usage, and by connecting with external parties it can provide the latest information on weather, news, etc.

[0075] The voice control system AI assistant for a smart home according to the embodiment includes a reception unit, a control unit, a learning unit, and a suggestion unit. The reception unit receives a voice command from a user. The user's voice command includes, for example, "Turn on the living room light," but is not limited to such an example. The reception unit receives the voice command using, for example, a microphone with a noise canceling function. The reception unit can also analyze the voice command using a voice recognition technology and convert it into text data. For example, the reception unit receives the voice command using a microphone with a noise canceling function, and converts the voice command into text data using a voice recognition technology. The control unit operates an electric device based on the voice command received by the reception unit. The electric device includes, for example, a light, an air conditioner, a television, and the like, but is not limited to such an example. The control unit performs an operation to turn on the living room light based on the voice command, for example. The control unit can also adjust the temperature of the air conditioner based on the voice command. For example, the control unit sets the temperature of the air conditioner based on the voice command. The learning unit learns the user's daily habits and preferences. The learning unit learns the user's daily habits and preferences, for example, using a machine learning algorithm. For example, the learning unit learns the user's habit of turning on the lights in the living room at 7:00 every morning. The learning unit can also learn the user's favorite music and TV programs. For example, the learning unit learns the user's favorite music playlist. The suggestion unit suggests to the user a predicted operation based on the information learned by the learning unit. For example, the suggestion unit suggests "Would you like to turn on the lights?" when the user enters the living room. The suggestion unit can also suggest playing music that the user likes. For example, the suggestion unit suggests "Would you like to play your favorite music?" when the user enters the living room. As a result, the AI ​​assistant of the smart home voice control system according to the embodiment can accept the user's voice commands, operate electrical appliances, learn the user's daily habits and preferences, and suggest predictive operations.

[0076] The reception unit receives a voice command from the user. The user's voice command includes, but is not limited to, for example, "Turn on the living room lights." The reception unit receives the voice command using, for example, a microphone with a noise canceling function. The noise canceling function can reduce ambient noise and capture the user's voice clearly. This makes it possible to accurately receive voice commands even in noisy environments. The reception unit can also analyze the voice command using voice recognition technology and convert it into text data. A model using deep learning is used for the voice recognition technology, and the voice data is analyzed with high accuracy. For example, the reception unit receives a voice command using a microphone with a noise canceling function and converts the voice command into text data using voice recognition technology. The voice command converted into text data is transmitted to the control unit, which is the next processing step. Furthermore, the reception unit can support multiple languages ​​and dialects, and can meet the diverse needs of users. For example, by supporting regional dialects in addition to major languages ​​such as English, Japanese, and Spanish, it is expected that the device will be used by a wider range of users. This enables the reception unit to receive the user's voice commands accurately and quickly, improving the operability of the entire system.

[0077] The control unit operates the electric appliance based on the voice command received by the reception unit. The electric appliance includes, but is not limited to, for example, a light, an air conditioner, and a television. For example, the control unit performs an operation of turning on the light in the living room based on the voice command. Specifically, the control unit analyzes the voice command and generates an operation command for the corresponding electric appliance. For example, when a voice command such as "Turn on the light in the living room" is transmitted from the reception unit, the control unit generates an instruction to turn on the light and transmits it to the lighting system. The control unit can also adjust the temperature of the air conditioner based on the voice command. For example, when the control unit receives a voice command such as "Set the temperature of the air conditioner to 25 degrees," the control unit generates an instruction to change the set temperature of the air conditioner to 25 degrees and transmits it to the air conditioner. Furthermore, the control unit can operate multiple electric appliances simultaneously. For example, when a composite voice command such as "Turn on the light in the living room and set the air conditioner to 25 degrees" is received, the control unit generates appropriate operation commands for both the light and the air conditioner and executes them simultaneously. This allows the control unit to quickly and accurately operate the electric appliance based on the user's voice command, thereby improving the convenience of the smart home.

[0078] The learning unit learns the user's daily habits and preferences. The learning unit learns the user's daily habits and preferences, for example, using a machine learning algorithm. Specifically, the learning unit collects the user's voice commands and operation history as data, and identifies the user's behavioral patterns and preferences by analyzing these data. For example, the learning unit learns that the user has a habit of turning on the living room lights at 7:00 every morning. Based on this information, the learning unit can predict the time of day when the user enters the living room and the frequency of use of the lights. The learning unit can also learn the music and TV programs that the user likes. For example, the learning unit learns that the user frequently plays a specific music playlist, and preferentially suggests the playlist. Furthermore, the learning unit can improve the learning accuracy by incorporating the user's feedback. For example, if the user responds positively to a proposed operation, the operation is reflected in future suggestions. Conversely, if the user responds negatively, the operation is excluded from the suggestion list. This allows the learning unit to provide a more personalized service based on the user's habits and preferences.

[0079] The suggestion unit suggests to the user a predicted operation based on the information learned by the learning unit. For example, when the user enters the living room, the suggestion unit suggests "Shall I turn on the light?". Specifically, the suggestion unit predicts an operation that the user is likely to perform next based on the user's behavioral patterns and preference data provided by the learning unit. For example, when the user enters the living room, the suggestion unit determines from past data that there is a high possibility that the user will turn on the light, and suggests "Shall I turn on the light?". The suggestion unit can also suggest playing music that the user likes. For example, when the user enters the living room, the suggestion unit suggests "Shall I play your favorite music?". Furthermore, the suggestion unit can continuously improve the content of the suggestion based on user feedback. For example, if the user responds positively to the suggestion, the suggestion unit continues to make the suggestion in the future. Conversely, if the user responds negatively, the suggestion unit reviews the suggestion and makes a more appropriate suggestion. In this way, the suggestion unit can suggest an optimal operation according to the user's needs, and further improve the convenience of the smart home.

[0080] The smart home voice control system AI assistant includes an alert unit that provides a security alert. The security alert includes, for example, a detection of a door opening and notifying the user, but is not limited to such an example. The alert unit detects the opening and closing of a door using, for example, a door sensor, and notifies the user when an abnormality occurs. The alert unit can also detect an abnormality using a window opening and closing or a motion detection sensor. For example, the alert unit detects that a window is open and notifies the user. This makes it possible to ensure the safety of the user by providing a security alert. A part or all of the above-mentioned processing in the alert unit may be performed using, for example, AI, or may be performed without using AI. For example, the alert unit can input data from a door sensor to AI and cause AI to perform abnormality detection.

[0081] The smart home voice control system AI assistant includes a monitoring unit that monitors energy usage and makes suggestions for power saving. The energy usage includes, for example, but is not limited to, power consumption measurement and data collection. The monitoring unit measures power consumption using, for example, a power meter and collects data in real time. The monitoring unit also includes an interface that visualizes the energy usage and can make suggestions for power saving to the user. For example, the monitoring unit displays a graph of power consumption and suggests specific actions for power saving. This makes it possible to promote efficient use of energy by monitoring the energy usage and making suggestions for power saving. Some or all of the above-mentioned processing in the monitoring unit may be performed using, for example, AI, or may be performed without using AI. For example, the monitoring unit can input power consumption data to AI and have AI execute suggestions for power saving.

[0082] The AI ​​assistant voice control system for smart home includes a linking unit that provides weather information and news to the user in cooperation with the outside. Weather information includes, for example, weather forecast, temperature, and probability of precipitation, but is not limited to such examples. News includes, for example, the latest news and news of a specific category, but is not limited to such examples. The linking unit, for example, acquires weather information and news through the Internet and notifies the user. The linking unit can also provide information customized according to the user's preferences. For example, the linking unit learns news categories in which the user is interested and notifies the user of news related to the category preferentially. This allows the user to be provided with the latest information on weather and news by providing the latest information in cooperation with the outside. Some or all of the above-mentioned processing in the linking unit may be performed, for example, using AI, or may be performed without using AI. For example, the linking unit inputs data acquired from the Internet into the AI ​​and allows the AI ​​to select information to be notified to the user.

[0083] The reception unit may have a noise canceling function. The noise canceling function includes, but is not limited to, active noise canceling and passive noise canceling, for example. The reception unit may use, for example, active noise canceling technology to reduce ambient noise and improve the recognition accuracy of the voice command. The reception unit may also use passive noise canceling technology to physically block noise. For example, the reception unit may use a microphone with a noise canceling function to accurately recognize the voice command. As a result, the noise canceling function can be provided to improve the recognition accuracy of the voice command. A part or all of the above-mentioned processing in the reception unit may be performed, for example, using AI, or may be performed without using AI. For example, the reception unit may cause AI to adjust the noise canceling function.

[0084] The learning unit can learn the daily habits and preferences of the user using a machine learning algorithm. Machine learning algorithms include, but are not limited to, deep learning and support vector machines. The learning unit can learn the daily habits and preferences of the user using, for example, deep learning. The learning unit can also analyze the user's behavioral patterns using a support vector machine. For example, the learning unit can learn the user's habit of turning on the living room lights at 7 o'clock every morning using deep learning. The learning unit can also learn the music and TV programs that the user likes using a support vector machine. This makes it possible to accurately learn the user's daily habits and preferences by using a machine learning algorithm. Some or all of the above-mentioned processing in the learning unit may be performed using, for example, AI, or may be performed without using AI. For example, the learning unit can input the user's behavioral data into AI and have the AI ​​execute the analysis of the behavioral patterns.

[0085] The suggestion unit can predict an operation by the user based on the information learned by the learning unit and the biometric information of the user. The biometric information includes, for example, heart rate, body temperature, blood pressure, etc., but is not limited to such examples. The suggestion unit predicts an operation based on, for example, the information learned by the learning unit and the heart rate of the user. The suggestion unit can also predict an operation based on the body temperature and blood pressure of the user. For example, the suggestion unit suggests a relaxing environment when the user's heart rate is rising. The suggestion unit can also suggest lowering the temperature of the air conditioner when the user's body temperature is high. This makes it possible to make a more appropriate suggestion by predicting an operation based on the learned information and the biometric information. A part or all of the above-mentioned processing in the suggestion unit may be performed using, for example, AI, or may be performed without using AI. For example, the suggestion unit can input the biometric data of the user to AI and cause the AI ​​to execute the prediction of the operation.

[0086] The monitoring unit may include an interface that visualizes the energy usage status in real time. Real time includes, for example, the frequency of data updates and delay times, but is not limited to such examples. The monitoring unit updates data, for example, every second, and displays the energy usage status in real time. The monitoring unit may also display data with minimal delay time. For example, the monitoring unit displays the energy usage status as a graph in real time, making it easier for the user to understand the energy usage status. This makes it easier for the user to understand the energy usage status by visualizing the energy usage status in real time. Some or all of the above-mentioned processing in the monitoring unit may be performed, for example, using AI, or may be performed without using AI. For example, the monitoring unit may input the energy usage data to AI and cause AI to execute visualization in real time.

[0087] The suggestion unit can predict an operation by the user by further using the schedule information of the user. The schedule information includes, for example, data of a calendar app and a type of schedule, but is not limited to such examples. The suggestion unit predicts an operation by using, for example, data of a calendar app of the user. The suggestion unit can also predict an operation based on the type of schedule of the user. For example, if the user has a meeting scheduled, the suggestion unit can suggest adjusting the temperature of the air conditioner before the start of the meeting. Also, if the user has a plan to go out, the suggestion unit can suggest turning off the lights before going out. This makes it possible to predict an operation more accurately by using the schedule information. A part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the suggestion unit can input the schedule data of the user to the AI ​​and cause the AI ​​to execute the prediction of the operation.

[0088] The learning unit can integrate and learn sensor information in the user's residence. The sensor information includes, for example, a temperature sensor and a motion sensor, but is not limited to such examples. The learning unit, for example, uses data from a temperature sensor to learn the environment in the user's residence. The learning unit can also learn the user's movements using data from a motion sensor. For example, the learning unit learns the temperature at which the user feels comfortable based on data from a temperature sensor. The learning unit can also learn the user's movement patterns based on data from a motion sensor. This improves the accuracy of learning by integrating the sensor information. A part or all of the above-mentioned processing in the learning unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the learning unit can input data from a sensor to an AI and cause the AI ​​to learn the environment and movement patterns.

[0089] The suggestion unit can suggest an operation to the user based on the health information of the user. The health information includes, for example, fitness tracker data and medical records, but is not limited to such examples. The suggestion unit can suggest an operation using, for example, fitness tracker data. The suggestion unit can also suggest an operation based on medical records. For example, the suggestion unit can suggest an environment in which the user can relax after exercise based on the fitness tracker data of the user. The suggestion unit can also suggest a health-conscious operation based on the medical records of the user. As a result, by making a suggestion based on the health information, an operation that takes the user's health into consideration is possible. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the suggestion unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the suggestion unit can input the user's health data to the AI ​​and cause the AI ​​to execute the operation suggestion.

[0090] The reception unit can estimate the user's emotion and adjust the reception sensitivity of the voice command based on the estimated user's emotion. For example, when the user is stressed, the reception unit increases the reception sensitivity of the voice command so that even a small voice volume can be recognized. In addition, when the user is relaxed, the reception unit can also accept the voice command at normal sensitivity. For example, when the user is in a hurry, the reception unit maximizes the reception sensitivity of the voice command so that the user can respond immediately. This improves the recognition accuracy of the voice command by adjusting the reception sensitivity based on the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the reception unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the reception unit can input the user's emotion data to the AI ​​and cause the AI ​​to adjust the reception sensitivity.

[0091] When receiving a voice command, the reception unit can select the optimal reception method by referring to the user's past command history. For example, the reception unit preferentially recognizes commands that the user has frequently used in the past. The reception unit can also learn specific phrases and expressions that the user has used in the past and preferentially accept similar commands. For example, the reception unit predicts commands to be used in a specific time period from the user's past command history and adjusts the reception sensitivity. In this way, the optimal reception method can be selected by referring to the past command history. A part or all of the above-mentioned processing in the reception unit may be performed, for example, using AI or may be performed without using AI. For example, the reception unit can input the user's command history data to AI and cause the AI ​​to select the optimal reception method.

[0092] The reception unit can optimize the effect of noise canceling by analyzing the user's current environmental sound when receiving a voice command. For example, when the user is in a noisy environment, the reception unit strengthens noise canceling to accurately recognize the voice command. In addition, when the user is in a quiet environment, the reception unit can minimize noise canceling to perform natural voice recognition. For example, when the user is moving, the reception unit adjusts the effect of noise canceling in real time according to changes in the environmental sound. As a result, the recognition accuracy of the voice command is improved by analyzing the environmental sound and optimizing the effect of noise canceling. A part or all of the above-mentioned processing in the reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the reception unit can input environmental sound data to AI and cause AI to optimize the effect of noise canceling.

[0093] The reception unit can estimate the user's emotion and determine the priority of voice commands to be received based on the estimated user's emotion. For example, when the user is stressed, the reception unit preferentially receives commands with high urgency. Also, when the user is relaxed, the reception unit can receive voice commands with normal priority. For example, when the user is in a hurry, the reception unit adjusts the priority so that all commands are immediately received. In this way, important commands can be preferentially received by determining the priority of voice commands based on the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the reception unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the reception unit can input the user's emotion data to the AI ​​and cause the AI ​​to execute the determination of the priority of voice commands.

[0094] When receiving a voice command, the reception unit can preferentially receive commands with high relevance in consideration of the geographical location information of the user. For example, when the user is at home, the reception unit preferentially receives commands for operating home appliances and lighting. In addition, when the user is in an office, the reception unit can also preferentially receive work-related commands. For example, when the user is out, the reception unit preferentially receives commands for navigation and weather information. In this way, by taking the geographical location information into consideration, it is possible to preferentially receive commands with high relevance. Part or all of the above-mentioned processing in the reception unit may be performed, for example, using AI, or may be performed without using AI. For example, the reception unit can input the geographical location data of the user to AI and cause the AI ​​to execute the determination of the priority order of commands with high relevance.

[0095] The reception unit can analyze the user's social media activity and receive related commands when receiving a voice command. For example, when the user mentions a specific event on social media, the reception unit preferentially receives commands related to the event. In addition, when the user mentions a specific place on social media, the reception unit can also preferentially receive commands related to the place. For example, when the user mentions a specific product on social media, the reception unit preferentially receives commands related to the product. In this way, by analyzing social media activity, related commands can be preferentially received. A part or all of the above-mentioned processing in the reception unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the reception unit can input the user's social media data into the AI ​​and cause the AI ​​to execute the reception of related commands.

[0096] The control unit can estimate the user's emotion and adjust the operation method of the electrical device based on the estimated user's emotion. For example, when the user is feeling stressed, the control unit adjusts the lighting to soft light to create a relaxing environment. In addition, when the user is relaxed, the control unit can operate the electrical device in a normal operation method. For example, when the user is in a hurry, the control unit selects the shortest operation procedure to quickly complete the operation. This allows for more appropriate operation by adjusting the operation method based on the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the control unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the control unit can input the user's emotion data to the AI ​​and cause the AI ​​to adjust the operation method.

[0097] The control unit can select the optimal operation method by referring to the user's past operation history when operating the electrical device. For example, the control unit preferentially selects an operation method that the user has frequently used in the past. The control unit can also predict an operation that the user will perform in a specific time period and select that operation method. For example, the control unit selects the most efficient operation method from the user's past operation history. In this way, the optimal operation method can be selected by referring to the past operation history. A part or all of the above-mentioned processing in the control unit may be performed using, for example, AI, or may be performed without using AI. For example, the control unit can input the user's operation history data to AI and cause the AI ​​to select the optimal operation method.

[0098] The control unit can adjust the timing of the operation based on the user's current living situation when operating the electrical appliance. For example, when the user is relaxed, the control unit adjusts the timing of the operation slowly. Also, when the user is in a hurry, the control unit can adjust the timing of the operation quickly. For example, when the user is performing a specific activity, the control unit adjusts the timing of the operation to match the activity. This allows for more appropriate operation by adjusting the timing of the operation based on the living situation. Some or all of the above-mentioned processing in the control unit may be performed using, for example, AI, or may be performed without using AI. For example, the control unit can input the user's living situation data to AI and cause AI to adjust the timing of the operation.

[0099] The control unit can estimate the user's emotion and determine the priority of the electrical appliances to be operated based on the estimated user's emotion. For example, when the user is stressed, the control unit prioritizes the operation of lighting and music to create a relaxing environment. In addition, when the user is relaxed, the control unit can also operate the electrical appliances with normal priority. For example, when the user is in a hurry, the control unit prioritizes the operation of electrical appliances that need to be operated quickly. In this way, by determining the priority of the electrical appliances based on the user's emotion, it is possible to operate important appliances with priority. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the control unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the control unit can input the user's emotion data to the AI ​​and cause the AI ​​to execute the determination of the priority of the electrical appliances.

[0100] The control unit can operate highly relevant devices with priority in consideration of the geographical location information of the user when operating electrical appliances. For example, when the user is at home, the control unit operates home appliances and lighting with priority. In addition, when the user is in the office, the control unit can also operate work-related devices with priority. For example, when the user is out, the control unit operates security-related devices with priority. In this way, highly relevant devices can be operated with priority by considering the geographical location information. Part or all of the above-mentioned processing in the control unit may be performed using, for example, AI, or may be performed without using AI. For example, the control unit can input the geographical location data of the user to AI and cause the AI ​​to execute the determination of the priority order of highly relevant devices.

[0101] The control unit can analyze the social media activity of the user when operating the electrical appliance, and operate the related appliance. For example, when the user mentions a specific event on social media, the control unit operates appliances related to the event. In addition, when the user mentions a specific place on social media, the control unit can also operate appliances related to the place. For example, when the user mentions a specific product on social media, the control unit operates appliances related to the product. In this way, by analyzing the social media activity, the related appliances can be operated preferentially. A part or all of the above-mentioned processing in the control unit may be performed using, for example, AI, or may be performed without using AI. For example, the control unit can input the user's social media data to AI and cause the AI ​​to execute the operation of the related appliance.

[0102] The learning unit can estimate the user's emotion and select learning data based on the estimated user's emotion. For example, when the user is stressed, the learning unit preferentially learns data related to an environment that allows the user to relax. In addition, when the user is relaxed, the learning unit can also use normal learning data. For example, when the user is in a hurry, the learning unit preferentially learns data that allows the user to respond quickly. This allows more appropriate learning by selecting learning data based on the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the learning unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the learning unit can input the user's emotion data to the AI ​​and cause the AI ​​to select the learning data.

[0103] The learning unit can optimize the learning algorithm by referring to past learning data during learning. For example, the learning unit selects the most effective algorithm from the past learning data and performs learning. The learning unit can also analyze the past learning data and optimize the parameters of the algorithm. For example, the learning unit improves the learning method for new data based on the past learning data. In this way, the learning algorithm can be optimized by referring to the past learning data. A part or all of the above-mentioned processing in the learning unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the learning unit can input the past learning data to an AI and cause the AI ​​to optimize the learning algorithm.

[0104] The learning unit can analyze the user's life pattern during learning to improve the accuracy of learning. The learning unit, for example, analyzes the user's life pattern and selects the optimal learning timing. The learning unit can also weight the learning data based on the user's life pattern. For example, the learning unit adjusts the learning algorithm taking into account the user's life pattern. This improves the accuracy of learning by analyzing the life pattern. A part or all of the above-mentioned processing in the learning unit may be performed using, for example, AI, or may be performed without using AI. For example, the learning unit can input the user's life pattern data to AI and cause the AI ​​to improve the accuracy of learning.

[0105] The learning unit can estimate the user's emotions and adjust the frequency of learning based on the estimated user's emotions. For example, when the user is stressed, the learning unit reduces the frequency of learning and prioritizes a relaxing environment. In addition, when the user is relaxed, the learning unit can also perform learning at a normal frequency. For example, when the user is in a hurry, the learning unit increases the frequency of learning so that the user can respond quickly. This allows more appropriate learning by adjusting the frequency of learning based on the user's emotions. The estimation of emotions is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the learning unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the learning unit can input the user's emotion data to the AI ​​and cause the AI ​​to adjust the frequency of learning.

[0106] The learning unit can weight the learning data in consideration of the geographical location information of the user during learning. For example, when the user is at home, the learning unit learns by emphasizing data related to behavior at home. Also, when the user is in the office, the learning unit can also learn by emphasizing work-related data. For example, when the user is out, the learning unit learns by emphasizing data related to behavior outside the office. This makes it possible to weight the learning data by taking the geographical location information into consideration. A part or all of the above-mentioned processing in the learning unit may be performed, for example, using AI, or may be performed without using AI. For example, the learning unit can input the geographical location data of the user to AI and cause the AI ​​to perform weighting of the learning data.

[0107] During learning, the learning unit can analyze the user's social media activities to complement the learning data. For example, the learning unit reflects the interests and concerns mentioned by the user on social media in the learning data. The learning unit can also analyze articles and posts shared by the user on social media to learn related data. For example, the learning unit extracts specific trends and patterns from the user's social media activities to complement the learning data. In this way, the learning data can be complemented by analyzing the social media activities. Some or all of the above-mentioned processing in the learning unit may be performed, for example, using AI or may be performed without using AI. For example, the learning unit can input the user's social media data into AI and cause the AI ​​to complement the learning data.

[0108] The suggestion unit can estimate the user's emotion and adjust the way of expressing the suggestion based on the estimated user's emotion. For example, when the user is stressed, the suggestion unit makes a suggestion in a simple and calm way of expression. Also, when the user is relaxed, the suggestion unit can make a suggestion including detailed information. For example, when the user is in a hurry, the suggestion unit makes a quick and concise suggestion. This enables a more appropriate suggestion by adjusting the way of expressing the suggestion based on the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the suggestion unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the suggestion unit can input the user's emotion data to the AI ​​and cause the AI ​​to adjust the way of expressing the suggestion.

[0109] At the time of proposal, the suggestion unit can select the optimal proposal method by referring to the user's past operation history. For example, the suggestion unit preferentially selects a proposal method that the user has accepted in the past. The suggestion unit can also select a proposal method suitable for a specific time period from the user's past operation history. For example, the suggestion unit analyzes the user's past operation history and selects the most effective proposal method. In this way, the optimal proposal method can be selected by referring to the past operation history. A part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI, or may be performed without using AI. For example, the suggestion unit can input the user's operation history data to AI and cause the AI ​​to select the optimal proposal method.

[0110] The suggestion unit can adjust the timing of the suggestion based on the user's current living situation at the time of suggestion. For example, when the user is relaxed, the suggestion unit slowly adjusts the timing of the suggestion. Also, when the user is in a hurry, the suggestion unit can quickly adjust the timing of the suggestion. For example, when the user is performing a specific activity, the suggestion unit adjusts the timing of the suggestion to match the activity. This allows for more appropriate suggestions by adjusting the timing of the suggestion based on the living situation. Some or all of the above-mentioned processing in the suggestion unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the suggestion unit can input the user's living situation data into the AI ​​and cause the AI ​​to adjust the timing of the suggestion.

[0111] The suggestion unit can estimate the user's emotion and determine the priority of suggestions based on the estimated user's emotion. For example, when the user is stressed, the suggestion unit prioritizes suggestions that will relax the user. Also, when the user is relaxed, the suggestion unit can make suggestions with normal priority. For example, when the user is in a hurry, the suggestion unit prioritizes suggestions that can be quickly responded to. In this way, important suggestions can be prioritized by determining the priority of suggestions based on the user's emotion. The estimation of emotions is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the suggestion unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the suggestion unit can input the user's emotion data to the AI ​​and cause the AI ​​to execute the determination of the priority of suggestions.

[0112] When making a proposal, the suggestion unit can give priority to a proposal with high relevance by taking into account the geographical location information of the user. For example, when the user is at home, the suggestion unit gives priority to a proposal related to activities at home. Also, when the user is in the office, the suggestion unit can give priority to a proposal related to work. For example, when the user is out, the suggestion unit gives priority to a proposal related to activities at the outing. In this way, by taking into account the geographical location information, a proposal with high relevance can be given priority. Part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI, or may be performed without using AI. For example, the suggestion unit can input the geographical location data of the user to AI and cause AI to execute the determination of the priority order of the proposal with high relevance.

[0113] At the time of making a proposal, the suggestion unit can analyze the user's social media activity and make a related proposal. For example, when the user mentions a specific event on social media, the suggestion unit makes a proposal related to the event. In addition, when the user mentions a specific place on social media, the suggestion unit can also make a proposal related to the place. For example, when the user mentions a specific product on social media, the suggestion unit makes a proposal related to the product. In this way, by analyzing social media activity, related proposals can be made preferentially. A part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI, or may be performed without using AI. For example, the suggestion unit can input the user's social media data into AI and cause the AI ​​to execute the related proposal.

[0114] The alert unit can estimate the user's emotion and adjust the notification method of the alert based on the estimated user's emotion. For example, when the user is feeling stressed, the alert unit notifies the user of the alert with a quiet sound or vibration. Also, when the user is relaxed, the alert unit can notify the user of the alert with a normal sound. For example, when the user is in a hurry, the alert unit notifies the user of the alert with a conspicuous sound or light. This allows for more appropriate notification by adjusting the notification method of the alert based on the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the alert unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the alert unit can input the user's emotion data to the AI ​​and cause the AI ​​to adjust the notification method of the alert.

[0115] At the time of an alert, the alert unit can select the optimal notification method by referring to the user's past alert history. For example, the alert unit preferentially selects a notification method that the user has accepted in the past. The alert unit can also select a notification method suitable for a specific time period from the user's past alert history. For example, the alert unit analyzes the user's past alert history and selects the most effective notification method. In this way, the optimal notification method can be selected by referring to the past alert history. A part or all of the above-mentioned processing in the alert unit may be performed, for example, using AI, or may be performed without using AI. For example, the alert unit can input the user's alert history data into AI and have the AI ​​select the optimal notification method.

[0116] The alert unit can estimate the user's emotion and determine the priority of the alert based on the estimated user's emotion. For example, when the user is feeling stressed, the alert unit prioritizes alerts with high urgency. In addition, when the user is relaxed, the alert unit can also notify alerts with normal priority. For example, when the user is in a hurry, the alert unit adjusts the priority so that all alerts are notified immediately. In this way, important alerts can be prioritized by determining the priority of the alert based on the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the alert unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the alert unit can input the user's emotion data to the AI ​​and cause the AI ​​to execute the determination of the priority of the alert.

[0117] The alert unit can give priority to highly relevant alerts at the time of an alert, taking into account the geographical location information of the user. For example, when the user is at home, the alert unit gives priority to security-related alerts at home. Also, when the user is in the office, the alert unit can give priority to work-related alerts. For example, when the user is out, the alert unit gives priority to security-related alerts at the out-of-office location. In this way, by taking into account the geographical location information, it is possible to give priority to highly relevant alerts. Part or all of the above-mentioned processing in the alert unit may be performed, for example, using AI, or may be performed without using AI. For example, the alert unit can input the geographical location data of the user to AI and cause AI to execute the determination of the priority order of highly relevant alerts.

[0118] The monitoring unit can estimate the user's emotion and adjust the display method of the energy usage status based on the estimated user's emotion. For example, when the user is stressed, the monitoring unit provides a simple and highly visible display method. In addition, when the user is relaxed, the monitoring unit can also provide a display method including detailed information. For example, when the user is in a hurry, the monitoring unit provides a display method that focuses on the main points. This makes it possible to display the energy usage status more appropriately by adjusting the display method based on the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the monitoring unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the monitoring unit can input the user's emotion data to the AI ​​and cause the AI ​​to adjust the display method.

[0119] The monitoring unit can select the optimal display method by referring to the user's past usage history when monitoring the energy usage status. For example, the monitoring unit preferentially selects a display method that the user has used in the past. The monitoring unit can also select a display method suitable for a specific time period from the user's past usage history. For example, the monitoring unit analyzes the user's past usage history and selects the most effective display method. In this way, the optimal display method can be selected by referring to the past usage history. A part or all of the above-mentioned processing in the monitoring unit may be performed using, for example, AI, or may be performed without using AI. For example, the monitoring unit can input the user's usage history data to AI and cause the AI ​​to select the optimal display method.

[0120] The monitoring unit can estimate the user's emotion and determine the priority of the energy usage status based on the estimated user's emotion. For example, when the user is stressed, the monitoring unit preferentially displays important energy usage status. Also, when the user is relaxed, the monitoring unit can display the energy usage status with normal priority. For example, when the user is in a hurry, the monitoring unit adjusts the priority to immediately display all energy usage status. In this way, important information can be preferentially displayed by determining the priority of the energy usage status based on the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the monitoring unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the monitoring unit can input the user's emotion data to the AI ​​and cause the AI ​​to execute the determination of the priority of the energy usage status.

[0121] The monitoring unit can display highly relevant information preferentially in consideration of the geographical location information of the user when monitoring the energy usage status. For example, when the user is at home, the monitoring unit preferentially displays the energy usage status at home. Also, when the user is in an office, the monitoring unit can preferentially display the energy usage status at the office. For example, when the user is out, the monitoring unit preferentially displays the energy usage status at the outside. In this way, highly relevant information can be displayed preferentially by considering the geographical location information. Part or all of the above-mentioned processing in the monitoring unit may be performed, for example, using AI, or may be performed without using AI. For example, the monitoring unit can input the geographical location data of the user to AI and cause AI to execute the display of highly relevant information.

[0122] The linking unit can estimate the user's emotion and adjust the notification method of the weather and news based on the estimated user's emotion. For example, when the user is feeling stressed, the linking unit notifies the user of the weather and news with a quiet sound or vibration. Also, when the user is relaxed, the linking unit can notify the user of the weather and news with a normal sound. For example, when the user is in a hurry, the linking unit notifies the user of the weather and news with a conspicuous sound or light. This allows for more appropriate notification by adjusting the notification method based on the user's emotion. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the linking unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the linking unit can input the user's emotion data to the AI ​​and cause the AI ​​to adjust the notification method.

[0123] The linking unit can select the optimal notification method by referring to the user's past notification history when notifying the user of weather or news. For example, the linking unit preferentially selects a notification method that the user has used in the past. The linking unit can also select a notification method suitable for a specific time period from the user's past notification history. For example, the linking unit analyzes the user's past notification history and selects the most effective notification method. In this way, the optimal notification method can be selected by referring to the past notification history. A part or all of the above-mentioned processing in the linking unit may be performed, for example, using AI, or may be performed without using AI. For example, the linking unit can input the user's notification history data into AI and have the AI ​​select the optimal notification method.

[0124] The linking unit can estimate the user's emotion and determine the priority of weather and news based on the estimated user's emotion. For example, when the user is stressed, the linking unit notifies the user of important weather and news with priority. In addition, when the user is relaxed, the linking unit can also notify the user of weather and news with normal priority. For example, when the user is in a hurry, the linking unit adjusts the priority so that all weather and news are notified immediately. In this way, important information can be notified with priority by determining the priority of weather and news based on the user's emotion. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the linking unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the linking unit can input the user's emotion data to the AI ​​and cause the AI ​​to execute the determination of the priority of weather and news.

[0125] When notifying the user of the weather or news, the linking unit can give priority to notifying the user of highly relevant information by taking into account the user's geographical location information. For example, when the user is at home, the linking unit gives priority to notifying the user of the weather and news around the user's home. Also, when the user is in the office, the linking unit can give priority to notifying the user of the weather and news around the office. For example, when the user is out, the linking unit gives priority to notifying the user of the weather and news at the user's destination. In this way, by taking into account the geographical location information, it is possible to give priority to notifying the user of highly relevant information. Part or all of the above-mentioned processing in the linking unit may be performed, for example, using AI, or may be performed without using AI. For example, the linking unit can input the user's geographical location data into AI and cause AI to execute notification of highly relevant information.

[0126] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0127] The voice control system AI assistant for smart homes can analyze the tone and speed of a user's voice to estimate the user's emotions. For example, if the user's voice is high-pitched, the reception unit can estimate that the user is excited and respond quickly. If the user's voice is low and slow, the reception unit can estimate that the user is relaxed and respond normally. If the user's voice is trembling, the reception unit can estimate that the user is nervous and respond calmly. This makes it possible to provide more personalized services by estimating emotions based on the tone and speed of the user's voice and responding appropriately.

[0128] The smart home voice control system AI assistant can analyze the user's behavioral patterns and automatically perform operations based on specific behaviors. For example, the learning unit can learn the user's habit of turning on the living room lights at 7:00 every morning, and automatically turn on the lights before the user enters the living room. The learning unit can also learn the user's habit of turning off the air conditioner at 10:00 every night, and automatically turn off the air conditioner before the user enters the bedroom. Furthermore, the learning unit can learn the user's habit of playing specific music at a specific time, and automatically play music when that time comes. This can improve the user's convenience by automatically performing operations based on the user's behavioral patterns.

[0129] The smart home voice control system AI assistant can monitor the user's health condition and suggest health-conscious operations. For example, if the user's heart rate is high, the suggestion unit can suggest a relaxing environment. Also, if the user's body temperature is high, the suggestion unit can suggest lowering the air conditioner temperature. Furthermore, the suggestion unit can analyze the user's sleep data and suggest an appropriate sleeping environment. This makes it possible to support the user's health by suggesting operations based on the user's health condition.

[0130] A smart home voice control system AI assistant can use the user's schedule information to automatically perform operations at specific times. For example, the suggestion unit can use data from the user's calendar app to adjust the temperature of the air conditioner before a meeting starts. In addition, if the user plans to go out, the suggestion unit can also suggest turning off the lights before going out. In addition, if the user plans to exercise, the suggestion unit can suggest an environment in which the user can relax after exercising. This makes it possible to predict operations more accurately by using schedule information.

[0131] The smart home voice control system AI assistant can estimate the user's emotions and suggest playing music based on the estimated emotions. For example, the suggestion unit can suggest relaxing music when the user is stressed. The suggestion unit can also suggest the user's favorite music when the user is relaxed. Furthermore, the suggestion unit can suggest energetic music when the user is excited. This can improve the user's mood by suggesting playing music based on the user's emotions.

[0132] The smart home voice control system AI assistant can use the geographic location information of the user to suggest operations at a specific location. For example, when the user is at home, the suggestion unit suggests operations at home. Also, when the user is in the office, the suggestion unit can suggest operations at the office. Furthermore, when the user is out, the suggestion unit can suggest operations outside the home. In this way, by using the geographic location information, more appropriate operations can be suggested.

[0133] The smart home voice control system AI assistant can analyze the user's social media activity and suggest related operations. For example, if the user mentions a specific event on social media, the suggestion unit can suggest operations related to the event. Also, if the user mentions a specific place on social media, the suggestion unit can suggest operations related to the place. Furthermore, if the user mentions a specific product on social media, the suggestion unit can suggest operations related to the product. In this way, by analyzing social media activity, related operations can be suggested.

[0134] The smart home voice control system AI assistant can estimate the user's emotions and adjust the display method of energy usage status based on the estimated emotions. For example, the monitoring unit can provide a simple and highly visible display method when the user is stressed. The monitoring unit can also provide a display method including detailed information when the user is relaxed. Furthermore, the monitoring unit can provide a display method that focuses on the main points when the user is in a hurry. This makes it possible to display the energy usage status more appropriately by adjusting the display method based on the user's emotions.

[0135] The smart home voice control system AI assistant can suggest the optimal operation method by referring to the user's past operation history. For example, the suggestion unit preferentially suggests an operation method that the user has frequently used in the past. The suggestion unit can also suggest an operation method suitable for a specific time period from the user's past operation history. Furthermore, the suggestion unit can analyze the user's past operation history and suggest the most efficient operation method. As a result, the optimal operation method can be suggested by referring to the past operation history.

[0136] The smart home voice control system AI assistant can estimate the user's emotions and adjust the way in which suggestions are expressed based on the estimated emotions. For example, the suggestion unit can make suggestions in a simple and calm way when the user is stressed. Also, the suggestion unit can make suggestions with detailed information when the user is relaxed. Furthermore, the suggestion unit can make quick and concise suggestions when the user is in a hurry. This allows more appropriate suggestions to be made by adjusting the way in which suggestions are expressed based on the user's emotions.

[0137] The process flow of the second embodiment will be briefly described below.

[0138] Step 1: The reception unit receives a voice command from a user. For example, the reception unit receives the voice command using a microphone equipped with a noise canceling function, and converts the voice command into text data using a voice recognition technology. Step 2: The control unit operates the electrical appliance based on the voice command received by the reception unit, for example, turning on the lights in the living room or setting the temperature of the air conditioner based on the voice command. Step 3: The learning unit learns the user's daily habits and preferences. For example, a machine learning algorithm is used to learn that the user has a habit of turning on the living room lights at 7am every morning, and their preferred music playlist. Step 4: The suggestion unit suggests predicted actions to the user based on the information learned by the learning unit. For example, when the user enters the living room, the suggestion unit suggests "Should I turn on the lights?" or "Should I play my favorite music?"

[0139] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0140] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0141] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0142] For example, the reception unit receives a voice command via the microphone 38B of the smart device 14 or the communication I / F 26 of the data processing device 12. For example, the control unit operates an electric device via the control unit 46A of the smart device 14 or the specific processing unit 290 of the data processing device 12. For example, the learning unit learns the user's daily habits and preferences via the specific processing unit 290 of the data processing device 12. For example, the suggestion unit suggests predictive operations via the control unit 46A of the smart device 14 or the specific processing unit 290 of the data processing device 12. For example, the alert unit provides a security alert via the camera 42 of the smart device 14 or the specific processing unit 290 of the data processing device 12. For example, the monitoring unit monitors the energy usage status via the output device 40 of the smart device 14 or the specific processing unit 290 of the data processing device 12 and makes suggestions for saving electricity. For example, the linking unit links with the outside via the communication I / F 44 of the smart device 14 or the communication I / F 26 of the data processing device 12 to provide weather information and news. The correspondence between each part and the device or control part is not limited to the above-mentioned example, and various modifications are possible.

[0143] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0144] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0145] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0146] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0147] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0148] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0149] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0150] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0151] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0152] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0153] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0154] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0155] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0156] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0157] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0158] For example, the reception unit receives a voice command via the microphone 238 of the smart glasses 214 or the communication I / F 26 of the data processing device 12. For example, the control unit operates an electric device via the control unit 46A of the smart glasses 214 or the specific processing unit 290 of the data processing device 12. For example, the learning unit learns the user's daily habits and preferences via the specific processing unit 290 of the data processing device 12. For example, the suggestion unit suggests predictive operations via the control unit 46A of the smart glasses 214 or the specific processing unit 290 of the data processing device 12. For example, the alert unit provides a security alert via the camera 42 of the smart glasses 214 or the specific processing unit 290 of the data processing device 12. For example, the monitoring unit monitors the energy usage status via the speaker 240 of the smart glasses 214 or the specific processing unit 290 of the data processing device 12 and makes suggestions for saving electricity. For example, the linking unit links with the outside via the communication I / F 44 of the smart glasses 214 or the communication I / F 26 of the data processing device 12 to provide weather information and news. The correspondence between each part and the device or control part is not limited to the above-mentioned example, and various modifications are possible.

[0159] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0160] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0161] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0162] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0163] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0164] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0165] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0166] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0167] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0168] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0169] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0170] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0171] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0172] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0173] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0174] For example, the reception unit receives a voice command via the microphone 238 of the headset terminal 314 or the communication I / F 26 of the data processing device 12. For example, the control unit operates an electric device via the control unit 46A of the headset terminal 314 or the specific processing unit 290 of the data processing device 12. For example, the learning unit learns the user's daily habits and preferences via the specific processing unit 290 of the data processing device 12. For example, the suggestion unit suggests predictive operations via the control unit 46A of the headset terminal 314 or the specific processing unit 290 of the data processing device 12. For example, the alert unit provides a security alert via the camera 42 of the headset terminal 314 or the specific processing unit 290 of the data processing device 12. For example, the monitoring unit monitors the energy usage status via the display 343 of the headset terminal 314 or the specific processing unit 290 of the data processing device 12 and makes suggestions for saving electricity. For example, the linking unit links with the outside via the communication I / F 44 of the headset terminal 314 or the communication I / F 26 of the data processing device 12 to provide weather information and news. The correspondence between each part and the device or control part is not limited to the above-mentioned example, and various modifications are possible.

[0175] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0176] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0177] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0178] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0179] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0180] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0181] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0182] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0183] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0184] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0185] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0186] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0187] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0188] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0189] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0190] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0191] For example, the reception unit receives voice commands via the microphone 238 of the robot 414 or the communication I / F 26 of the data processing device 12. For example, the control unit operates electric equipment via the control unit 46A of the robot 414 or the specific processing unit 290 of the data processing device 12. For example, the learning unit learns the user's daily habits and preferences via the specific processing unit 290 of the data processing device 12. For example, the suggestion unit suggests predictive operations via the control unit 46A of the robot 414 or the specific processing unit 290 of the data processing device 12. For example, the alert unit provides security alerts via the camera 42 of the robot 414 or the specific processing unit 290 of the data processing device 12. For example, the monitoring unit monitors the energy usage status via the speaker 240 of the robot 414 or the specific processing unit 290 of the data processing device 12 and makes suggestions for saving electricity. For example, the collaboration unit collaborates with the outside via the communication I / F 44 of the robot 414 or the communication I / F 26 of the data processing device 12 to provide weather information and news. The correspondence between each part and the device or control part is not limited to the above-mentioned example, and various modifications are possible.

[0192] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0193] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0194] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0195] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out on emotion map 400 you go, the more visible the emotions become (the more they are expressed in behavior).

[0196] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0197] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0198] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0199] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0200] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0201] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0202] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0203] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0204] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0205] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0206] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0207] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0208] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0209] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0210] (Appendix 1) A reception unit that receives a voice command from a user; a control unit that operates an electrical device based on the voice command received by the reception unit; A learning unit that learns the behavioral patterns and preferences of the user; A suggestion unit that suggests to the user an operation predicted based on the information learned by the learning unit. A system characterized by: (Appendix 2) Equipped with an alert section that provides security alerts 2. The system of claim 1, (Appendix 3) Equipped with a monitoring section that monitors energy usage and makes suggestions for saving electricity 2. The system of claim 1, (Appendix 4) A linking unit is provided to provide the user with weather information and news in cooperation with external parties. 2. The system of claim 1, (Appendix 5) The reception unit is Equipped with noise canceling function 2. The system of claim 1, (Appendix 6) The learning unit is Using machine learning algorithms to learn the user's daily habits and preferences 2. The system of claim 1, (Appendix 7) The suggestion unit, Predicting an operation to be performed by the user based on the information learned by the learning unit and biometric information of the user 2. The system of claim 1, (Appendix 8) The monitoring unit is Equipped with an interface that visualizes energy usage in real time 4. The system of claim 3. (Appendix 9) The suggestion unit, Further using schedule information of the user, the operation by the user is predicted. 2. The system of claim 1, (Appendix 10) The learning unit is Integrating sensor information within the user's residence to learn 2. The system of claim 1, (Appendix 11) The suggestion unit, Suggesting the operation to the user based on the user's health information 2. The system of claim 1, (Appendix 12) The reception unit is Estimating a user's emotion and adjusting the sensitivity of voice commands based on the estimated user's emotion 2. The system of claim 1, (Appendix 13) The reception unit is When accepting a voice command, the system selects the optimal acceptance method by referring to the user's past command history. 2. The system of claim 1, (Appendix 14) The reception unit is When receiving a voice command, the device analyzes the user's current ambient sound to optimize noise cancellation. 2. The system of claim 1, (Appendix 15) The reception unit is Estimate the user's emotions and prioritize voice commands based on the estimated user's emotions. 2. The system of claim 1, (Appendix 16) The reception unit is When accepting voice commands, the system takes into account the user's geographic location information and prioritizes commands that are more relevant to the user. 2. The system of claim 1, (Appendix 17) The reception unit is When receiving a voice command, the system analyzes the user's social media activity and receives related commands. 2. The system of claim 1, (Appendix 18) The control unit is To estimate a user's emotion and adjust an operation method of an electrical device based on the estimated user's emotion 2. The system of claim 1, (Appendix 19) The control unit is When operating electrical equipment, the optimal operation method is selected by referring to the user's past operation history. 2. The system of claim 1, (Appendix 20) The control unit is When operating electrical appliances, the timing of operation is adjusted based on the user's current life situation. 2. The system of claim 1, (Appendix 21) The control unit is To estimate a user's emotion and determine a priority of an electric device to be operated based on the estimated user's emotion. 2. The system of claim 1, (Appendix 22) The control unit is When operating electrical devices, the system takes into account the user's geographic location information and prioritizes the operation of devices that are more relevant to the user. 2. The system of claim 1, (Appendix 23) The control unit is Analyze the user's social media activity when operating electrical devices and operate related devices 2. The system of claim 1, (Appendix 24) The learning unit is Estimate the user's emotions and select learning data based on the estimated user emotions. 2. The system of claim 1, (Appendix 25) The learning unit is During training, the learning algorithm is optimized by referencing past training data. 2. The system of claim 1, (Appendix 26) The learning unit is Analyze the user's life patterns during learning to improve the accuracy of learning 2. The system of claim 1, (Appendix 27) The learning unit is Infer user emotions and adjust the frequency of learning based on the estimated user emotions. 2. The system of claim 1, (Appendix 28) The learning unit is During training, the training data is weighted taking into account the user's geographic location. 2. The system of claim 1, (Appendix 29) The learning unit is During learning, the learning data is supplemented by analyzing users' social media activity. 2. The system of claim 1, (Appendix 30) The suggestion unit, Infer user emotions and adjust the presentation of suggestions based on the inferred user emotions 2. The system of claim 1, (Appendix 31) The suggestion unit, When making a proposal, the system selects the optimal proposal method by referring to the user's past operation history. 2. The system of claim 1, (Appendix 32) The suggestion unit, When making suggestions, adjust the timing of the suggestions based on the user's current life situation 2. The system of claim 1, (Appendix 33) The suggestion unit, Estimate user sentiment and prioritize suggestions based on the estimated user sentiment 2. The system of claim 1, (Appendix 34) The suggestion unit, When making suggestions, the app takes into account the user's geographic location to prioritize more relevant suggestions. 2. The system of claim 1, (Appendix 35) The suggestion unit, When making a suggestion, analyze the user's social media activity and make relevant suggestions 2. The system of claim 1, (Appendix 36) The alert unit is Infer user emotions and adjust alert notification methods based on the estimated user emotions 3. The system of claim 2, (Appendix 37) The alert unit is When an alert occurs, the system selects the optimal notification method by referring to the user's past alert history. 3. The system of claim 2, (Appendix 38) The alert unit is Infer user emotions and prioritize alerts based on the inferred user emotions 3. The system of claim 2, (Appendix 39) The alert unit is At the time of alert, the system takes into account the user's geographic location to prioritize the most relevant alerts. 3. The system of claim 2, (Appendix 40) The monitoring unit is Inferring a user's emotion and adjusting the way energy usage is displayed based on the estimated user's emotion 4. The system of claim 3. (Appendix 41) The monitoring unit is When monitoring energy usage, select the optimal display method by referring to the user's past usage history. 4. The system of claim 3. (Appendix 42) The monitoring unit is To estimate a user's emotion and prioritize energy usage based on the estimated user's emotion. 4. The system of claim 3. (Appendix 43) The monitoring unit is When monitoring energy usage, the system takes into account the user's geographic location to prioritize the most relevant information. 4. The system of claim 3. (Appendix 44) The linking portion is Infer user emotions and adjust weather and news notifications based on the inferred user emotions 5. The system of claim 4. (Appendix 45) The linking portion is When notifying users of weather or news, the system selects the optimal notification method by referring to the user's past notification history. 5. The system of claim 4. (Appendix 46) The linking portion is Estimate user emotions and prioritize weather and news based on the estimated user emotions 5. The system of claim 4. (Appendix 47) The linking portion is When notifying users of weather and news, the app takes their geographic location into account to prioritize more relevant information. 5. The system of claim 4. [Explanation of symbols]

[0211] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A reception unit that receives a voice command from a user; a control unit that operates an electrical device based on the voice command received by the reception unit; A learning unit that learns the behavioral patterns and preferences of the user; A suggestion unit that suggests to the user an operation predicted based on the information learned by the learning unit. A system characterized by:

2. Equipped with an alert section that provides security alerts 2. The system of claim 1 .

3. Equipped with a monitoring section that monitors energy usage and makes suggestions for saving electricity 2. The system of claim 1 .

4. A linking unit is provided to provide the user with weather information and news in cooperation with external parties.

2. The system of claim 1 .

5. The reception unit is Equipped with noise canceling function 2. The system of claim 1 .

6. The learning unit is Using machine learning algorithms to learn the user's daily habits and preferences 2. The system of claim 1 .

7. The suggestion unit, Predicting an operation to be performed by the user based on the information learned by the learning unit and biometric information of the user 2. The system of claim 1 .

8. The monitoring unit is Equipped with an interface that visualizes energy usage in real time 4. The system of claim 3.

Images