System

Abstract

An object of a system according to an embodiment is to efficiently automate or remotely execute daily tasks in a home.SOLUTION: A system includes a learning unit, an execution unit, and a provision unit. The learning unit learns specific daily work in the home of the user. The execution unit automatically or remotely executes a specific daily task based on the information learned by the learning unit. The providing unit provides the user with a result of the execution by the execution unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technologies have difficulty efficiently automating or remotely performing routine household tasks, and there is room for improvement.

[0005] The system according to the embodiment aims to efficiently automate or remotely perform routine household tasks. [Means for solving the problem]

[0006] The system according to the embodiment includes a learning unit, an execution unit, and a providing unit. The learning unit learns specific daily tasks performed in a user's home. The execution unit automates or remotely performs the specific daily tasks based on the information learned by the learning unit. The providing unit provides the user with the results of the tasks performed by the execution unit. [Effects of the Invention]

[0007] An embodiment of the system can efficiently automate or remotely perform routine household tasks. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) A home automation system according to an embodiment of the present invention is a system that learns a user's daily household tasks and performs them automatically or remotely. The home automation system learns a user's daily household tasks, automates or remotely performs the tasks based on the learned information, and provides the results to the user. For example, the home automation system learns how to handle laundry and has a remote-controlled robot iron the laundry. For example, the home automation system analyzes a user's eating habits, suggests meals, provides recipes, and automatically cooks meals. For example, the home automation system manages household priorities and deadlines and performs laundry, cleaning, and other tasks at the appropriate time. For example, the home automation system learns to identify and deal with pests and automatically performs extermination activities at the appropriate time. For example, the home automation system monitors the growth status of plants and optimizes watering, fertilization, light exposure, and other tasks. This allows the home automation system to significantly reduce the time spent on housework and help improve the user's lifestyle. This allows the home automation system to significantly reduce the time spent on housework and help improve the user's lifestyle. It will also be a useful system for elderly people and people with disabilities in terms of supporting their independence in daily life.

[0029] A home automation system according to an embodiment includes a learning unit, an execution unit, and a provision unit. The learning unit learns a user's daily household tasks. For example, the learning unit can learn how to handle laundry. The learning unit can analyze the user's eating habits, suggest meals, provide recipes, and automatically cook meals. The learning unit can manage household priorities and deadlines, and perform laundry, cleaning, and other tasks at appropriate times. The learning unit can learn how to identify and deal with pests, and automatically perform extermination activities at appropriate times. The learning unit can monitor the growth status of plants and optimize watering, fertilization, light exposure time, and other tasks. The execution unit automates or remotely performs daily tasks based on the information learned by the learning unit. For example, the execution unit can enable a remote-controlled robot to iron clothes. The execution unit can enable a remote-controlled cooking robot to automatically cook based on a suggested recipe. The execution unit can suggest the best day to hang out laundry based on a weather forecast. The execution unit can adjust the cleaning timing depending on the room usage status. The execution unit can identify pests using a camera, and a remote-controlled extermination robot can perform extermination activities. The execution unit can monitor the moisture content and nutritional status of plants using sensors, and a remote-controlled planter can automatically add water and fertilizer. The providing unit provides the user with the results of the execution performed by the execution unit. For example, the providing unit can estimate the user's emotions and adjust the way the information is presented based on the estimated user's emotions. When providing the results of the execution performed by the execution unit, the providing unit can improve the accuracy of the information by reflecting the user's past feedback. The providing unit can customize the content of the information to be provided based on the user's lifestyle and family composition. When providing the results of the execution performed by the execution unit, the providing unit can provide the information in an optimal format based on the user's device or platform. As a result, the home automation system according to the embodiment can efficiently automate or remotely perform daily household tasks for the user and provide the results.

[0030] The learning unit learns how to handle specific laundry items, allowing the remote-controlled robot to iron them. The learning unit, for example, analyzes the material and wrinkle condition of the laundry and learns how to iron at the optimal temperature and pressure. For example, the learning unit can suggest ironing at a low temperature for cotton clothes. The learning unit can also suggest ironing at a low pressure for silk clothes. The learning unit can also suggest ironing at a high temperature and high pressure for heavily wrinkled clothes. In this way, the remote-controlled robot learns how to handle laundry and irons it, thereby making laundry management more efficient. Some or all of the above-mentioned processing in the learning unit may be performed using, for example, AI, or may be performed without using AI. For example, the learning unit can input the material and wrinkle condition of the laundry items into a generation AI and have the generation AI execute the optimal ironing method.

[0031] The learning unit can analyze the user's diet, suggest specific meals, provide recipes, and automatically cook. The learning unit can, for example, analyze the user's past diet history and health status and suggest nutritionally balanced meals. For example, if the user has previously eaten many high-calorie meals, the learning unit can suggest low-calorie, nutritionally balanced meals. Furthermore, if the user has previously eaten few vegetables, the learning unit can suggest meals that are high in vegetables. Furthermore, if the user has previously had a specific allergy, the learning unit can suggest meals that do not contain that allergen. This analysis of the user's diet, suggesting meals, providing recipes, and automatically cooking food improves the efficiency of dietary management. Some or all of the above-described processing by the learning unit may be performed using, or without, AI. For example, the learning unit can input the user's diet history and health status into the generation AI and cause the generation AI to suggest nutritionally balanced meals.

[0032] The learning unit manages the priorities and deadlines of housework, and can perform laundry, cleaning, etc. at specific times. The learning unit, for example, suggests the optimal day to hang out laundry based on the weather forecast. For example, the learning unit can suggest choosing a sunny day to hang out laundry. The learning unit also adjusts the timing of cleaning depending on the room usage. For example, the learning unit can suggest prioritizing cleaning of rooms that are used frequently. The learning unit can also suggest regularly cleaning rooms that are used less frequently. This improves the efficiency of housework by managing the priorities and deadlines of housework and performing laundry, cleaning, etc. at appropriate times. Some or all of the above-mentioned processing in the learning unit may be performed using, for example, AI, or may be performed without AI. For example, the learning unit can input weather forecast data and room usage data into the generation AI and have the generation AI execute an optimal housework schedule.

[0033] The learning unit learns how to identify and treat pests and can automatically perform extermination activities at specific times. For example, the learning unit identifies pests using a camera and learns how the remote-controlled extermination robot performs extermination activities. For example, the learning unit can identify mosquitoes and suggest using mosquito coils or electric mosquito repellents. The learning unit can also identify cockroaches and suggest using cockroach traps or insecticide sprays. The learning unit can also identify mites and suggest using mite removal sheets or vacuum cleaners. This allows the robot to learn how to identify and treat pests and automatically perform extermination activities at appropriate times, thereby streamlining pest control. Some or all of the above-described processing in the learning unit may be performed using, for example, AI, or without AI. For example, the learning unit can input image data of pests captured by a camera into the generation AI and have the generation AI execute the optimal extermination method.

[0034] The learning unit can monitor the plant's growth status and specifically optimize watering, fertilization, light exposure time, and the like. For example, the learning unit monitors the plant's moisture content and nutritional status using a sensor, and the remote-controlled planter learns how to automatically add water and fertilizer. For example, the learning unit can suggest automatically adding water when the plant's moisture content decreases. The learning unit can also suggest automatically adding fertilizer when the plant's nutritional status is insufficient. The learning unit can also suggest adjusting the light exposure time to optimize the plant's light exposure time. This allows the plant's growth status to be monitored and the optimization of watering, fertilization, light exposure time, and the like, resulting in more efficient plant care. Some or all of the above-described processing in the learning unit may be performed using, or without, AI. For example, the learning unit can input plant data acquired by a sensor into the generation AI and have the generation AI execute the optimal care method.

[0035] The execution unit enables the remote-controlled robot to specifically iron the laundry. For example, the execution unit enables the remote-controlled robot to analyze the material and wrinkle state of the laundry in real time and iron the laundry at the optimal temperature and pressure. For example, the execution unit can iron cotton clothes at a low temperature. The execution unit can also iron silk clothes at a low pressure. The execution unit can also iron heavily wrinkled clothes at a high temperature and high pressure. This allows the remote-controlled robot to iron the laundry, making ironing more efficient. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without using AI. For example, the execution unit can input the material and wrinkle state of the laundry into the generation AI and have the generation AI execute the optimal ironing method.

[0036] The execution unit enables the remote-controlled cooking robot to automatically cook based on a specific suggested recipe. For example, the execution unit allows the remote-controlled cooking robot to analyze a user's eating habits and automatically cook based on a suggested recipe. For example, the execution unit can automatically cook a nutritionally balanced meal taking into account the user's past eating history and health condition. The execution unit can also customize recipes according to the user's preferences and automatically cook the meal. The execution unit can also select an optimal cooking method taking into account the freshness and inventory status of ingredients and automatically cook the meal. This allows the remote-controlled cooking robot to automatically cook based on a suggested recipe, thereby improving cooking efficiency. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without using AI. For example, the execution unit can input the suggested recipe into a generation AI and have the generation AI execute the optimal cooking method.

[0037] The execution unit can suggest a specific day for hanging out laundry based on a weather forecast. The execution unit, for example, analyzes weather forecast data and suggests the optimal day for hanging out laundry. For example, the execution unit can suggest choosing a sunny day to hang out laundry. The execution unit can also suggest the optimal day for hanging out laundry by taking into account weather conditions such as humidity and wind speed. The execution unit can also adjust the hanging method depending on the amount and type of laundry. In this way, by suggesting the optimal day for hanging out laundry based on the weather forecast, laundry management becomes more efficient. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without using AI. For example, the execution unit can input weather forecast data into a generation AI and have the generation AI execute the optimal method for hanging out laundry.

[0038] The execution unit can adjust the specific cleaning timing depending on the room usage status. The execution unit, for example, analyzes the frequency of use and degree of dirtiness of the room and adjusts the cleaning timing. For example, the execution unit can suggest that frequently used rooms be cleaned first. The execution unit can also suggest that less frequently used rooms be cleaned regularly. The execution unit can also suggest a special cleaning method (e.g., steam cleaning) for heavily soiled rooms. This allows for cleaning to be made more efficient by adjusting the cleaning timing depending on the room usage status. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without using AI. For example, the execution unit can input room usage status data into the generation AI and have the generation AI execute an optimal cleaning schedule.

[0039] The execution unit can identify pests using a camera, and the remote-controlled extermination robot can carry out extermination activities. The execution unit can, for example, identify pests using a camera, and the remote-controlled extermination robot can carry out extermination activities. For example, the execution unit can identify mosquitoes and use mosquito coils or electric mosquito repellents. The execution unit can also identify cockroaches and use cockroach traps or insecticide sprays. The execution unit can also identify mites and use mite removal sheets or vacuum cleaners. In this way, pests are identified using a camera, and the remote-controlled extermination robot carries out extermination activities, thereby making pest extermination more efficient. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without using AI. For example, the execution unit can input image data of pests acquired by a camera into a generation AI, and have the generation AI execute an optimal extermination method.

[0040] The execution unit monitors the water content and nutritional status of the plant using sensors, and the remote-controlled planter can automatically add water and fertilizer. The execution unit, for example, monitors the water content and nutritional status of the plant using sensors, and the remote-controlled planter can automatically add water and fertilizer. For example, the execution unit can automatically add water when the water content of the plant decreases. The execution unit can also automatically add fertilizer when the plant's nutritional status is insufficient. The execution unit can also adjust the care method according to the plant's growth stage and season. This makes plant management more efficient by monitoring the water content and nutritional status of the plant using sensors and the remote-controlled planter automatically adding water and fertilizer. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without AI. For example, the execution unit can input plant data obtained by sensors into the generation AI and have the generation AI execute the optimal care method.

[0041] When providing the results executed by the execution unit, the providing unit can improve the accuracy of the information by reflecting the user's past feedback. The providing unit, for example, collects the user's past feedback and improves the accuracy of the information. For example, the providing unit can preferentially provide information formats for which the user has given positive feedback in the past. The providing unit can also avoid information formats for which the user has given negative feedback in the past. The providing unit can also analyze the user's past feedback and provide the optimal information format. This improves the accuracy of the information by reflecting the user's past feedback. Some or all of the above-mentioned processing in the providing unit may be performed, for example, using AI or without AI. For example, the providing unit can input the user's feedback data into the generation AI and cause the generation AI to improve the accuracy of the information.

[0042] The providing unit can customize the content of the information to be provided according to the user's lifestyle and family composition. The providing unit, for example, analyzes the user's lifestyle and family composition and customizes the content of the information. For example, if the user is active early in the morning, the providing unit can prioritize providing information related to morning hours. Furthermore, if the user is a nocturnal person, the providing unit can prioritize providing information related to evening hours. Furthermore, the providing unit can provide information useful to all family members according to the user's family composition. This enables more appropriate information to be provided by customizing the information according to the user's lifestyle and family composition. Some or all of the above-described processing by the providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the providing unit can input the user's lifestyle and family composition data into the generation AI and cause the generation AI to customize the information.

[0043] When providing the results of the execution by the execution unit, the providing unit can provide information in an optimal format depending on the user's device and platform. The providing unit, for example, analyzes the user's device and platform and provides the information in the optimal format. For example, if the user is using a smartphone, the providing unit can provide information in a format optimized for mobile devices. Furthermore, if the user is using a tablet, the providing unit can provide information in a format optimized for a large screen. Furthermore, if the user is using a personal computer, the providing unit can provide information in a format optimized for desktops. This improves the visibility of the information by providing information depending on the user's device and platform. Some or all of the above-described processing by the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input data of the user's device and platform into the generation AI and have the generation AI execute the optimal information format.

[0044] When providing the results of the execution by the execution unit, the providing unit can prioritize providing highly relevant information by taking into account the user's geographical location information. The providing unit, for example, analyzes the user's geographical location information and provides highly relevant information. For example, when the user is in a specific area, the providing unit can provide information related to that area. Furthermore, when the user is traveling, the providing unit can provide information related to the travel destination. Furthermore, when the user is at home, the providing unit can provide information related to the area around the user's home. This makes it possible to provide highly relevant information by taking the user's geographical location information into consideration. Some or all of the above-described processing by the providing unit may be performed using AI, for example, or may be performed without using AI. For example, the providing unit can input the user's geographical location information to a generation AI and cause the generation AI to execute highly relevant information.

[0045] The providing unit can customize the content of the information to be provided by analyzing the user's social media activities and online behavior. The providing unit, for example, analyzes the user's social media activities and online behavior to customize the content of the information. For example, the providing unit can provide information related to topics in which the user has shown interest on social media. The providing unit can also analyze the user's online behavior and provide related information. The providing unit can also provide related information by referring to the activities of the user's friends on social media. This makes it possible to provide more relevant information by analyzing the user's social media activities and online behavior. Some or all of the above-described processing by the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input the user's social media activities and online behavior data into a generation AI and cause the generation AI to customize the information.

[0046] When providing the results executed by the execution unit, the providing unit can customize the format of the information by reflecting the user's past feedback. The providing unit, for example, collects the user's past feedback and customizes the format of the information. For example, the providing unit can preferentially provide information formats for which the user has given positive feedback in the past. The providing unit can also avoid information formats for which the user has given negative feedback in the past. The providing unit can also analyze the user's past feedback and provide the optimal information format. In this way, the information format is optimized for the user by reflecting the user's past feedback. Some or all of the above-described processing in the providing unit may be performed, for example, using AI or without AI. For example, the providing unit can input the user's feedback data to a generation AI and cause the generation AI to customize the information format.

[0047] When learning how to handle laundry, the learning unit can suggest an optimal washing method based on the type of material and the degree of dirt. The learning unit, for example, analyzes the material and degree of dirt of the laundry and suggests an optimal washing method. For example, the learning unit can suggest washing at low temperature and using fabric softener for cotton clothes. The learning unit can also suggest hand washing and using special detergent for silk clothes. The learning unit can also suggest using a detergent specially designed for oil stains in advance for heavily oil-stained clothes. This improves laundry efficiency by suggesting an optimal washing method based on the type of material and the degree of dirt. Some or all of the above-described processing in the learning unit may be performed using, for example, AI, or may be performed without AI. For example, the learning unit can input the material and degree of dirt of the laundry into the generation AI and have the generation AI execute the optimal washing method.

[0048] When analyzing the user's diet, the learning unit can suggest nutritionally balanced meals taking into account the user's past dietary history and health condition. The learning unit, for example, analyzes the user's past dietary history and health condition and suggests nutritionally balanced meals. For example, if the user has previously eaten many high-calorie meals, the learning unit can suggest low-calorie, nutritionally balanced meals. Furthermore, if the user has previously eaten few vegetables, the learning unit can suggest meals that are high in vegetables. Furthermore, if the user has previously had a specific allergy, the learning unit can suggest meals that do not contain the allergen. This improves the user's health management by suggesting nutritionally balanced meals taking into account the user's past dietary history and health condition. Some or all of the above-described processing by the learning unit may be performed using, for example, AI, or may be performed without AI. For example, the learning unit can input the user's dietary history and health condition into the generation AI and cause the generation AI to suggest nutritionally balanced meals.

[0049] The learning unit can suggest the optimal timing for managing housework priorities and deadlines, taking into account the schedules of all family members. The learning unit, for example, analyzes the schedules of all family members and suggests the optimal timing for housework. For example, the learning unit can suggest cleaning during a time when all family members are out. The learning unit can also suggest preparing meals during a time when all family members are at home. The learning unit can also suggest doing laundry during a time when all family members are asleep. This improves the efficiency of housework by suggesting the optimal timing in consideration of the schedules of all family members. Some or all of the above-mentioned processing in the learning unit may be performed using, for example, AI, or may be performed without using AI. For example, the learning unit can input schedule data of all family members into the generation AI and have the generation AI execute the optimal housework schedule.

[0050] When learning how to identify and deal with pests, the learning unit can propose optimal extermination methods according to the local climate and season. The learning unit, for example, analyzes the local climate and season and proposes optimal extermination methods. For example, the learning unit can propose mosquito extermination methods in the summer. The learning unit can also propose cockroach extermination methods in the winter. The learning unit can also propose extermination methods for pests that prefer humidity in the rainy season. This makes pest extermination more efficient by proposing optimal extermination methods according to the local climate and season. Some or all of the above-mentioned processing in the learning unit may be performed using, or without, AI, for example. For example, the learning unit can input local climate and seasonal data into the generation AI and have the generation AI execute the optimal extermination method.

[0051] When inspecting the growth status of a plant, the learning unit can suggest an optimal care method according to the type and growth stage of the plant. The learning unit, for example, analyzes the type and growth stage of the plant and suggests the optimal care method. For example, the learning unit can suggest fertilizer to promote growth for young plants. The learning unit can also suggest watering frequency for maintenance of mature plants. The learning unit can also suggest appropriate treatment methods for diseased plants. In this way, by suggesting the optimal care method according to the type and growth stage of the plant, plant management becomes more efficient. Some or all of the above-described processing in the learning unit may be performed using, for example, AI, or may be performed without using AI. For example, the learning unit can input plant type and growth stage data into the generation AI and have the generation AI execute the optimal care method.

[0052] The learning unit can learn the user's lifestyle rhythm and propose an optimal housework schedule. The learning unit, for example, analyzes the user's lifestyle rhythm and proposes an optimal housework schedule. For example, if the user is active early in the morning, the learning unit can propose a schedule that concentrates housework in the morning hours. Furthermore, if the user is a nocturnal person, the learning unit can propose a schedule that concentrates housework in the evening hours. Furthermore, if the user has an irregular lifestyle, the learning unit can propose a flexible housework schedule. In this way, by learning the user's lifestyle rhythm and proposing an optimal housework schedule, housework efficiency is improved. Some or all of the above-mentioned processing in the learning unit may be performed using, for example, AI, or may be performed without using AI. For example, the learning unit can input the user's lifestyle rhythm data into a generation AI and cause the generation AI to execute an optimal housework schedule.

[0053] When the remote-controlled robot irons, the execution unit can analyze the material and wrinkle state of the laundry in real time and iron it at the optimal temperature and pressure. For example, the execution unit can cause the remote-controlled robot to analyze the material and wrinkle state of the laundry in real time and iron it at the optimal temperature and pressure. For example, the execution unit can iron cotton clothes at a low temperature. The execution unit can also iron silk clothes at a low pressure. The execution unit can also iron heavily wrinkled clothes at a high temperature and high pressure. This improves ironing efficiency by analyzing the material and wrinkle state of the laundry in real time and ironing it at the optimal temperature and pressure. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without AI. For example, the execution unit can input the material and wrinkle state of the laundry into a generation AI and have the generation AI execute the optimal ironing method.

[0054] When the remote-controlled cooking robot performs automatic cooking, the execution unit can adjust the cooking method taking into account the freshness and stock status of ingredients. For example, the execution unit allows the remote-controlled cooking robot to analyze the freshness and stock status of ingredients, select the optimal cooking method, and perform automatic cooking. For example, the execution unit can select a simple cooking method when using highly fresh ingredients. Furthermore, the execution unit can extend the cooking time when using less fresh ingredients to cook safely. Furthermore, the execution unit can combine ingredients that are in low stock with other ingredients when using ingredients that are in low stock. This improves cooking efficiency by adjusting the cooking method taking into account the freshness and stock status of ingredients. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without AI. For example, the execution unit can input ingredient freshness and stock status data into the generation AI and have the generation AI execute the optimal cooking method.

[0055] When suggesting the optimal day to hang out laundry based on the weather forecast, the execution unit can adjust the hanging method depending on the amount and type of laundry. The execution unit, for example, analyzes weather forecast data and suggests the optimal day to hang out laundry. For example, the execution unit can suggest choosing a sunny day to hang out laundry. The execution unit can also suggest the optimal day to hang out laundry by taking into account weather conditions such as humidity and wind speed. The execution unit can also adjust the hanging method depending on the amount and type of laundry. For example, if there is a large amount of laundry, the execution unit can suggest hanging it in a well-ventilated place. If there is laundry made of delicate materials, the execution unit can suggest hanging it in the shade. If there is laundry made of materials that are difficult to dry, the execution unit can suggest using a dryer in addition. In this way, laundry management is made more efficient by adjusting the hanging method depending on the amount and type of laundry. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without using AI. For example, the execution unit can input weather forecast data and data on the amount and type of laundry into the generation AI, and have the generation AI execute the optimal drying method.

[0056] When adjusting the cleaning timing according to the room usage status, the execution unit can select a cleaning method taking into account the degree of dirtiness and frequency of use of the room. The execution unit, for example, analyzes the frequency of use and degree of dirtiness of the room and adjusts the cleaning timing. For example, the execution unit can suggest cleaning a frequently used room every day. The execution unit can also suggest cleaning a less frequently used room once a week. The execution unit can also suggest a special cleaning method (e.g., steam cleaning) for a heavily soiled room. This improves cleaning efficiency by adjusting the cleaning timing according to the room usage status. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without using AI. For example, the execution unit can input room usage status data into a generation AI and have the generation AI execute an optimal cleaning schedule.

[0057] The execution unit can identify pests using a camera, and when the remote-controlled extermination robot performs extermination activities, select the optimal extermination method depending on the type and number of pests. The execution unit, for example, identifies pests using a camera, and the remote-controlled extermination robot performs extermination activities. For example, if there are many mosquitoes, the execution unit can use mosquito coils or an electric mosquito repellent. If there are many cockroaches, the execution unit can use cockroach traps or insecticide spray. If there are many mites, the execution unit can use mite removal sheets or a vacuum cleaner. This improves the efficiency of pest extermination by selecting the optimal extermination method depending on the type and number of pests. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without AI. For example, the execution unit can input image data of pests captured by a camera into the generation AI and have the generation AI execute the optimal extermination method.

[0058] The execution unit monitors the water content and nutrient status of the plant using sensors, and when the remote-controlled planter automatically adds water or fertilizer, it can adjust the care method according to the plant's growth stage and season. For example, the execution unit monitors the water content and nutrient status of the plant using sensors, and the remote-controlled planter automatically adds water or fertilizer. For example, the execution unit can frequently add water or fertilizer to plants in the growing stage. The execution unit can also refrain from adding water or fertilizer to plants in the dormant stage. The execution unit can also suggest different care methods for each season (e.g., moving the plant indoors in winter). This allows for more efficient plant management by adjusting the care method according to the plant's growth stage and season. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without AI. For example, the execution unit can input plant data obtained by sensors into a generation AI and have the generation AI execute an optimal care method.

[0059] When providing the results executed by the execution unit, the providing unit can improve the accuracy of the information by reflecting the user's past feedback. The providing unit, for example, collects the user's past feedback and improves the accuracy of the information. For example, the providing unit can preferentially provide information formats for which the user has given positive feedback in the past. The providing unit can also avoid information formats for which the user has given negative feedback in the past. The providing unit can also analyze the user's past feedback and provide the optimal information format. This improves the accuracy of the information by reflecting the user's past feedback. Some or all of the above-mentioned processing in the providing unit may be performed, for example, using AI or without AI. For example, the providing unit can input the user's feedback data into the generation AI and cause the generation AI to improve the accuracy of the information.

[0060] The providing unit can customize the content of the information to be provided according to the user's lifestyle and family composition. The providing unit, for example, analyzes the user's lifestyle and family composition and customizes the content of the information. For example, if the user is active early in the morning, the providing unit can prioritize providing information related to morning hours. Furthermore, if the user is a nocturnal person, the providing unit can prioritize providing information related to evening hours. Furthermore, the providing unit can provide information useful to all family members according to the user's family composition. This enables more appropriate information to be provided by customizing the information according to the user's lifestyle and family composition. Some or all of the above-described processing by the providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the providing unit can input the user's lifestyle and family composition data into the generation AI and cause the generation AI to customize the information.

[0061] When providing the results of the execution by the execution unit, the providing unit can provide information in an optimal format depending on the user's device and platform. The providing unit, for example, analyzes the user's device and platform and provides the information in the optimal format. For example, if the user is using a smartphone, the providing unit can provide information in a format optimized for mobile devices. Furthermore, if the user is using a tablet, the providing unit can provide information in a format optimized for a large screen. Furthermore, if the user is using a personal computer, the providing unit can provide information in a format optimized for desktops. This improves the visibility of the information by providing information depending on the user's device and platform. Some or all of the above-described processing by the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input data of the user's device and platform into the generation AI and have the generation AI execute the optimal information format.

[0062] When providing the results of the execution by the execution unit, the providing unit can prioritize providing highly relevant information by taking into account the user's geographical location information. The providing unit, for example, analyzes the user's geographical location information and provides highly relevant information. For example, when the user is in a specific area, the providing unit can provide information related to that area. Furthermore, when the user is traveling, the providing unit can provide information related to the travel destination. Furthermore, when the user is at home, the providing unit can provide information related to the area around the user's home. This makes it possible to provide highly relevant information by taking the user's geographical location information into consideration. Some or all of the above-described processing by the providing unit may be performed using AI, for example, or may be performed without using AI. For example, the providing unit can input the user's geographical location information to a generation AI and cause the generation AI to execute highly relevant information.

[0063] The providing unit can customize the content of the information to be provided by analyzing the user's social media activities and online behavior. The providing unit, for example, analyzes the user's social media activities and online behavior to customize the content of the information. For example, the providing unit can provide information related to topics in which the user has shown interest on social media. The providing unit can also analyze the user's online behavior and provide related information. The providing unit can also provide related information by referring to the activities of the user's friends on social media. This makes it possible to provide more relevant information by analyzing the user's social media activities and online behavior. Some or all of the above-described processing by the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input the user's social media activities and online behavior data into a generation AI and cause the generation AI to customize the information.

[0064] When providing the results executed by the execution unit, the providing unit can customize the format of the information by reflecting the user's past feedback. The providing unit, for example, collects the user's past feedback and customizes the format of the information. For example, the providing unit can preferentially provide information formats for which the user has given positive feedback in the past. The providing unit can also avoid information formats for which the user has given negative feedback in the past. The providing unit can also analyze the user's past feedback and provide the optimal information format. In this way, the information format is optimized for the user by reflecting the user's past feedback. Some or all of the above-described processing in the providing unit may be performed, for example, using AI or without AI. For example, the providing unit can input the user's feedback data to a generation AI and cause the generation AI to customize the information format.

[0065] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0066] The home automation system may further include an energy management unit. The energy management unit may monitor power consumption in the home and suggest efficient energy usage. For example, the energy management unit may adjust the usage schedule of home appliances to avoid peak power consumption times. The energy management unit may also work with a solar power generation system to optimize power usage according to the amount of power generated. The energy management unit may also analyze power consumption data and recommend the use of energy-efficient home appliances. This may improve the efficiency of energy consumption in the home and contribute to reducing power costs.

[0067] The home automation system may further include a health management unit. The health management unit may monitor the user's health condition and make suggestions for maintaining good health. For example, the health management unit may analyze the user's sleep patterns and suggest an optimal sleeping environment. The health management unit may also monitor the user's exercise volume and suggest an appropriate exercise plan. The health management unit may also analyze the user's diet and suggest nutritionally balanced meals. This may improve the efficiency of the user's health management and help maintain good health.

[0068] The home automation system can further include a security management unit. The security management unit can monitor home security and issue an alert if it detects an abnormality. For example, the security management unit can monitor the opening and closing of doors and windows and issue an alert if it detects suspicious movement. The security management unit can also analyze camera footage and issue an alert if it identifies a suspicious person. The security management unit can also issue an alert if it detects an abnormality such as a fire or gas leak. This strengthens home security, allowing people to live with peace of mind.

[0069] The home automation system may further include a communication unit. The communication unit supports communication between users and facilitates information sharing within the home. For example, the communication unit may support sending and receiving messages between family members. The communication unit may also share and coordinate family schedules. The communication unit may also share important information within the home (e.g., shopping lists and progress of household chores). This facilitates communication within the home and makes information sharing more efficient.

[0070] The home automation system may further include an environment monitoring unit. The environment monitoring unit may monitor the environmental conditions in the home and make suggestions to maintain a comfortable environment. For example, the environment monitoring unit may monitor the indoor temperature and humidity and suggest optimal air conditioning settings. The environment monitoring unit may also monitor air quality and suggest the use of an air purifier as needed. The environment monitoring unit may also adjust the brightness and color temperature of lighting to provide a comfortable lighting environment. This optimizes the home environment and provides a comfortable living space.

[0071] The processing flow of the first embodiment will be briefly explained below.

[0072] Step 1: The learning module learns the user's daily household tasks, such as how to handle laundry, analyze diet, manage household priorities and deadlines, identify and deal with pests, and monitor plant growth. Step 2: The execution unit automates or remotely executes routine tasks based on the information learned by the learning unit. For example, ironing with a remote-controlled robot, automatic cooking with a cooking robot, laundry drying suggestions based on the weather forecast, adjusting cleaning schedules based on room usage, identifying pests with cameras and exterminating them with an extermination robot, monitoring plant moisture and nutrient status with sensors and automatically adding water and fertilizer with a planter, etc. Step 3: The providing unit provides the results of the execution by the execution unit to the user. For example, it estimates the user's emotions and adjusts the way information is presented based on the estimated emotions, improves the accuracy of the information by reflecting past feedback, customizes the information according to the user's lifestyle and family structure, and provides information in the optimal format according to the user's device and platform.

[0073] (Example 2) A home automation system according to an embodiment of the present invention is a system that learns a user's daily household tasks and performs them automatically or remotely. The home automation system learns a user's daily household tasks, automates or remotely performs the tasks based on the learned information, and provides the results to the user. For example, the home automation system learns how to handle laundry and has a remote-controlled robot iron the laundry. For example, the home automation system analyzes a user's eating habits, suggests meals, provides recipes, and automatically cooks meals. For example, the home automation system manages household priorities and deadlines and performs laundry, cleaning, and other tasks at the appropriate time. For example, the home automation system learns to identify and deal with pests and automatically performs extermination activities at the appropriate time. For example, the home automation system monitors the growth status of plants and optimizes watering, fertilization, light exposure, and other tasks. This allows the home automation system to significantly reduce the time spent on housework and help improve the user's lifestyle. This allows the home automation system to significantly reduce the time spent on housework and help improve the user's lifestyle. It will also be a useful system for elderly people and people with disabilities in terms of supporting their independence in daily life.

[0074] A home automation system according to an embodiment includes a learning unit, an execution unit, and a provision unit. The learning unit learns a user's daily household tasks. For example, the learning unit can learn how to handle laundry. The learning unit can analyze the user's eating habits, suggest meals, provide recipes, and automatically cook meals. The learning unit can manage household priorities and deadlines, and perform laundry, cleaning, and other tasks at appropriate times. The learning unit can learn how to identify and deal with pests, and automatically perform extermination activities at appropriate times. The learning unit can monitor the growth status of plants and optimize watering, fertilization, light exposure time, and other tasks. The execution unit automates or remotely performs daily tasks based on the information learned by the learning unit. For example, the execution unit can enable a remote-controlled robot to iron clothes. The execution unit can enable a remote-controlled cooking robot to automatically cook based on a suggested recipe. The execution unit can suggest the best day to hang out laundry based on a weather forecast. The execution unit can adjust the cleaning timing depending on the room usage status. The execution unit can identify pests using a camera, and a remote-controlled extermination robot can perform extermination activities. The execution unit can monitor the moisture content and nutritional status of plants using sensors, and a remote-controlled planter can automatically add water and fertilizer. The providing unit provides the user with the results of the execution performed by the execution unit. For example, the providing unit can estimate the user's emotions and adjust the way the information is presented based on the estimated user's emotions. When providing the results of the execution performed by the execution unit, the providing unit can improve the accuracy of the information by reflecting the user's past feedback. The providing unit can customize the content of the information to be provided based on the user's lifestyle and family composition. When providing the results of the execution performed by the execution unit, the providing unit can provide the information in an optimal format based on the user's device or platform. As a result, the home automation system according to the embodiment can efficiently automate or remotely perform daily household tasks for the user and provide the results.

[0075] The learning unit learns how to handle specific laundry items, allowing the remote-controlled robot to iron them. The learning unit, for example, analyzes the material and wrinkle condition of the laundry and learns how to iron at the optimal temperature and pressure. For example, the learning unit can suggest ironing at a low temperature for cotton clothes. The learning unit can also suggest ironing at a low pressure for silk clothes. The learning unit can also suggest ironing at a high temperature and high pressure for heavily wrinkled clothes. In this way, the remote-controlled robot learns how to handle laundry and irons it, thereby making laundry management more efficient. Some or all of the above-mentioned processing in the learning unit may be performed using, for example, AI, or may be performed without using AI. For example, the learning unit can input the material and wrinkle condition of the laundry items into a generation AI and have the generation AI execute the optimal ironing method.

[0076] The learning unit can analyze the user's diet, suggest specific meals, provide recipes, and automatically cook. The learning unit can, for example, analyze the user's past diet history and health status and suggest nutritionally balanced meals. For example, if the user has previously eaten many high-calorie meals, the learning unit can suggest low-calorie, nutritionally balanced meals. Furthermore, if the user has previously eaten few vegetables, the learning unit can suggest meals that are high in vegetables. Furthermore, if the user has previously had a specific allergy, the learning unit can suggest meals that do not contain that allergen. This analysis of the user's diet, suggesting meals, providing recipes, and automatically cooking food improves the efficiency of dietary management. Some or all of the above-described processing by the learning unit may be performed using, or without, AI. For example, the learning unit can input the user's diet history and health status into the generation AI and cause the generation AI to suggest nutritionally balanced meals.

[0077] The learning unit manages the priorities and deadlines of housework, and can perform laundry, cleaning, etc. at specific times. The learning unit, for example, suggests the optimal day to hang out laundry based on the weather forecast. For example, the learning unit can suggest choosing a sunny day to hang out laundry. The learning unit also adjusts the timing of cleaning depending on the room usage. For example, the learning unit can suggest prioritizing cleaning of rooms that are used frequently. The learning unit can also suggest regularly cleaning rooms that are used less frequently. This improves the efficiency of housework by managing the priorities and deadlines of housework and performing laundry, cleaning, etc. at appropriate times. Some or all of the above-mentioned processing in the learning unit may be performed using, for example, AI, or may be performed without AI. For example, the learning unit can input weather forecast data and room usage data into the generation AI and have the generation AI execute an optimal housework schedule.

[0078] The learning unit learns how to identify and treat pests and can automatically perform extermination activities at specific times. For example, the learning unit identifies pests using a camera and learns how the remote-controlled extermination robot performs extermination activities. For example, the learning unit can identify mosquitoes and suggest using mosquito coils or electric mosquito repellents. The learning unit can also identify cockroaches and suggest using cockroach traps or insecticide sprays. The learning unit can also identify mites and suggest using mite removal sheets or vacuum cleaners. This allows the robot to learn how to identify and treat pests and automatically perform extermination activities at appropriate times, thereby streamlining pest control. Some or all of the above-described processing in the learning unit may be performed using, for example, AI, or without AI. For example, the learning unit can input image data of pests captured by a camera into the generation AI and have the generation AI execute the optimal extermination method.

[0079] The learning unit can monitor the plant's growth status and specifically optimize watering, fertilization, light exposure time, and the like. For example, the learning unit monitors the plant's moisture content and nutritional status using a sensor, and the remote-controlled planter learns how to automatically add water and fertilizer. For example, the learning unit can suggest automatically adding water when the plant's moisture content decreases. The learning unit can also suggest automatically adding fertilizer when the plant's nutritional status is insufficient. The learning unit can also suggest adjusting the light exposure time to optimize the plant's light exposure time. This allows the plant's growth status to be monitored and the optimization of watering, fertilization, light exposure time, and the like, resulting in more efficient plant care. Some or all of the above-described processing in the learning unit may be performed using, or without, AI. For example, the learning unit can input plant data acquired by a sensor into the generation AI and have the generation AI execute the optimal care method.

[0080] The execution unit enables the remote-controlled robot to specifically iron the laundry. For example, the execution unit enables the remote-controlled robot to analyze the material and wrinkle state of the laundry in real time and iron the laundry at the optimal temperature and pressure. For example, the execution unit can iron cotton clothes at a low temperature. The execution unit can also iron silk clothes at a low pressure. The execution unit can also iron heavily wrinkled clothes at a high temperature and high pressure. This allows the remote-controlled robot to iron the laundry, making ironing more efficient. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without using AI. For example, the execution unit can input the material and wrinkle state of the laundry into the generation AI and have the generation AI execute the optimal ironing method.

[0081] The execution unit enables the remote-controlled cooking robot to automatically cook based on a specific suggested recipe. For example, the execution unit allows the remote-controlled cooking robot to analyze a user's eating habits and automatically cook based on a suggested recipe. For example, the execution unit can automatically cook a nutritionally balanced meal taking into account the user's past eating history and health condition. The execution unit can also customize recipes according to the user's preferences and automatically cook the meal. The execution unit can also select an optimal cooking method taking into account the freshness and inventory status of ingredients and automatically cook the meal. This allows the remote-controlled cooking robot to automatically cook based on a suggested recipe, thereby improving cooking efficiency. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without using AI. For example, the execution unit can input the suggested recipe into a generation AI and have the generation AI execute the optimal cooking method.

[0082] The execution unit can suggest a specific day for hanging out laundry based on a weather forecast. The execution unit, for example, analyzes weather forecast data and suggests the optimal day for hanging out laundry. For example, the execution unit can suggest choosing a sunny day to hang out laundry. The execution unit can also suggest the optimal day for hanging out laundry by taking into account weather conditions such as humidity and wind speed. The execution unit can also adjust the hanging method depending on the amount and type of laundry. In this way, by suggesting the optimal day for hanging out laundry based on the weather forecast, laundry management becomes more efficient. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without using AI. For example, the execution unit can input weather forecast data into a generation AI and have the generation AI execute the optimal method for hanging out laundry.

[0083] The execution unit can adjust the specific cleaning timing depending on the room usage status. The execution unit, for example, analyzes the frequency of use and degree of dirtiness of the room and adjusts the cleaning timing. For example, the execution unit can suggest that frequently used rooms be cleaned first. The execution unit can also suggest that less frequently used rooms be cleaned regularly. The execution unit can also suggest a special cleaning method (e.g., steam cleaning) for heavily soiled rooms. This allows for cleaning to be made more efficient by adjusting the cleaning timing depending on the room usage status. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without using AI. For example, the execution unit can input room usage status data into the generation AI and have the generation AI execute an optimal cleaning schedule.

[0084] The execution unit can identify pests using a camera, and the remote-controlled extermination robot can carry out extermination activities. The execution unit can, for example, identify pests using a camera, and the remote-controlled extermination robot can carry out extermination activities. For example, the execution unit can identify mosquitoes and use mosquito coils or electric mosquito repellents. The execution unit can also identify cockroaches and use cockroach traps or insecticide sprays. The execution unit can also identify mites and use mite removal sheets or vacuum cleaners. In this way, pests are identified using a camera, and the remote-controlled extermination robot carries out extermination activities, thereby making pest extermination more efficient. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without using AI. For example, the execution unit can input image data of pests acquired by a camera into a generation AI, and have the generation AI execute an optimal extermination method.

[0085] The execution unit monitors the water content and nutritional status of the plant using sensors, and the remote-controlled planter can automatically add water and fertilizer. The execution unit, for example, monitors the water content and nutritional status of the plant using sensors, and the remote-controlled planter can automatically add water and fertilizer. For example, the execution unit can automatically add water when the water content of the plant decreases. The execution unit can also automatically add fertilizer when the plant's nutritional status is insufficient. The execution unit can also adjust the care method according to the plant's growth stage and season. This makes plant management more efficient by monitoring the water content and nutritional status of the plant using sensors and the remote-controlled planter automatically adding water and fertilizer. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without AI. For example, the execution unit can input plant data obtained by sensors into the generation AI and have the generation AI execute the optimal care method.

[0086] The providing unit can estimate the user's emotions and adjust the way information is presented based on the estimated user emotions. The providing unit, for example, analyzes the user's facial expressions and voice to estimate emotions. For example, if the user is nervous, the providing unit can provide simple, highly visible information. Furthermore, if the user is relaxed, the providing unit can provide detailed information. Furthermore, if the user is in a hurry, the providing unit can provide information that focuses on the main points. This allows optimal information to be provided to the user by adjusting the way information is presented based on the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the providing unit may be performed using, for example, an AI. For example, the providing unit may input the user's facial expressions and voice data into the generation AI and cause the generation AI to estimate emotions.

[0087] When providing the results executed by the execution unit, the providing unit can improve the accuracy of the information by reflecting the user's past feedback. The providing unit, for example, collects the user's past feedback and improves the accuracy of the information. For example, the providing unit can preferentially provide information formats for which the user has given positive feedback in the past. The providing unit can also avoid information formats for which the user has given negative feedback in the past. The providing unit can also analyze the user's past feedback and provide the optimal information format. This improves the accuracy of the information by reflecting the user's past feedback. Some or all of the above-mentioned processing in the providing unit may be performed, for example, using AI or without AI. For example, the providing unit can input the user's feedback data into the generation AI and cause the generation AI to improve the accuracy of the information.

[0088] The providing unit can customize the content of the information to be provided according to the user's lifestyle and family composition. The providing unit, for example, analyzes the user's lifestyle and family composition and customizes the content of the information. For example, if the user is active early in the morning, the providing unit can prioritize providing information related to morning hours. Furthermore, if the user is a nocturnal person, the providing unit can prioritize providing information related to evening hours. Furthermore, the providing unit can provide information useful to all family members according to the user's family composition. This enables more appropriate information to be provided by customizing the information according to the user's lifestyle and family composition. Some or all of the above-described processing by the providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the providing unit can input the user's lifestyle and family composition data into the generation AI and cause the generation AI to customize the information.

[0089] When providing the results of the execution by the execution unit, the providing unit can provide information in an optimal format depending on the user's device and platform. The providing unit, for example, analyzes the user's device and platform and provides the information in the optimal format. For example, if the user is using a smartphone, the providing unit can provide information in a format optimized for mobile devices. Furthermore, if the user is using a tablet, the providing unit can provide information in a format optimized for a large screen. Furthermore, if the user is using a personal computer, the providing unit can provide information in a format optimized for desktops. This improves the visibility of the information by providing information depending on the user's device and platform. Some or all of the above-described processing by the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input data of the user's device and platform into the generation AI and have the generation AI execute the optimal information format.

[0090] The providing unit can estimate the user's emotions and determine the priority of information to be provided based on the estimated user emotions. The providing unit, for example, analyzes the user's facial expressions and voice to estimate emotions. For example, if the user is feeling stressed, the providing unit can prioritize providing relaxing information. Furthermore, if the user is in a hurry, the providing unit can prioritize providing important information. Furthermore, if the user is relaxed, the providing unit can provide detailed information. This enables optimal information to be provided to the user by determining the priority of information based on the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the providing unit may be performed using, for example, an AI. For example, the providing unit may input the user's facial expressions and voice data into the generation AI and cause the generation AI to estimate emotions.

[0091] When providing the results of the execution by the execution unit, the providing unit can prioritize providing highly relevant information by taking into account the user's geographical location information. The providing unit, for example, analyzes the user's geographical location information and provides highly relevant information. For example, when the user is in a specific area, the providing unit can provide information related to that area. Furthermore, when the user is traveling, the providing unit can provide information related to the travel destination. Furthermore, when the user is at home, the providing unit can provide information related to the area around the user's home. This makes it possible to provide highly relevant information by taking the user's geographical location information into consideration. Some or all of the above-described processing by the providing unit may be performed using AI, for example, or may be performed without using AI. For example, the providing unit can input the user's geographical location information to a generation AI and cause the generation AI to execute highly relevant information.

[0092] The providing unit can customize the content of the information to be provided by analyzing the user's social media activities and online behavior. The providing unit, for example, analyzes the user's social media activities and online behavior to customize the content of the information. For example, the providing unit can provide information related to topics in which the user has shown interest on social media. The providing unit can also analyze the user's online behavior and provide related information. The providing unit can also provide related information by referring to the activities of the user's friends on social media. This makes it possible to provide more relevant information by analyzing the user's social media activities and online behavior. Some or all of the above-described processing by the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input the user's social media activities and online behavior data into a generation AI and cause the generation AI to customize the information.

[0093] When providing the results executed by the execution unit, the providing unit can customize the format of the information by reflecting the user's past feedback. The providing unit, for example, collects the user's past feedback and customizes the format of the information. For example, the providing unit can preferentially provide information formats for which the user has given positive feedback in the past. The providing unit can also avoid information formats for which the user has given negative feedback in the past. The providing unit can also analyze the user's past feedback and provide the optimal information format. In this way, the information format is optimized for the user by reflecting the user's past feedback. Some or all of the above-described processing in the providing unit may be performed, for example, using AI or without AI. For example, the providing unit can input the user's feedback data to a generation AI and cause the generation AI to customize the information format.

[0094] The learning unit can estimate the user's emotions and adjust learning priorities based on the estimated user emotions. The learning unit, for example, analyzes the user's facial expressions and voice to estimate emotions. For example, when the user is feeling stressed, the learning unit can prioritize learning relaxing chores (e.g., watering plants). When the user is in a hurry, the learning unit can postpone time-consuming chores (e.g., ironing) and prioritize quick chores. When the user is relaxed, the learning unit can proceed with learning according to a normal chore schedule. This allows for more effective learning by adjusting learning priorities based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generative AI. The generative AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the learning unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the learning unit can input the user's facial expressions and voice data into the generation AI and have the generation AI estimate emotions.

[0095] When learning how to handle laundry, the learning unit can suggest an optimal washing method based on the type of material and the degree of dirt. The learning unit, for example, analyzes the material and degree of dirt of the laundry and suggests an optimal washing method. For example, the learning unit can suggest washing at low temperature and using fabric softener for cotton clothes. The learning unit can also suggest hand washing and using special detergent for silk clothes. The learning unit can also suggest using a detergent specially designed for oil stains in advance for heavily oil-stained clothes. This improves laundry efficiency by suggesting an optimal washing method based on the type of material and the degree of dirt. Some or all of the above-described processing in the learning unit may be performed using, for example, AI, or may be performed without AI. For example, the learning unit can input the material and degree of dirt of the laundry into the generation AI and have the generation AI execute the optimal washing method.

[0096] When analyzing the user's diet, the learning unit can suggest nutritionally balanced meals taking into account the user's past dietary history and health condition. The learning unit, for example, analyzes the user's past dietary history and health condition and suggests nutritionally balanced meals. For example, if the user has previously eaten many high-calorie meals, the learning unit can suggest low-calorie, nutritionally balanced meals. Furthermore, if the user has previously eaten few vegetables, the learning unit can suggest meals that are high in vegetables. Furthermore, if the user has previously had a specific allergy, the learning unit can suggest meals that do not contain the allergen. This improves the user's health management by suggesting nutritionally balanced meals taking into account the user's past dietary history and health condition. Some or all of the above-described processing by the learning unit may be performed using, for example, AI, or may be performed without AI. For example, the learning unit can input the user's dietary history and health condition into the generation AI and cause the generation AI to suggest nutritionally balanced meals.

[0097] The learning unit can suggest the optimal timing for managing housework priorities and deadlines, taking into account the schedules of all family members. The learning unit, for example, analyzes the schedules of all family members and suggests the optimal timing for housework. For example, the learning unit can suggest cleaning during a time when all family members are out. The learning unit can also suggest preparing meals during a time when all family members are at home. The learning unit can also suggest doing laundry during a time when all family members are asleep. This improves the efficiency of housework by suggesting the optimal timing in consideration of the schedules of all family members. Some or all of the above-mentioned processing in the learning unit may be performed using, for example, AI, or may be performed without using AI. For example, the learning unit can input schedule data of all family members into the generation AI and have the generation AI execute the optimal housework schedule.

[0098] The learning unit can estimate the user's emotions and select the type of housework to learn based on the estimated user emotions. The learning unit, for example, analyzes the user's facial expressions and voice to estimate emotions. For example, when the user is feeling stressed, the learning unit can prioritize learning relaxing housework (e.g., watering plants). When the user is in a hurry, the learning unit can postpone time-consuming housework (e.g., ironing) and prioritize housework that can be completed in a short time. When the user is relaxed, the learning unit can proceed with learning according to the user's normal housework schedule. This enables more effective learning by selecting the type of housework to learn based on the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the learning unit may be performed using, for example, an AI, or without an AI. For example, the learning unit can input the user's facial expressions and voice data into the generation AI and have the generation AI estimate emotions.

[0099] When learning how to identify and deal with pests, the learning unit can propose optimal extermination methods according to the local climate and season. The learning unit, for example, analyzes the local climate and season and proposes optimal extermination methods. For example, the learning unit can propose mosquito extermination methods in the summer. The learning unit can also propose cockroach extermination methods in the winter. The learning unit can also propose extermination methods for pests that prefer humidity in the rainy season. This makes pest extermination more efficient by proposing optimal extermination methods according to the local climate and season. Some or all of the above-mentioned processing in the learning unit may be performed using, or without, AI, for example. For example, the learning unit can input local climate and seasonal data into the generation AI and have the generation AI execute the optimal extermination method.

[0100] When inspecting the growth status of a plant, the learning unit can suggest an optimal care method according to the type and growth stage of the plant. The learning unit, for example, analyzes the type and growth stage of the plant and suggests the optimal care method. For example, the learning unit can suggest fertilizer to promote growth for young plants. The learning unit can also suggest watering frequency for maintenance of mature plants. The learning unit can also suggest appropriate treatment methods for diseased plants. In this way, by suggesting the optimal care method according to the type and growth stage of the plant, plant management becomes more efficient. Some or all of the above-described processing in the learning unit may be performed using, for example, AI, or may be performed without using AI. For example, the learning unit can input plant type and growth stage data into the generation AI and have the generation AI execute the optimal care method.

[0101] The learning unit can learn the user's lifestyle rhythm and propose an optimal housework schedule. The learning unit, for example, analyzes the user's lifestyle rhythm and proposes an optimal housework schedule. For example, if the user is active early in the morning, the learning unit can propose a schedule that concentrates housework in the morning hours. Furthermore, if the user is a nocturnal person, the learning unit can propose a schedule that concentrates housework in the evening hours. Furthermore, if the user has an irregular lifestyle, the learning unit can propose a flexible housework schedule. In this way, by learning the user's lifestyle rhythm and proposing an optimal housework schedule, housework efficiency is improved. Some or all of the above-mentioned processing in the learning unit may be performed using, for example, AI, or may be performed without using AI. For example, the learning unit can input the user's lifestyle rhythm data into a generation AI and cause the generation AI to execute an optimal housework schedule.

[0102] The execution unit can estimate the user's emotions and adjust the order of housework to be performed based on the estimated user's emotions. The execution unit, for example, analyzes the user's facial expressions and voice to estimate emotions. For example, when the user is feeling stressed, the execution unit can prioritize relaxing housework (e.g., watering plants). When the user is in a hurry, the execution unit can postpone time-consuming housework (e.g., ironing) and prioritize housework that can be completed in a short time. When the user is relaxed, the execution unit can proceed with the execution according to the user's regular housework schedule. This allows for more efficient housework performance by adjusting the order of housework based on the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the execution unit may be performed using, for example, an AI, or without an AI. For example, the execution unit can input the user's facial expressions and voice data into the generation AI and have the generation AI estimate emotions.

[0103] When the remote-controlled robot irons, the execution unit can analyze the material and wrinkle state of the laundry in real time and iron it at the optimal temperature and pressure. For example, the execution unit can cause the remote-controlled robot to analyze the material and wrinkle state of the laundry in real time and iron it at the optimal temperature and pressure. For example, the execution unit can iron cotton clothes at a low temperature. The execution unit can also iron silk clothes at a low pressure. The execution unit can also iron heavily wrinkled clothes at a high temperature and high pressure. This improves ironing efficiency by analyzing the material and wrinkle state of the laundry in real time and ironing it at the optimal temperature and pressure. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without AI. For example, the execution unit can input the material and wrinkle state of the laundry into a generation AI and have the generation AI execute the optimal ironing method.

[0104] When the remote-controlled cooking robot performs automatic cooking, the execution unit can adjust the cooking method taking into account the freshness and stock status of ingredients. For example, the execution unit allows the remote-controlled cooking robot to analyze the freshness and stock status of ingredients, select the optimal cooking method, and perform automatic cooking. For example, the execution unit can select a simple cooking method when using highly fresh ingredients. Furthermore, the execution unit can extend the cooking time when using less fresh ingredients to cook safely. Furthermore, the execution unit can combine ingredients that are in low stock with other ingredients when using ingredients that are in low stock. This improves cooking efficiency by adjusting the cooking method taking into account the freshness and stock status of ingredients. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without AI. For example, the execution unit can input ingredient freshness and stock status data into the generation AI and have the generation AI execute the optimal cooking method.

[0105] When suggesting the optimal day to hang out laundry based on the weather forecast, the execution unit can adjust the hanging method depending on the amount and type of laundry. The execution unit, for example, analyzes weather forecast data and suggests the optimal day to hang out laundry. For example, the execution unit can suggest choosing a sunny day to hang out laundry. The execution unit can also suggest the optimal day to hang out laundry by taking into account weather conditions such as humidity and wind speed. The execution unit can also adjust the hanging method depending on the amount and type of laundry. For example, if there is a large amount of laundry, the execution unit can suggest hanging it in a well-ventilated place. If there is laundry made of delicate materials, the execution unit can suggest hanging it in the shade. If there is laundry made of materials that are difficult to dry, the execution unit can suggest using a dryer in addition. In this way, laundry management is made more efficient by adjusting the hanging method depending on the amount and type of laundry. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without using AI. For example, the execution unit can input weather forecast data and data on the amount and type of laundry into the generation AI, and have the generation AI execute the optimal drying method.

[0106] The execution unit can estimate the user's emotions and adjust the frequency of housework to be performed based on the estimated user emotions. The execution unit, for example, analyzes the user's facial expressions and voice to estimate emotions. For example, if the user is feeling stressed, the execution unit can reduce the frequency of housework to increase the time the user can relax. Furthermore, if the user is in a hurry, the execution unit can frequently perform housework that can be completed in a short time. Furthermore, if the user is relaxed, the execution unit can proceed with the execution according to the user's regular housework schedule. This allows for more effective housework performance by adjusting the frequency of housework based on the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the execution unit may be performed using, for example, an AI. For example, the execution unit can input the user's facial expressions and voice data into the generation AI and have the generation AI perform emotion estimation.

[0107] When adjusting the cleaning timing according to the room usage status, the execution unit can select a cleaning method taking into account the degree of dirtiness and frequency of use of the room. The execution unit, for example, analyzes the frequency of use and degree of dirtiness of the room and adjusts the cleaning timing. For example, the execution unit can suggest cleaning a frequently used room every day. The execution unit can also suggest cleaning a less frequently used room once a week. The execution unit can also suggest a special cleaning method (e.g., steam cleaning) for a heavily soiled room. This improves cleaning efficiency by adjusting the cleaning timing according to the room usage status. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without using AI. For example, the execution unit can input room usage status data into a generation AI and have the generation AI execute an optimal cleaning schedule.

[0108] The execution unit can identify pests using a camera, and when the remote-controlled extermination robot performs extermination activities, select the optimal extermination method depending on the type and number of pests. The execution unit, for example, identifies pests using a camera, and the remote-controlled extermination robot performs extermination activities. For example, if there are many mosquitoes, the execution unit can use mosquito coils or an electric mosquito repellent. If there are many cockroaches, the execution unit can use cockroach traps or insecticide spray. If there are many mites, the execution unit can use mite removal sheets or a vacuum cleaner. This improves the efficiency of pest extermination by selecting the optimal extermination method depending on the type and number of pests. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without AI. For example, the execution unit can input image data of pests captured by a camera into the generation AI and have the generation AI execute the optimal extermination method.

[0109] The execution unit monitors the water content and nutrient status of the plant using sensors, and when the remote-controlled planter automatically adds water or fertilizer, it can adjust the care method according to the plant's growth stage and season. For example, the execution unit monitors the water content and nutrient status of the plant using sensors, and the remote-controlled planter automatically adds water or fertilizer. For example, the execution unit can frequently add water or fertilizer to plants in the growing stage. The execution unit can also refrain from adding water or fertilizer to plants in the dormant stage. The execution unit can also suggest different care methods for each season (e.g., moving the plant indoors in winter). This allows for more efficient plant management by adjusting the care method according to the plant's growth stage and season. Some or all of the above-mentioned processing in the execution unit may be performed using, for example, AI, or may be performed without AI. For example, the execution unit can input plant data obtained by sensors into a generation AI and have the generation AI execute an optimal care method.

[0110] The providing unit can estimate the user's emotions and adjust the way information is presented based on the estimated user emotions. The providing unit, for example, analyzes the user's facial expressions and voice to estimate emotions. For example, if the user is nervous, the providing unit can provide simple, highly visible information. Furthermore, if the user is relaxed, the providing unit can provide detailed information. Furthermore, if the user is in a hurry, the providing unit can provide information that focuses on the main points. This allows optimal information to be provided to the user by adjusting the way information is presented based on the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the providing unit may be performed using, for example, an AI. For example, the providing unit may input the user's facial expressions and voice data into the generation AI and cause the generation AI to estimate emotions.

[0111] When providing the results executed by the execution unit, the providing unit can improve the accuracy of the information by reflecting the user's past feedback. The providing unit, for example, collects the user's past feedback and improves the accuracy of the information. For example, the providing unit can preferentially provide information formats for which the user has given positive feedback in the past. The providing unit can also avoid information formats for which the user has given negative feedback in the past. The providing unit can also analyze the user's past feedback and provide the optimal information format. This improves the accuracy of the information by reflecting the user's past feedback. Some or all of the above-mentioned processing in the providing unit may be performed, for example, using AI or without AI. For example, the providing unit can input the user's feedback data into the generation AI and cause the generation AI to improve the accuracy of the information.

[0112] The providing unit can customize the content of the information to be provided according to the user's lifestyle and family composition. The providing unit, for example, analyzes the user's lifestyle and family composition and customizes the content of the information. For example, if the user is active early in the morning, the providing unit can prioritize providing information related to morning hours. Furthermore, if the user is a nocturnal person, the providing unit can prioritize providing information related to evening hours. Furthermore, the providing unit can provide information useful to all family members according to the user's family composition. This enables more appropriate information to be provided by customizing the information according to the user's lifestyle and family composition. Some or all of the above-described processing by the providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the providing unit can input the user's lifestyle and family composition data into the generation AI and cause the generation AI to customize the information.

[0113] When providing the results of the execution by the execution unit, the providing unit can provide information in an optimal format depending on the user's device and platform. The providing unit, for example, analyzes the user's device and platform and provides the information in the optimal format. For example, if the user is using a smartphone, the providing unit can provide information in a format optimized for mobile devices. Furthermore, if the user is using a tablet, the providing unit can provide information in a format optimized for a large screen. Furthermore, if the user is using a personal computer, the providing unit can provide information in a format optimized for desktops. This improves the visibility of the information by providing information depending on the user's device and platform. Some or all of the above-described processing by the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input data of the user's device and platform into the generation AI and have the generation AI execute the optimal information format.

[0114] The providing unit can estimate the user's emotions and determine the priority of information to be provided based on the estimated user emotions. The providing unit, for example, analyzes the user's facial expressions and voice to estimate emotions. For example, if the user is feeling stressed, the providing unit can prioritize providing relaxing information. Furthermore, if the user is in a hurry, the providing unit can prioritize providing important information. Furthermore, if the user is relaxed, the providing unit can provide detailed information. This enables optimal information to be provided to the user by determining the priority of information based on the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the providing unit may be performed using, for example, an AI. For example, the providing unit may input the user's facial expressions and voice data into the generation AI and cause the generation AI to estimate emotions.

[0115] When providing the results of the execution by the execution unit, the providing unit can prioritize providing highly relevant information by taking into account the user's geographical location information. The providing unit, for example, analyzes the user's geographical location information and provides highly relevant information. For example, when the user is in a specific area, the providing unit can provide information related to that area. Furthermore, when the user is traveling, the providing unit can provide information related to the travel destination. Furthermore, when the user is at home, the providing unit can provide information related to the area around the user's home. This makes it possible to provide highly relevant information by taking the user's geographical location information into consideration. Some or all of the above-described processing by the providing unit may be performed using AI, for example, or may be performed without using AI. For example, the providing unit can input the user's geographical location information to a generation AI and cause the generation AI to execute highly relevant information.

[0116] The providing unit can customize the content of the information to be provided by analyzing the user's social media activities and online behavior. The providing unit, for example, analyzes the user's social media activities and online behavior to customize the content of the information. For example, the providing unit can provide information related to topics in which the user has shown interest on social media. The providing unit can also analyze the user's online behavior and provide related information. The providing unit can also provide related information by referring to the activities of the user's friends on social media. This makes it possible to provide more relevant information by analyzing the user's social media activities and online behavior. Some or all of the above-described processing by the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input the user's social media activities and online behavior data into a generation AI and cause the generation AI to customize the information.

[0117] When providing the results executed by the execution unit, the providing unit can customize the format of the information by reflecting the user's past feedback. The providing unit, for example, collects the user's past feedback and customizes the format of the information. For example, the providing unit can preferentially provide information formats for which the user has given positive feedback in the past. The providing unit can also avoid information formats for which the user has given negative feedback in the past. The providing unit can also analyze the user's past feedback and provide the optimal information format. In this way, the information format is optimized for the user by reflecting the user's past feedback. Some or all of the above-described processing in the providing unit may be performed, for example, using AI or without AI. For example, the providing unit can input the user's feedback data to a generation AI and cause the generation AI to customize the information format. === Hard Collateral 1-1 === Each of the multiple elements, including the learning unit, execution unit, and provision unit, described above, is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the learning unit learns the user's daily household tasks using the camera 42 and microphone 38B of the smart device 14, and processes the learning results using the control unit 46A. The execution unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and automates or remotely executes the daily tasks based on the learned information. The provision unit provides the execution results to the user, for example, using the output device 40 of the smart device 14. === Hard Collateral 1-2 === Each of the multiple elements, including the learning unit, execution unit, and provision unit, described above, is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the learning unit learns the user's daily household tasks using the camera 42 and microphone 238 of the smart glasses 214, and processes the learning results using the control unit 46A. The execution unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and automates or remotely executes the daily tasks based on the learned information. The provision unit provides the execution results to the user using, for example, the speaker 240 of the smart glasses 214. === Hard Collateral 1-3 === Each of the multiple elements including the learning unit, execution unit, and provision unit described above is realized, for example, by at least one of the headset type terminal 314 and the data processing device 12. For example, the learning unit learns the user's daily household tasks using the camera 42 and microphone 238 of the headset type terminal 314, and processes the learning results using the control unit 46A. The execution unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and automates or remotely executes the daily tasks based on the learned information. The provision unit provides the execution results to the user using, for example, the display 343 of the headset type terminal 314. === Hard Collateral 1-4 === Each of the multiple elements including the learning unit, execution unit, and provision unit described above is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the learning unit learns the user's daily household tasks using the camera 42 and microphone 238 of the robot 414, and processes the learning results using the control unit 46A. The execution unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and automates or remotely executes the daily tasks based on the learned information. The provision unit provides the execution results to the user using, for example, the speaker 240 of the robot 414.

[0118] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0119] The home automation system may further include an energy management unit. The energy management unit may monitor power consumption in the home and suggest efficient energy usage. For example, the energy management unit may adjust the usage schedule of home appliances to avoid peak power consumption times. The energy management unit may also work with a solar power generation system to optimize power usage according to the amount of power generated. The energy management unit may also analyze power consumption data and recommend the use of energy-efficient home appliances. This may improve the efficiency of energy consumption in the home and contribute to reducing power costs.

[0120] The home automation system may further include a health management unit. The health management unit may monitor the user's health condition and make suggestions for maintaining good health. For example, the health management unit may analyze the user's sleep patterns and suggest an optimal sleeping environment. The health management unit may also monitor the user's exercise volume and suggest an appropriate exercise plan. The health management unit may also analyze the user's diet and suggest nutritionally balanced meals. This may improve the efficiency of the user's health management and help maintain good health.

[0121] The home automation system can further include a security management unit. The security management unit can monitor home security and issue an alert if it detects an abnormality. For example, the security management unit can monitor the opening and closing of doors and windows and issue an alert if it detects suspicious movement. The security management unit can also analyze camera footage and issue an alert if it identifies a suspicious person. The security management unit can also issue an alert if it detects an abnormality such as a fire or gas leak. This strengthens home security, allowing people to live with peace of mind.

[0122] The home automation system may further include a communication unit. The communication unit supports communication between users and facilitates information sharing within the home. For example, the communication unit may support sending and receiving messages between family members. The communication unit may also share and coordinate family schedules. The communication unit may also share important information within the home (e.g., shopping lists and progress of household chores). This facilitates communication within the home and makes information sharing more efficient.

[0123] The home automation system may further include an entertainment unit. The entertainment unit can support the user's entertainment and provide a relaxing environment. For example, the entertainment unit can suggest music and movies according to the user's preferences. The entertainment unit can also integrate and remotely control audiovisual devices in the home. The entertainment unit can also estimate the user's emotions and provide relaxing content. This enhances the user's entertainment and provides a relaxing environment.

[0124] The home automation system may further include an education support unit. The education support unit may support the user's learning and provide an effective learning environment. For example, the education support unit may monitor the user's learning progress and propose an appropriate learning plan. The education support unit may also provide learning content based on the user's interests and concerns. The education support unit may also estimate the user's emotions and provide support to increase the user's motivation to learn. This may improve the efficiency of the user's learning and provide an effective learning environment.

[0125] The home automation system may further include a pet care unit. The pet care unit can manage the health and comfort of pets in the home. For example, the pet care unit can automate the supply of food and water for pets. The pet care unit can also monitor the amount of exercise the pet receives and suggest an appropriate exercise plan. The pet care unit can also estimate the emotions of the pet and provide an environment to reduce stress. This improves the health and comfort of pets and reduces the burden on owners.

[0126] The home automation system may further include a remote help unit. The remote help unit can provide remote support when a user has a problem at home. For example, the remote help unit can remotely troubleshoot home appliances. The remote help unit can also provide advice on home repairs and maintenance. The remote help unit can also estimate the user's emotions and provide appropriate support. This allows the user to receive prompt support when they have a problem at home.

[0127] The home automation system may further include an emergency response unit. The emergency response unit can respond quickly to emergencies in the home. For example, the emergency response unit can automatically notify emergency contacts when it detects a fire or gas leak. The emergency response unit can also detect falls in elderly or physically disabled people and quickly call for help. The emergency response unit can also estimate the user's emotions and provide support to reduce stress during an emergency. This allows for a quick and appropriate response to emergencies in the home.

[0128] The home automation system may further include an environment monitoring unit. The environment monitoring unit may monitor the environmental conditions in the home and make suggestions to maintain a comfortable environment. For example, the environment monitoring unit may monitor the indoor temperature and humidity and suggest optimal air conditioning settings. The environment monitoring unit may also monitor air quality and suggest the use of an air purifier as needed. The environment monitoring unit may also adjust the brightness and color temperature of lighting to provide a comfortable lighting environment. This optimizes the home environment and provides a comfortable living space.

[0129] The processing flow of the second embodiment will be briefly explained below.

[0130] Step 1: The learning module learns the user's daily household tasks, such as how to handle laundry, analyze diet, manage household priorities and deadlines, identify and deal with pests, and monitor plant growth. Step 2: The execution unit automates or remotely executes routine tasks based on the information learned by the learning unit. For example, ironing with a remote-controlled robot, automatic cooking with a cooking robot, laundry drying suggestions based on the weather forecast, adjusting cleaning schedules based on room usage, identifying pests with cameras and exterminating them with an extermination robot, monitoring plant moisture and nutrient status with sensors and automatically adding water and fertilizer with a planter, etc. Step 3: The providing unit provides the results of the execution by the execution unit to the user. For example, it estimates the user's emotions and adjusts the way information is presented based on the estimated emotions, improves the accuracy of the information by reflecting past feedback, customizes the information according to the user's lifestyle and family structure, and provides information in the optimal format according to the user's device and platform.

[0131] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0132] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AIs include the data generation model 58, such as a neural network model (e.g., a neural network model), and a neural network model (e.g., a neural network model). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input to the data generation model 58. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0133] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0134] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0135] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0136] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0137] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0138] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0139] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0140] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0141] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0142] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0143] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0144] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0145] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0146] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0147] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0148] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AI other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0149] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0150] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0151] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0152] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0153] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0154] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0155] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0156] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0157] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0158] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0159] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0160] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0161] In the headset type terminal 314, the identification process is performed by the processor 46. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0162] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0163] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0164] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AI other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0165] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0166] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0167] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0168] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0169] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0170] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0171] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0172] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0173] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0174] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0175] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0176] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0177] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0178] In the robot 414, the processor 46 performs the identification process. The storage 50 stores the identification program 60. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as the control unit 46A in accordance with the identification program 60 executed on the RAM 48. The robot 414 also has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform the same process as the identification processing unit 290 using these models.

[0179] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0180] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0181] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AI other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0182] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0183] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0184] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0185] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0186] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0187] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0188] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[0189] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0190] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0191] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0192] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0193] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0194] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0195] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0196] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0197] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0198] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0199] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0200] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0201] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0202] [Explanation of symbols]

[0203] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. a learning unit that learns specific daily tasks performed at home by the user; an execution unit that automates or remotely executes specific routine tasks based on the information learned by the learning unit; a providing unit that provides a result executed by the executing unit to a user. A system characterized by:

2. The learning unit Remote-controlled robot learns how to handle specific laundry items and irons them The system of claim 1 .

3. The learning unit Analyzes the user's eating habits, suggests specific meals, provides recipes, and automatically cooks meals The system of claim 1 .

4. The learning unit Manage household priorities and deadlines, and do laundry, cleaning, etc. at specific times The system of claim 1 .

5. The learning unit Learn to identify and control pests, and then automatically carry out pest control activities at specific times The system of claim 1 .

6. The learning unit Audit the plant's growth status and specifically optimize watering, fertilizer addition, light exposure time, etc. The system of claim 1 .

7. The execution unit: Remote-controlled robot irons concrete The system of claim 1 .

8. The execution unit: Remotely controlled cooking robots will automatically cook food based on specific recipe suggestions. The system of claim 1 .

Images