system

Abstract

The system according to this embodiment aims to centrally manage information on home appliances and efficiently perform troubleshooting, warranty period management, and replacement suggestions. [Solution] The system according to the embodiment comprises a collection unit, a management unit, a troubleshooting unit, a warranty management unit, a proposal unit, a maintenance unit, and a parts proposal unit. The collection unit collects information on home appliances. The management unit digitally manages the information collected by the collection unit. The troubleshooting unit performs troubleshooting based on the information managed by the management unit. The warranty management unit manages the warranty period based on the information managed by the management unit. The proposal unit makes replacement proposals based on the information managed by the management unit. The maintenance unit manages the maintenance schedule based on the information managed by the management unit. The parts proposal unit proposes and provides purchase support for spare parts based on the information managed by the management unit.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, there was a problem that it was difficult to efficiently perform troubleshooting of household appliances, management of warranty periods, replacement purchase proposals, etc.

[0005] The system according to the embodiment aims to centrally manage information on household appliances and efficiently perform troubleshooting, management of warranty periods, replacement purchase proposals, etc.

Means for Solving the Problems

[0006] The system according to this embodiment comprises a data collection unit, a management unit, a troubleshooting unit, a warranty management unit, a proposal unit, a maintenance unit, and a parts proposal unit. The data collection unit collects information on home appliances. The management unit digitally manages the information collected by the data collection unit. The troubleshooting unit performs troubleshooting based on the information managed by the management unit. The warranty management unit manages the warranty period based on the information managed by the management unit. The proposal unit makes replacement suggestions based on the information managed by the management unit. The maintenance unit manages the maintenance schedule based on the information managed by the management unit. The parts proposal unit proposes and provides purchase support for spare parts based on the information managed by the management unit. [Effects of the Invention]

[0007] The system according to this embodiment can centrally manage information on home appliances and efficiently perform troubleshooting, warranty period management, and replacement suggestions. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8]This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the tagged storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the tagged communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. Also, the reception device 38, the output device 40, and the camera 42 are connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by contact of an indicator (e.g., a pen or a finger, etc.) by detecting the contact of the indicator. The microphone 38B receives user input by voice by detecting the voice of the user. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, a specific processing unit 290 (see FIG. 2) acquires data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The Smart Home Appliance Concierge, according to an embodiment of the present invention, is an AI application for facilitating the management of home appliances. The Smart Home Appliance Concierge alleviates anxieties and concerns regarding home appliance troubles and management. For example, when a user purchases a home appliance, they can take a picture of the instruction manual cover with their smartphone camera, which automatically collects and registers information about the appliance. This information includes the appliance's model name, manufacturer, purchase date, and warranty period. Next, based on the registered appliance information, the AI ​​digitally manages the instruction manuals, making them readily accessible when needed. Furthermore, the AI ​​supports troubleshooting of home appliances. For example, if an appliance malfunctions, the user can describe the problem by voice, and the AI ​​will search for appropriate solutions from the instruction manual or the manufacturer's Q&A site and provide voice guidance. It also manages the warranty period and sends notifications before the warranty expires to prompt necessary action. In addition, the AI ​​analyzes the usage status and market information of home appliances to suggest replacement timings and recommend new products. Furthermore, it automatically creates a home appliance maintenance schedule and notifies users of regular maintenance times. For example, it supports often-forgotten maintenance such as cleaning air conditioner filters or refrigerator drain pipes. Finally, the AI ​​also provides support for suggesting and purchasing spare parts. It analyzes the amount and usage of spare parts stored for home appliances and suggests the necessary spare parts. It also provides support such as finding suppliers for spare parts and comparing prices. In this way, the smart home appliance concierge centralizes the management of home appliances and supports troubleshooting, maintenance, and replacement suggestions, making users' lives more convenient and comfortable.

[0029] The smart home appliance concierge according to this embodiment comprises a collection unit, a management unit, a troubleshooting unit, a warranty management unit, a proposal unit, a maintenance unit, and a parts proposal unit. The collection unit collects information about home appliances. For example, the collection unit can collect information about home appliances by taking a picture of the cover of the instruction manual with a smartphone camera. The collection unit can collect information such as the model name, manufacturer, purchase date, and warranty period of the home appliance. The collection unit can also automatically collect information about home appliances using AI. The management unit digitally manages the information collected by the collection unit. For example, the management unit can digitize instruction manuals so that they can be accessed immediately when needed. For example, the management unit can securely store information about home appliances using cloud storage. The management unit can also efficiently manage information about home appliances using AI. The troubleshooting unit performs troubleshooting based on the information managed by the management unit. For example, when a user describes the problem by voice, the troubleshooting unit can search for appropriate solutions from the instruction manual or the manufacturer's Q&A site and provide voice guidance. The troubleshooting department can, for example, use AI to identify the cause of a problem and suggest appropriate solutions. The troubleshooting department can also use AI to automatically guide users through troubleshooting. The warranty management department manages warranty periods based on information managed by the management department. The warranty management department can, for example, send notifications before the warranty period expires. The warranty management department can, for example, use AI to efficiently manage warranty periods. Furthermore, the warranty management department can use AI to automatically extend or renew warranties. The recommendation department makes replacement suggestions based on information managed by the management department. The recommendation department can, for example, analyze appliance usage and market information to suggest replacement timing and recommend new products. The recommendation department can, for example, use AI to provide users with optimal replacement suggestions. Furthermore, the recommendation department can use AI to automatically notify users of when it's time to replace their appliances. The maintenance department manages maintenance schedules based on information managed by the management department.The maintenance department can, for example, automatically create maintenance schedules for home appliances and notify users of the timing of regular maintenance. The maintenance department can, for example, use AI to efficiently perform maintenance on home appliances. The maintenance department can also use AI to manage the maintenance history of home appliances. The parts proposal department provides support for proposing and purchasing spare parts based on information managed by the management department. The parts proposal department can, for example, analyze the amount of spare parts stored and their usage status for home appliances and propose the necessary spare parts. The parts proposal department can, for example, use AI to provide support such as finding suppliers for spare parts and comparing prices. The parts proposal department can also use AI to automatically manage the inventory of spare parts. As a result, the smart home appliance concierge according to this embodiment can centrally perform information gathering, management, troubleshooting, warranty period management, replacement proposals, maintenance schedule management, and spare parts proposals for home appliances.

[0030] The data collection unit collects information about home appliances. For example, it can collect information about home appliances by taking a picture of the instruction manual cover with a smartphone camera. Specifically, when the smartphone camera is used to photograph the instruction manual cover, image recognition technology is used to automatically extract information such as the model name, manufacturer, purchase date, and warranty period. Furthermore, the data collection unit can also collect detailed information such as the serial number and manufacturing date of the home appliance. This eliminates the need for users to manually enter information. The data collection unit can also automatically collect information about home appliances using AI. For example, if the home appliance is connected to the internet, the AI ​​can access the home appliance's internal database and collect detailed information such as operating status and error logs. This allows the data collection unit to understand the status of the home appliance in real time and quickly collect necessary information. Furthermore, the data collection unit can also collect information about home appliances through a dedicated app installed on the user's smartphone or tablet. For example, users can easily obtain product information by scanning the barcode or QR code (registered trademark) of the home appliance. In this way, the data collection unit can collect information about home appliances in a variety of ways, enabling convenient and efficient information collection for users.

[0031] The management department digitally manages the information collected by the collection department. For example, the management department can digitize instruction manuals and make them readily accessible when needed. Specifically, it can convert images of collected instruction manuals into text data and save them as searchable digital documents. This allows users to easily search for instruction manuals from their smartphones or tablets and quickly obtain the information they need. The management department can also securely store information about home appliances using cloud storage. Using cloud storage makes it easy to back up and restore data, reducing the risk of data loss. Furthermore, the management department can use AI to efficiently manage information about home appliances. For example, AI can automatically classify collected data and link related information, allowing users to quickly find the information they need. In addition, the management department can centrally manage the operating status and maintenance history of home appliances, allowing users to always be aware of the condition of their appliances. In this way, the management department can efficiently and securely manage information about home appliances and provide users with convenient access to information.

[0032] The troubleshooting department performs troubleshooting based on information managed by the management department. For example, when a user describes a problem by voice, the troubleshooting department can search for appropriate solutions from the instruction manual or the manufacturer's Q&A site and provide voice guidance. Specifically, when a user speaks about the problem to a smart speaker or smartphone, the department uses voice recognition technology to analyze the problem and search for relevant information. The troubleshooting department can also use AI to identify the cause of a problem and suggest appropriate solutions. The AI ​​analyzes problem patterns based on past trouble data and the manufacturer's technical information to identify the most appropriate solution. Furthermore, the troubleshooting department can use AI to automatically guide users through troubleshooting. For example, when a user inputs the details of a problem, the AI ​​automatically searches for a solution and guides them step by step. This allows users to resolve problems quickly and accurately. In addition, the troubleshooting department can collect user feedback and continuously improve the accuracy and effectiveness of troubleshooting. This enables the troubleshooting department to provide users with reliable troubleshooting solutions.

[0033] The Warranty Management Department manages warranty periods based on information managed by the Management Department. For example, the Warranty Management Department can send notifications before the warranty period expires. Specifically, as the warranty period approaches its end, it sends notifications to the user's smartphone or email to encourage extension or renewal of the warranty. The Warranty Management Department can efficiently manage warranty periods using AI, for example. The AI ​​automatically calculates the warranty period based on collected data and sends notifications at the appropriate time. The Warranty Management Department can also use AI to automatically extend or renew warranty periods. For example, if a user wishes to extend the warranty period, the AI ​​automatically performs the extension procedure and asks the user for confirmation. This allows the Warranty Management Department to ensure that users do not forget about their warranty period and can manage it properly. Furthermore, the Warranty Management Department centrally manages warranty-related information, making it easy for users to check warranty details and conditions. This allows the Warranty Management Department to provide users with convenient and reliable warranty management.

[0034] The Proposal Department makes replacement suggestions based on information managed by the Management Department. For example, the Proposal Department can analyze appliance usage and market information to suggest replacement timing and recommended new products. Specifically, it calculates the optimal timing for replacement based on the frequency of appliance use and operating time, and notifies the user. The Proposal Department can also use AI to provide users with optimal replacement suggestions. The AI ​​analyzes appliance performance and market trends and recommends new products that meet the user's needs. Furthermore, the Proposal Department can use AI to automatically notify users when it is time to replace appliances. For example, if an appliance's performance deteriorates or its energy efficiency worsens, the AI ​​will automatically suggest replacement and notify the user. In this way, the Proposal Department helps users replace appliances at the optimal time. In addition, the Proposal Department can make customized suggestions based on the user's preferences and lifestyle. In this way, the Proposal Department can provide users with optimal replacement suggestions and improve their appliance usage experience.

[0035] The Maintenance Department manages maintenance schedules based on information managed by the Management Department. For example, the Maintenance Department can automatically create maintenance schedules for home appliances and notify users of the timing of regular maintenance. Specifically, it calculates the optimal maintenance schedule based on the usage status of the appliance and the manufacturer's recommended maintenance cycle, and notifies the user. The Maintenance Department can also perform home appliance maintenance efficiently using AI. The AI ​​analyzes the operating data and past maintenance history of the appliance and proposes the optimal maintenance timing and method. Furthermore, the Maintenance Department can use AI to manage the maintenance history of home appliances. For example, it can digitize past maintenance records and make them easily accessible to users. This allows the Maintenance Department to manage home appliance maintenance efficiently and effectively, extending the lifespan of the appliances. In addition, the Maintenance Department can educate users about the importance of maintenance and encourage them to perform regular maintenance. This allows the Maintenance Department to maintain the optimal performance of home appliances and improve user satisfaction.

[0036] The Parts Proposal Department provides support for the proposal and purchase of spare parts based on information managed by the Management Department. For example, the Parts Proposal Department can analyze the storage volume and usage status of spare parts for home appliances and propose the necessary spare parts. Specifically, it evaluates the need for spare parts based on the frequency of use and past failure history of home appliances and proposes them to users. The Parts Proposal Department can also use AI to provide support such as finding suppliers and comparing prices for spare parts. The AI ​​collects price information from multiple online stores and manufacturer websites and recommends the best supplier. Furthermore, the Parts Proposal Department can use AI to automatically manage spare parts inventory. For example, if inventory is low, the AI ​​automatically proposes replenishment and notifies the user. This allows the Parts Proposal Department to properly manage and quickly obtain the spare parts that users need. In addition, the Parts Proposal Department can educate users about the importance of spare parts and guide them on proper storage and usage methods. This allows the Parts Proposal Department to reduce the risk of home appliance failure and improve user satisfaction.

[0037] The data collection unit can collect information about home appliances by taking a picture of the instruction manual cover with a smartphone camera. For example, the data collection unit takes a picture of the instruction manual cover with a smartphone camera and saves it as image data. The data collection unit can then use AI to analyze the image data and extract information such as the home appliance model name, manufacturer, purchase date, and warranty period. The data collection unit also has a function to automatically adjust the resolution and shooting angle when taking a picture of the instruction manual cover. For example, the data collection unit optimizes the resolution of the smartphone camera to ensure image clarity. The data collection unit also has a function to automatically correct light reflections and shadows during shooting. This makes it easy to collect information about home appliances by taking a picture of the instruction manual cover. Some or all of the above processing in the data collection unit may be performed using AI or not. For example, the data collection unit can input image data taken with a smartphone camera into a generating AI and have the generating AI perform the process of extracting home appliance information from the image data.

[0038] The troubleshooting unit can, when a user describes a problem by voice, search for appropriate solutions from the instruction manual or the manufacturer's Q&A site and provide voice guidance. For example, if a user says "The refrigerator isn't cooling," the troubleshooting unit can use AI to search the instruction manual or the manufacturer's Q&A site and provide voice guidance on appropriate solutions. Similarly, if a user says "The washing machine isn't working," the troubleshooting unit can use AI to identify the cause of the problem and provide voice guidance on appropriate solutions. Furthermore, if a user says "The air conditioner isn't working," the troubleshooting unit can use AI to troubleshoot the air conditioner and provide voice guidance on appropriate solutions. This allows users to receive voice guidance on appropriate solutions simply by describing the problem by voice. Some or all of the above processes in the troubleshooting unit may be performed using AI or not. For example, the troubleshooting unit can input the user's voice data into a generating AI, analyze the problem, and have the generating AI provide guidance on appropriate solutions.

[0039] The warranty management department can send notifications before the warranty period expires. For example, the warranty management department can send an email notification to the user one month before the warranty period for an appliance expires. It can also send an app notification to the user one week before the warranty period expires. Furthermore, the warranty management department can send an SMS notification to the user the day before the warranty period expires. This allows users to take appropriate action by sending notifications before the warranty period expires. Some or all of the above processes in the warranty management department may be performed using AI or not. For example, the warranty management department can input appliance warranty period data into a generating AI and have the generating AI send a notification when the warranty period expires.

[0040] The suggestion department can analyze the usage status and market information of home appliances to propose replacement timings and recommended new products. For example, the suggestion department can analyze the lifespan and frequency of malfunctions of home appliances to propose replacement timings. It can also analyze new product information on the market and propose recommended new products to users. Furthermore, the suggestion department can monitor the usage status of home appliances in real time and propose the optimal replacement time. In this way, by analyzing the usage status and market information of home appliances, it is possible to propose appropriate replacement timings and new products. Some or all of the above processes in the suggestion department may be performed using AI or not. For example, the suggestion department can input home appliance usage data and market information into a generating AI and have the generating AI propose replacement timings and new products.

[0041] The maintenance unit can automatically create maintenance schedules for home appliances and notify users of regular maintenance times. For example, the maintenance unit can automatically incorporate regular maintenance items into the schedule, such as cleaning air conditioner filters or refrigerator drain pipes. The maintenance unit can also notify users of maintenance times via email or app notifications. Furthermore, the maintenance unit can monitor the usage status of home appliances in real time and suggest necessary maintenance. This allows for automatic creation of maintenance schedules and notifications of regular maintenance times, ensuring that maintenance is not forgotten. Some or all of the above processes in the maintenance unit may be performed using AI, or not. For example, the maintenance unit can input home appliance usage data into a generating AI, which can then create and notify users of the maintenance schedule.

[0042] The parts suggestion department can analyze the storage volume and usage status of spare parts for home appliances, suggest necessary spare parts, and provide support such as comparing suppliers and prices. For example, the parts suggestion department can manage the inventory of spare parts for home appliances and notify users if necessary parts are in short supply. It can also compare suppliers and prices for spare parts and suggest the optimal supplier. Furthermore, the parts suggestion department can monitor the usage status of spare parts and notify users when replacement is necessary. In this way, by analyzing the storage volume and usage status of spare parts for home appliances, it can appropriately suggest necessary spare parts and provide support for comparing suppliers and prices. Some or all of the above processes in the parts suggestion department may be performed using AI or not. For example, the parts suggestion department can input spare parts data for home appliances into a generating AI and have the generating AI suggest spare parts and provide purchase support.

[0043] The data collection unit can analyze the purchase history of home appliances and select the optimal data collection method. For example, the data collection unit can analyze the brands and models of home appliances that the user has purchased in the past and prioritize the collection of information on the same brands and models. The data collection unit can also analyze the frequency of use of home appliances that the user has purchased in the past and prioritize the collection of information on appliances that are used frequently. Furthermore, the data collection unit can analyze the trouble history of home appliances that the user has purchased in the past and prioritize the collection of information on appliances that have had many problems. In this way, by analyzing the purchase history of home appliances, the optimal data collection method can be selected and information can be collected efficiently. Some or all of the above processing in the data collection unit may be performed using AI or not. For example, the data collection unit can input the home appliance purchase history data into a generating AI and have the generating AI select the optimal data collection method.

[0044] The data collection unit can filter the collected home appliance information based on the user's current living situation and areas of interest. For example, if a user purchases a new home appliance, the data collection unit will prioritize collecting information related to that appliance. Furthermore, if a user has shown interest in a particular home appliance, the data collection unit can prioritize collecting information related to that appliance. Additionally, if a user is in a specific living situation (e.g., moving, remodeling), the data collection unit can prioritize collecting home appliance information related to that situation. This allows for the efficient collection of highly relevant information by filtering it based on the user's living situation and areas of interest. Some or all of the above processing in the data collection unit may be performed using AI, or not. For example, the data collection unit can input data on the user's living situation and areas of interest into a generating AI, and have the generating AI perform the information filtering.

[0045] The data collection unit can prioritize the collection of highly relevant information by considering the user's geographical location when collecting information on home appliances. For example, if the user lives in a specific area, the data collection unit will prioritize the collection of information on home appliances available in that area. Furthermore, if the user is traveling, the data collection unit can prioritize the collection of information on home appliances available at their travel destination. Additionally, if the user is planning to move, the data collection unit can prioritize the collection of information on home appliances available at their new location. This allows for the efficient collection of highly relevant information by considering the user's geographical location. Some or all of the above processing in the data collection unit may be performed using AI, or without AI. For example, the data collection unit can input the user's geographical location information into a generating AI, which can then collect highly relevant information.

[0046] The data collection unit can analyze the user's social media activity and collect relevant information when collecting information on home appliances. For example, if a user mentions a specific home appliance on social media, the data collection unit will prioritize collecting information related to that appliance. Furthermore, if a user follows a specific home appliance brand on social media, the data collection unit can prioritize collecting information related to that brand. Additionally, if a user frequently posts about a specific home appliance on social media, the data collection unit can prioritize collecting information related to that appliance. This allows for the efficient collection of relevant information by analyzing the user's social media activity. Some or all of the above processing in the data collection unit may be performed using AI or not. For example, the data collection unit can input the user's social media data into a generating AI and have the generating AI collect relevant information.

[0047] The management department can adjust the level of detail in its management based on the importance of the appliance information. For example, the management department can prioritize the management of important appliance information (e.g., warranty period, troubleshooting information). It can also manage information on appliances that are frequently used in detail. Furthermore, it can manage information on appliances that experience frequent problems in detail. In this way, by adjusting the level of detail in management based on the importance of the appliance information, important information can be prioritized. Some or all of the above processes in the management department may be performed using AI or not. For example, the management department can input importance data of appliance information into a generating AI and have the generating AI perform the process of adjusting the level of detail in management.

[0048] The management unit can apply different management algorithms depending on the category of home appliance. For example, the management unit can apply a management algorithm that emphasizes usage frequency and maintenance information to kitchen appliances. It can also apply a management algorithm that emphasizes software update information to entertainment appliances. Furthermore, it can apply a management algorithm that emphasizes consumable replacement information to household appliances. By applying different management algorithms depending on the category of home appliance, optimal management can be achieved for each category. Some or all of the above processing in the management unit may be performed using AI, or not. For example, the management unit can input home appliance category data into a generating AI and have the generating AI apply the management algorithm.

[0049] The management department can determine management priorities based on the timing of appliance information submission. For example, the management department may prioritize the management of recently submitted appliance information. It may also prioritize the management of older appliance information. Furthermore, it may prioritize the management of appliance information whose submission dates are concentrated within a specific period. This allows for the prioritization of the latest information by determining management priorities based on the timing of appliance information submission. Some or all of the above processes in the management department may be performed using AI or not. For example, the management department may input appliance information submission timing data into a generating AI and have the generating AI perform the process of determining management priorities.

[0050] The management unit can adjust the order of management based on the relevance of home appliance information. For example, the management unit can prioritize the management of highly relevant home appliance information. It can also postpone the management of less relevant home appliance information. Furthermore, the management unit can group and manage relevant home appliance information. This allows for the prioritization of highly relevant information by adjusting the order of management based on the relevance of home appliance information. Some or all of the above processes in the management unit may be performed using AI or not. For example, the management unit can input relevance data of home appliance information into a generating AI and have the generating AI perform the process of adjusting the order of management.

[0051] The troubleshooting unit can select the optimal solution by referring to the appliance's past trouble history during troubleshooting. For example, the troubleshooting unit can analyze the appliance's past trouble history and suggest solutions for similar problems that may occur. The troubleshooting unit can also identify the cause of a problem from the appliance's past trouble history and suggest the optimal solution. Furthermore, the troubleshooting unit can suggest measures to prevent the recurrence of problems based on the appliance's past trouble history. In this way, by referring to the appliance's past trouble history, the optimal solution can be selected and problems can be resolved efficiently. Some or all of the above processes in the troubleshooting unit may be performed using AI, or they may not be performed using AI. For example, the troubleshooting unit can input the appliance's past trouble history data into a generating AI and have the generating AI select the optimal solution.

[0052] The troubleshooting unit can apply different troubleshooting methods depending on the category of home appliance. For example, the troubleshooting unit can apply troubleshooting methods that prioritize usage frequency and maintenance information to kitchen appliances. It can also apply troubleshooting methods that prioritize software update information to entertainment appliances. Furthermore, it can apply troubleshooting methods that prioritize consumable replacement information to household appliances. By applying different troubleshooting methods to each category of appliance, the troubleshooting unit can perform troubleshooting that is optimal for each category. Some or all of the above processes in the troubleshooting unit may be performed using AI or not. For example, the troubleshooting unit can input appliance category data into a generating AI and have the generating AI apply the troubleshooting methods.

[0053] The troubleshooting unit can select the optimal solution during troubleshooting by considering the geographical distribution of home appliances. For example, if a home appliance is widely used in a particular region, the troubleshooting unit can propose a solution that takes into account the characteristics of that region. Furthermore, if a home appliance is used under specific climatic conditions, the troubleshooting unit can propose a solution that takes those conditions into account. Additionally, if a home appliance is used in a particular cultural area, the troubleshooting unit can propose a solution that takes into account the characteristics of that cultural area. This allows for the selection of the optimal solution tailored to regional characteristics by considering the geographical distribution of home appliances. Some or all of the above-described processes in the troubleshooting unit may be performed using AI, or they may not. For example, the troubleshooting unit can input geographical distribution data of home appliances into a generating AI and have the generating AI select the optimal solution.

[0054] The troubleshooting unit can propose solutions by referring to relevant documentation for home appliances during troubleshooting. For example, the troubleshooting unit can propose the best solution by referring to the home appliance's instruction manual. It can also propose the best solution by referring to the home appliance manufacturer's Q&A site. Furthermore, the troubleshooting unit can propose the best solution by analyzing relevant documentation for home appliances. This allows for efficient troubleshooting by proposing the best solution through the referencing of relevant documentation for home appliances. Some or all of the above processes in the troubleshooting unit may be performed using AI or not. For example, the troubleshooting unit can input data on relevant documentation for home appliances into a generating AI and have the generating AI propose solutions.

[0055] The warranty management department can select the optimal management method by referring to the past warranty history of home appliances when managing the warranty period. For example, the warranty management department can analyze the past warranty history of home appliances and propose the optimal method for managing the warranty period of similar home appliances. The warranty management department can also propose how to handle cases where the warranty period needs to be extended based on the past warranty history of home appliances. Furthermore, the warranty management department can propose measures to prevent recurrence of warranty period issues based on the past warranty history of home appliances. In this way, by referring to the past warranty history of home appliances, the optimal warranty period management method can be selected and managed efficiently. Some or all of the above processes in the warranty management department may be performed using AI or not. For example, the warranty management department can input past warranty history data of home appliances into a generating AI and have the generating AI select the optimal management method.

[0056] The warranty management department can select the optimal management method when managing the warranty period, taking into account the geographical location information of the home appliance. For example, if the home appliance is widely used in a particular region, the warranty management department can propose a warranty period management method that takes into account the characteristics of that region. Furthermore, if the home appliance is used under specific climatic conditions, the warranty management department can also propose a warranty period management method that takes into account those climatic conditions. In addition, if the home appliance is used in a particular cultural area, the warranty management department can propose a warranty period management method that takes into account the characteristics of that cultural area. In this way, by considering the geographical location information of the home appliance, the optimal warranty period management method according to regional characteristics can be selected. Some or all of the above processing in the warranty management department may be performed using AI, or it may be performed without using AI. For example, the warranty management department can input the geographical location information data of the home appliance into a generating AI and have the generating AI select the optimal management method.

[0057] The proposal unit can adjust the level of detail in its proposals based on the importance of the appliances. For example, the proposal unit can provide detailed proposals for important appliances (e.g., refrigerators, washing machines). It can also provide detailed proposals for appliances that are frequently used. Furthermore, it can provide detailed proposals for appliances that are prone to problems. By adjusting the level of detail in proposals based on the importance of the appliances, it is possible to provide detailed proposals for important appliances. Some or all of the above processing in the proposal unit may be performed using AI or not. For example, the proposal unit can input appliance importance data into a generating AI and have the generating AI perform the process of adjusting the level of detail in the proposals.

[0058] The suggestion unit can apply different suggestion algorithms depending on the category of the home appliance when making suggestions. For example, the suggestion unit can apply a suggestion algorithm that emphasizes usage frequency and maintenance information to kitchen appliances. It can also apply a suggestion algorithm that emphasizes software update information to entertainment appliances. Furthermore, it can apply a suggestion algorithm that emphasizes consumable replacement information to household appliances. By applying different suggestion algorithms depending on the category of the home appliance, the unit can make optimal suggestions for each category. Some or all of the above processing in the suggestion unit may be performed using AI or not. For example, the suggestion unit can input home appliance category data into a generating AI and have the generating AI apply the suggestion algorithm.

[0059] The proposal department can determine the priority of proposals based on the submission timing of home appliances. For example, the proposal department can make proposals based on recently submitted home appliance information. It can also make proposals based on older home appliance information. Furthermore, it can make proposals based on home appliance information where submission timings are concentrated within a specific period. By determining the priority of proposals based on the submission timing of home appliances, it is possible to make proposals based on the latest information. Some or all of the above processing in the proposal department may be performed using AI or not. For example, the proposal department can input home appliance submission timing data into a generating AI and have the generating AI perform the process of determining the priority of proposals.

[0060] The suggestion unit can adjust the order of suggestions based on the relevance of the home appliances. For example, the suggestion unit can make suggestions based on highly relevant home appliance information. It can also make suggestions based on less relevant home appliance information. Furthermore, the suggestion unit can group relevant home appliance information and make suggestions based on that group. By adjusting the order of suggestions based on the relevance of the home appliances, it is possible to make suggestions based on highly relevant information. Some or all of the above processing in the suggestion unit may be performed using AI or not. For example, the suggestion unit can input relevance data of home appliance information into a generating AI and have the generating AI perform the process of adjusting the order of suggestions.

[0061] The maintenance department can create an optimal maintenance schedule by referring to the appliance's past maintenance history. For example, the maintenance department can analyze the appliance's past maintenance history and propose an optimal maintenance schedule for similar appliances. The maintenance department can also propose methods to optimize maintenance frequency based on the appliance's past maintenance history. Furthermore, the maintenance department can propose measures to prevent recurrence of maintenance issues based on the appliance's past maintenance history. This allows for the creation of an optimal maintenance schedule and efficient maintenance by referring to the appliance's past maintenance history. Some or all of the above processes in the maintenance department may be performed using AI or not. For example, the maintenance department can input the appliance's past maintenance history data into a generating AI and have the generating AI create an optimal schedule.

[0062] The maintenance unit can apply different maintenance methods to each category of home appliance when creating maintenance schedules. For example, the maintenance unit can apply maintenance methods that prioritize usage frequency and maintenance information to kitchen appliances. It can also apply maintenance methods that prioritize software update information to entertainment appliances. Furthermore, it can apply maintenance methods that prioritize consumable replacement information to household appliances. By applying different maintenance methods to each category of appliance, optimal maintenance can be performed for each category. Some or all of the above processing in the maintenance unit may be performed using AI or not. For example, the maintenance unit can input appliance category data into a generating AI and have the generating AI apply the maintenance methods.

[0063] The maintenance department can create an optimal maintenance schedule by considering the geographical location of the home appliances. For example, if a home appliance is widely used in a particular region, the maintenance department can propose a maintenance schedule that takes into account the characteristics of that region. Furthermore, if a home appliance is used under specific climatic conditions, the maintenance department can propose a maintenance schedule that takes those conditions into account. Additionally, if a home appliance is used in a particular cultural area, the maintenance department can propose a maintenance schedule that takes into account the characteristics of that cultural area. This allows for the creation of an optimal maintenance schedule tailored to regional characteristics by considering the geographical location of the home appliances. Some or all of the above processes in the maintenance department may be performed using AI, or they may not. For example, the maintenance department can input geographical location data of the home appliances into a generating AI and have the generating AI create the optimal schedule.

[0064] The maintenance department can propose a maintenance schedule by referring to relevant documentation for home appliances when creating the schedule. For example, the maintenance department can propose an optimal maintenance schedule by referring to the instruction manual for the home appliance. It can also propose an optimal maintenance schedule by referring to the home appliance manufacturer's Q&A site. Furthermore, the maintenance department can propose an optimal maintenance schedule by analyzing relevant documentation for home appliances. This allows for efficient maintenance by proposing an optimal maintenance schedule through the referencing of relevant documentation for home appliances. Some or all of the above processes in the maintenance department may be performed using AI or not. For example, the maintenance department can input data on relevant documentation for home appliances into a generating AI and have the generating AI propose a schedule.

[0065] The parts proposal department can select the optimal proposal method when proposing parts by referring to the past parts usage history of home appliances. For example, the parts proposal department can analyze the past parts usage history of home appliances and propose the optimal parts proposal method for similar home appliances. The parts proposal department can also propose methods to optimize the frequency of parts replacement based on the past parts usage history of home appliances. Furthermore, the parts proposal department can propose measures to prevent recurrence of parts problems based on the past parts usage history of home appliances. In this way, by referring to the past parts usage history of home appliances, the optimal parts proposal method can be selected and parts can be proposed efficiently. Some or all of the above processes in the parts proposal department may be performed using AI or not. For example, the parts proposal department can input past parts usage history data of home appliances into a generating AI and have the generating AI select the optimal proposal method.

[0066] The parts suggestion department can apply different suggestion methods to each category of home appliance when suggesting parts. For example, for kitchen appliances, the parts suggestion department can apply a parts suggestion method that emphasizes usage frequency and maintenance information. For entertainment appliances, the parts suggestion department can also apply a parts suggestion method that emphasizes software update information. Furthermore, for household appliances, the parts suggestion department can apply a parts suggestion method that emphasizes consumable replacement information. By applying different suggestion methods to each category of home appliance, the department can provide optimal parts suggestions for each category. Some or all of the above processing in the parts suggestion department may be performed using AI, or not. For example, the parts suggestion department can input home appliance category data into a generating AI and have the generating AI apply the suggestion method.

[0067] The parts proposal department can select the optimal proposal method when proposing parts, taking into account the geographical location information of the home appliance. For example, if a home appliance is widely used in a particular region, the parts proposal department can propose a parts proposal method that takes into account the characteristics of that region. Furthermore, if a home appliance is used under specific climatic conditions, the parts proposal department can propose a parts proposal method that takes those conditions into account. In addition, if a home appliance is used in a particular cultural area, the parts proposal department can propose a parts proposal method that takes into account the characteristics of that cultural area. In this way, by considering the geographical location information of the home appliance, the optimal parts proposal method can be selected according to regional characteristics. Some or all of the above processing in the parts proposal department may be performed using AI, or it may be performed without using AI. For example, the parts proposal department can input geographical location information data of the home appliance into a generating AI and have the generating AI select the optimal proposal method.

[0068] The parts proposal department can propose a parts proposal method by referring to relevant literature on home appliances when proposing parts. For example, the parts proposal department can propose the optimal parts proposal method by referring to the instruction manual of a home appliance. It can also propose the optimal parts proposal method by referring to the Q&A site of the home appliance manufacturer. Furthermore, the parts proposal department can propose the optimal parts proposal method by analyzing relevant literature on home appliances. In this way, by referring to relevant literature on home appliances, the optimal parts proposal method can be proposed, and parts can be proposed efficiently. Some or all of the above processing in the parts proposal department may be performed using AI or not. For example, the parts proposal department can input data on relevant literature on home appliances into a generating AI and have the generating AI propose a proposal method.

[0069] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0070] The smart home appliance concierge can monitor the usage of home appliances in real time and automatically initiate troubleshooting if an abnormality is detected. For example, if the temperature of a refrigerator rises rapidly, the system will automatically detect the abnormality, notify the user, and suggest appropriate countermeasures. Similarly, if a washing machine detects abnormal vibrations, the system can automatically initiate troubleshooting and guide the user on how to resolve the issue. Furthermore, if an air conditioner filter is clogged, the system can automatically detect the abnormality and send a notification prompting the user to clean the filter. This allows for early detection of abnormalities and the provision of appropriate countermeasures by monitoring the usage of home appliances in real time. Some or all of the above processes in the troubleshooting unit may be performed using AI or not. For example, the troubleshooting unit can input home appliance usage data into a generating AI, which can then detect abnormalities and suggest countermeasures.

[0071] The smart home appliance concierge can analyze the purchase history of home appliances and select the optimal data collection method. For example, it can analyze the brands and models of home appliances that the user has purchased in the past and prioritize the collection of information on the same brands and models. It can also analyze the frequency of use of home appliances that the user has purchased in the past and prioritize the collection of information on appliances that are used frequently. Furthermore, it can analyze the trouble history of home appliances that the user has purchased in the past and prioritize the collection of information on appliances that have had many problems. In this way, by analyzing the purchase history of home appliances, the optimal data collection method can be selected and information can be collected efficiently. Some or all of the above processing in the collection unit may be performed using AI or not. For example, the collection unit can input home appliance purchase history data into a generating AI and have the generating AI select the optimal data collection method.

[0072] The smart home appliance concierge can prioritize collecting highly relevant information by considering the user's geographical location when gathering information about home appliances. For example, if the user lives in a specific area, it can prioritize collecting information about home appliances available in that area. Furthermore, if the user is traveling, it can prioritize collecting information about home appliances available at their travel destination. Additionally, if the user is planning to move, it can prioritize collecting information about home appliances available at their new address. This allows for the efficient collection of highly relevant information by considering the user's geographical location. Some or all of the above processing in the collection unit may be performed using AI, or without AI. For example, the collection unit can input the user's geographical location information into a generating AI, which can then collect highly relevant information.

[0073] The smart home appliance concierge can collect relevant information by analyzing the user's social media activity when gathering information about home appliances. For example, if a user mentions a specific home appliance on social media, it can prioritize collecting information related to that appliance. Similarly, if a user follows a specific home appliance brand on social media, it can prioritize collecting information related to that brand. Furthermore, if a user frequently posts about a specific home appliance on social media, it can prioritize collecting information related to that appliance. This allows for the efficient collection of relevant information by analyzing the user's social media activity. Some or all of the above processing in the collection unit may be performed using AI or not. For example, the collection unit can input the user's social media data into a generating AI, which can then collect the relevant information.

[0074] The smart home appliance concierge can apply different management algorithms to each category of home appliance. For example, kitchen appliances can be managed using an algorithm that prioritizes usage frequency and maintenance information. Entertainment appliances can be managed using an algorithm that prioritizes software update information. Furthermore, household appliances can be managed using an algorithm that prioritizes consumable replacement information. By applying different management algorithms to each category of appliance, optimal management can be achieved for each category. Some or all of the above processing in the management unit may be performed using AI, or not. For example, the management unit can input appliance category data into a generating AI and have the generating AI apply the management algorithm.

[0075] The following briefly describes the processing flow for example form 1.

[0076] Step 1: The data collection unit collects information about home appliances. For example, the data collection unit can collect information about home appliances by taking a picture of the instruction manual cover with a smartphone camera. The data collection unit can also collect information such as the model name, manufacturer, purchase date, and warranty period of the home appliance. In addition, the data collection unit can automatically collect information about home appliances using AI. Step 2: The management department digitally manages the information collected by the collection department. For example, the management department can digitize instruction manuals so that they can be accessed immediately when needed. For example, the management department can use cloud storage to securely store information about home appliances. The management department can also use AI to efficiently manage information about home appliances. Step 3: The troubleshooting department performs troubleshooting based on information managed by the management department. For example, when a user describes the problem by voice, the troubleshooting department can search for appropriate solutions from the instruction manual or the manufacturer's Q&A site and provide voice guidance. For example, the troubleshooting department can use AI to identify the cause of the problem and suggest appropriate solutions. The troubleshooting department can also use AI to automatically guide users through resolving the problem. Step 4: The warranty management department manages the warranty period based on the information managed by the management department. The warranty management department can, for example, send notifications before the warranty period expires. The warranty management department can, for example, use AI to efficiently manage the warranty period. The warranty management department can also use AI to automatically extend or renew the warranty period. Step 5: The Proposal Department makes replacement suggestions based on information managed by the Management Department. For example, the Proposal Department can analyze appliance usage and market information to suggest replacement timing and recommended new products. For example, the Proposal Department can use AI to provide users with optimal replacement suggestions. The Proposal Department can also use AI to automatically notify users when it is time to replace their appliances. Step 6: The maintenance department manages the maintenance schedule based on the information managed by the management department. The maintenance department can, for example, automatically create maintenance schedules for home appliances and notify them of the timing of regular maintenance. The maintenance department can, for example, use AI to perform maintenance on home appliances efficiently. The maintenance department can also use AI to manage the maintenance history of home appliances. Step 7: The parts proposal department provides support for proposing and purchasing spare parts based on information managed by the management department. For example, the parts proposal department can analyze the storage volume and usage status of spare parts for home appliances and propose the necessary spare parts. For example, the parts proposal department can use AI to provide support such as finding suppliers for spare parts and comparing prices. The parts proposal department can also use AI to automatically manage spare parts inventory.

[0077] (Example of form 2) The Smart Home Appliance Concierge, according to an embodiment of the present invention, is an AI application for facilitating the management of home appliances. The Smart Home Appliance Concierge alleviates anxieties and concerns regarding home appliance troubles and management. For example, when a user purchases a home appliance, they can take a picture of the instruction manual cover with their smartphone camera, which automatically collects and registers information about the appliance. This information includes the appliance's model name, manufacturer, purchase date, and warranty period. Next, based on the registered appliance information, the AI ​​digitally manages the instruction manuals, making them readily accessible when needed. Furthermore, the AI ​​supports troubleshooting of home appliances. For example, if an appliance malfunctions, the user can describe the problem by voice, and the AI ​​will search for appropriate solutions from the instruction manual or the manufacturer's Q&A site and provide voice guidance. It also manages the warranty period and sends notifications before the warranty expires to prompt necessary action. In addition, the AI ​​analyzes the usage status and market information of home appliances to suggest replacement timings and recommend new products. Furthermore, it automatically creates a home appliance maintenance schedule and notifies users of regular maintenance times. For example, it supports often-forgotten maintenance such as cleaning air conditioner filters or refrigerator drain pipes. Finally, the AI ​​also provides support for suggesting and purchasing spare parts. It analyzes the amount and usage of spare parts stored for home appliances and suggests the necessary spare parts. It also provides support such as finding suppliers for spare parts and comparing prices. In this way, the smart home appliance concierge centralizes the management of home appliances and supports troubleshooting, maintenance, and replacement suggestions, making users' lives more convenient and comfortable.

[0078] The smart home appliance concierge according to this embodiment comprises a collection unit, a management unit, a troubleshooting unit, a warranty management unit, a proposal unit, a maintenance unit, and a parts proposal unit. The collection unit collects information about home appliances. For example, the collection unit can collect information about home appliances by taking a picture of the cover of the instruction manual with a smartphone camera. The collection unit can collect information such as the model name, manufacturer, purchase date, and warranty period of the home appliance. The collection unit can also automatically collect information about home appliances using AI. The management unit digitally manages the information collected by the collection unit. For example, the management unit can digitize instruction manuals so that they can be accessed immediately when needed. For example, the management unit can securely store information about home appliances using cloud storage. The management unit can also efficiently manage information about home appliances using AI. The troubleshooting unit performs troubleshooting based on the information managed by the management unit. For example, when a user describes the problem by voice, the troubleshooting unit can search for appropriate solutions from the instruction manual or the manufacturer's Q&A site and provide voice guidance. The troubleshooting department can, for example, use AI to identify the cause of a problem and suggest appropriate solutions. The troubleshooting department can also use AI to automatically guide users through troubleshooting. The warranty management department manages warranty periods based on information managed by the management department. The warranty management department can, for example, send notifications before the warranty period expires. The warranty management department can, for example, use AI to efficiently manage warranty periods. Furthermore, the warranty management department can use AI to automatically extend or renew warranties. The recommendation department makes replacement suggestions based on information managed by the management department. The recommendation department can, for example, analyze appliance usage and market information to suggest replacement timing and recommend new products. The recommendation department can, for example, use AI to provide users with optimal replacement suggestions. Furthermore, the recommendation department can use AI to automatically notify users of when it's time to replace their appliances. The maintenance department manages maintenance schedules based on information managed by the management department.The maintenance department can, for example, automatically create maintenance schedules for home appliances and notify users of the timing of regular maintenance. The maintenance department can, for example, use AI to efficiently perform maintenance on home appliances. The maintenance department can also use AI to manage the maintenance history of home appliances. The parts proposal department provides support for proposing and purchasing spare parts based on information managed by the management department. The parts proposal department can, for example, analyze the amount of spare parts stored and their usage status for home appliances and propose the necessary spare parts. The parts proposal department can, for example, use AI to provide support such as finding suppliers for spare parts and comparing prices. The parts proposal department can also use AI to automatically manage the inventory of spare parts. As a result, the smart home appliance concierge according to this embodiment can centrally perform information gathering, management, troubleshooting, warranty period management, replacement proposals, maintenance schedule management, and spare parts proposals for home appliances.

[0079] The data collection unit collects information about home appliances. For example, it can collect information about home appliances by taking a picture of the instruction manual cover with a smartphone camera. Specifically, when the smartphone camera is used to photograph the instruction manual cover, image recognition technology is used to automatically extract information such as the model name, manufacturer, purchase date, and warranty period. Furthermore, the data collection unit can also collect detailed information such as the serial number and manufacturing date of the home appliance. This eliminates the need for users to manually enter information. The data collection unit can also automatically collect information about home appliances using AI. For example, if the home appliance is connected to the internet, the AI ​​can access the home appliance's internal database and collect detailed information such as operating status and error logs. This allows the data collection unit to understand the status of the home appliance in real time and quickly collect necessary information. In addition, the data collection unit can also collect information about home appliances through a dedicated app installed on the user's smartphone or tablet. For example, users can easily obtain product information by scanning the barcode or QR code of the home appliance. In this way, the data collection unit can collect information about home appliances in a variety of ways, enabling convenient and efficient information collection for users.

[0080] The management department digitally manages the information collected by the collection department. For example, the management department can digitize instruction manuals and make them readily accessible when needed. Specifically, it can convert images of collected instruction manuals into text data and save them as searchable digital documents. This allows users to easily search for instruction manuals from their smartphones or tablets and quickly obtain the information they need. The management department can also securely store information about home appliances using cloud storage. Using cloud storage makes it easy to back up and restore data, reducing the risk of data loss. Furthermore, the management department can use AI to efficiently manage information about home appliances. For example, AI can automatically classify collected data and link related information, allowing users to quickly find the information they need. In addition, the management department can centrally manage the operating status and maintenance history of home appliances, allowing users to always be aware of the condition of their appliances. In this way, the management department can efficiently and securely manage information about home appliances and provide users with convenient access to information.

[0081] The troubleshooting department performs troubleshooting based on information managed by the management department. For example, when a user describes a problem by voice, the troubleshooting department can search for appropriate solutions from the instruction manual or the manufacturer's Q&A site and provide voice guidance. Specifically, when a user speaks about the problem to a smart speaker or smartphone, the department uses voice recognition technology to analyze the problem and search for relevant information. The troubleshooting department can also use AI to identify the cause of a problem and suggest appropriate solutions. The AI ​​analyzes problem patterns based on past trouble data and the manufacturer's technical information to identify the most appropriate solution. Furthermore, the troubleshooting department can use AI to automatically guide users through troubleshooting. For example, when a user inputs the details of a problem, the AI ​​automatically searches for a solution and guides them step by step. This allows users to resolve problems quickly and accurately. In addition, the troubleshooting department can collect user feedback and continuously improve the accuracy and effectiveness of troubleshooting. This enables the troubleshooting department to provide users with reliable troubleshooting solutions.

[0082] The Warranty Management Department manages warranty periods based on information managed by the Management Department. For example, the Warranty Management Department can send notifications before the warranty period expires. Specifically, as the warranty period approaches its end, it sends notifications to the user's smartphone or email to encourage extension or renewal of the warranty. The Warranty Management Department can efficiently manage warranty periods using AI, for example. The AI ​​automatically calculates the warranty period based on collected data and sends notifications at the appropriate time. The Warranty Management Department can also use AI to automatically extend or renew warranty periods. For example, if a user wishes to extend the warranty period, the AI ​​automatically performs the extension procedure and asks the user for confirmation. This allows the Warranty Management Department to ensure that users do not forget about their warranty period and can manage it properly. Furthermore, the Warranty Management Department centrally manages warranty-related information, making it easy for users to check warranty details and conditions. This allows the Warranty Management Department to provide users with convenient and reliable warranty management.

[0083] The Proposal Department makes replacement suggestions based on information managed by the Management Department. For example, the Proposal Department can analyze appliance usage and market information to suggest replacement timing and recommended new products. Specifically, it calculates the optimal timing for replacement based on the frequency of appliance use and operating time, and notifies the user. The Proposal Department can also use AI to provide users with optimal replacement suggestions. The AI ​​analyzes appliance performance and market trends and recommends new products that meet the user's needs. Furthermore, the Proposal Department can use AI to automatically notify users when it is time to replace appliances. For example, if an appliance's performance deteriorates or its energy efficiency worsens, the AI ​​will automatically suggest replacement and notify the user. In this way, the Proposal Department helps users replace appliances at the optimal time. In addition, the Proposal Department can make customized suggestions based on the user's preferences and lifestyle. In this way, the Proposal Department can provide users with optimal replacement suggestions and improve their appliance usage experience.

[0084] The Maintenance Department manages maintenance schedules based on information managed by the Management Department. For example, the Maintenance Department can automatically create maintenance schedules for home appliances and notify users of the timing of regular maintenance. Specifically, it calculates the optimal maintenance schedule based on the usage status of the appliance and the manufacturer's recommended maintenance cycle, and notifies the user. The Maintenance Department can also perform home appliance maintenance efficiently using AI. The AI ​​analyzes the operating data and past maintenance history of the appliance and proposes the optimal maintenance timing and method. Furthermore, the Maintenance Department can use AI to manage the maintenance history of home appliances. For example, it can digitize past maintenance records and make them easily accessible to users. This allows the Maintenance Department to manage home appliance maintenance efficiently and effectively, extending the lifespan of the appliances. In addition, the Maintenance Department can educate users about the importance of maintenance and encourage them to perform regular maintenance. This allows the Maintenance Department to maintain the optimal performance of home appliances and improve user satisfaction.

[0085] The Parts Proposal Department provides support for the proposal and purchase of spare parts based on information managed by the Management Department. For example, the Parts Proposal Department can analyze the storage volume and usage status of spare parts for home appliances and propose the necessary spare parts. Specifically, it evaluates the need for spare parts based on the frequency of use and past failure history of home appliances and proposes them to users. The Parts Proposal Department can also use AI to provide support such as finding suppliers and comparing prices for spare parts. The AI ​​collects price information from multiple online stores and manufacturer websites and recommends the best supplier. Furthermore, the Parts Proposal Department can use AI to automatically manage spare parts inventory. For example, if inventory is low, the AI ​​automatically proposes replenishment and notifies the user. This allows the Parts Proposal Department to properly manage and quickly obtain the spare parts that users need. In addition, the Parts Proposal Department can educate users about the importance of spare parts and guide them on proper storage and usage methods. This allows the Parts Proposal Department to reduce the risk of home appliance failure and improve user satisfaction.

[0086] The data collection unit can collect information about home appliances by taking a picture of the instruction manual cover with a smartphone camera. For example, the data collection unit takes a picture of the instruction manual cover with a smartphone camera and saves it as image data. The data collection unit can then use AI to analyze the image data and extract information such as the home appliance model name, manufacturer, purchase date, and warranty period. The data collection unit also has a function to automatically adjust the resolution and shooting angle when taking a picture of the instruction manual cover. For example, the data collection unit optimizes the resolution of the smartphone camera to ensure image clarity. The data collection unit also has a function to automatically correct light reflections and shadows during shooting. This makes it easy to collect information about home appliances by taking a picture of the instruction manual cover. Some or all of the above processing in the data collection unit may be performed using AI or not. For example, the data collection unit can input image data taken with a smartphone camera into a generating AI and have the generating AI perform the process of extracting home appliance information from the image data.

[0087] The troubleshooting unit can, when a user describes a problem by voice, search for appropriate solutions from the instruction manual or the manufacturer's Q&A site and provide voice guidance. For example, if a user says "The refrigerator isn't cooling," the troubleshooting unit can use AI to search the instruction manual or the manufacturer's Q&A site and provide voice guidance on appropriate solutions. Similarly, if a user says "The washing machine isn't working," the troubleshooting unit can use AI to identify the cause of the problem and provide voice guidance on appropriate solutions. Furthermore, if a user says "The air conditioner isn't working," the troubleshooting unit can use AI to troubleshoot the air conditioner and provide voice guidance on appropriate solutions. This allows users to receive voice guidance on appropriate solutions simply by describing the problem by voice. Some or all of the above processes in the troubleshooting unit may be performed using AI or not. For example, the troubleshooting unit can input the user's voice data into a generating AI, analyze the problem, and have the generating AI provide guidance on appropriate solutions.

[0088] The warranty management department can send notifications before the warranty period expires. For example, the warranty management department can send an email notification to the user one month before the warranty period for an appliance expires. It can also send an app notification to the user one week before the warranty period expires. Furthermore, the warranty management department can send an SMS notification to the user the day before the warranty period expires. This allows users to take appropriate action by sending notifications before the warranty period expires. Some or all of the above processes in the warranty management department may be performed using AI or not. For example, the warranty management department can input appliance warranty period data into a generating AI and have the generating AI send a notification when the warranty period expires.

[0089] The suggestion department can analyze the usage status and market information of home appliances to propose replacement timings and recommended new products. For example, the suggestion department can analyze the lifespan and frequency of malfunctions of home appliances to propose replacement timings. It can also analyze new product information on the market and propose recommended new products to users. Furthermore, the suggestion department can monitor the usage status of home appliances in real time and propose the optimal replacement time. In this way, by analyzing the usage status and market information of home appliances, it is possible to propose appropriate replacement timings and new products. Some or all of the above processes in the suggestion department may be performed using AI or not. For example, the suggestion department can input home appliance usage data and market information into a generating AI and have the generating AI propose replacement timings and new products.

[0090] The maintenance unit can automatically create maintenance schedules for home appliances and notify users of regular maintenance times. For example, the maintenance unit can automatically incorporate regular maintenance items into the schedule, such as cleaning air conditioner filters or refrigerator drain pipes. The maintenance unit can also notify users of maintenance times via email or app notifications. Furthermore, the maintenance unit can monitor the usage status of home appliances in real time and suggest necessary maintenance. This allows for automatic creation of maintenance schedules and notifications of regular maintenance times, ensuring that maintenance is not forgotten. Some or all of the above processes in the maintenance unit may be performed using AI, or not. For example, the maintenance unit can input home appliance usage data into a generating AI, which can then create and notify users of the maintenance schedule.

[0091] The parts suggestion department can analyze the storage volume and usage status of spare parts for home appliances, suggest necessary spare parts, and provide support such as comparing suppliers and prices. For example, the parts suggestion department can manage the inventory of spare parts for home appliances and notify users if necessary parts are in short supply. It can also compare suppliers and prices for spare parts and suggest the optimal supplier. Furthermore, the parts suggestion department can monitor the usage status of spare parts and notify users when replacement is necessary. In this way, by analyzing the storage volume and usage status of spare parts for home appliances, it can appropriately suggest necessary spare parts and provide support for comparing suppliers and prices. Some or all of the above processes in the parts suggestion department may be performed using AI or not. For example, the parts suggestion department can input spare parts data for home appliances into a generating AI and have the generating AI suggest spare parts and provide purchase support.

[0092] The data collection unit can estimate the user's emotions and adjust the timing of collecting home appliance information based on the estimated emotions. For example, if the user is stressed, the data collection unit can delay the collection timing and collect data when the user is relaxed. The data collection unit can also adjust the collection timing if the user is busy and collect data when the user has free time. Furthermore, if the user is excited, the data collection unit can advance the collection timing and collect data when the user has calmed down. This allows for the reduction of user stress and the collection of information at the appropriate time by adjusting the timing of home appliance information collection based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the data collection unit may be performed using AI or not. For example, the data collection unit can input user emotion data into a generative AI and have the generative AI adjust the collection timing.

[0093] The data collection unit can analyze the purchase history of home appliances and select the optimal data collection method. For example, the data collection unit can analyze the brands and models of home appliances that the user has purchased in the past and prioritize the collection of information on the same brands and models. The data collection unit can also analyze the frequency of use of home appliances that the user has purchased in the past and prioritize the collection of information on appliances that are used frequently. Furthermore, the data collection unit can analyze the trouble history of home appliances that the user has purchased in the past and prioritize the collection of information on appliances that have had many problems. In this way, by analyzing the purchase history of home appliances, the optimal data collection method can be selected and information can be collected efficiently. Some or all of the above processing in the data collection unit may be performed using AI or not. For example, the data collection unit can input the home appliance purchase history data into a generating AI and have the generating AI select the optimal data collection method.

[0094] The data collection unit can filter the collected home appliance information based on the user's current living situation and areas of interest. For example, if a user purchases a new home appliance, the data collection unit will prioritize collecting information related to that appliance. Furthermore, if a user has shown interest in a particular home appliance, the data collection unit can prioritize collecting information related to that appliance. Additionally, if a user is in a specific living situation (e.g., moving, remodeling), the data collection unit can prioritize collecting home appliance information related to that situation. This allows for the efficient collection of highly relevant information by filtering it based on the user's living situation and areas of interest. Some or all of the above processing in the data collection unit may be performed using AI, or not. For example, the data collection unit can input data on the user's living situation and areas of interest into a generating AI, and have the generating AI perform the information filtering.

[0095] The data collection unit can estimate the user's emotions and determine the priority of appliance information to collect based on the estimated emotions. For example, if the user is feeling stressed, the data collection unit will prioritize collecting appliance information that helps reduce stress. It can also prioritize collecting appliance information that promotes relaxation if the user is relaxed. Furthermore, if the user is excited, the data collection unit can prioritize collecting appliance information that calms excitement. This allows for the collection of information tailored to the user's needs by prioritizing appliance information based on their emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processing in the data collection unit may be performed using AI or not. For example, the data collection unit can input user emotion data into a generative AI and have the generative AI perform the process of determining the priority of information.

[0096] The data collection unit can prioritize the collection of highly relevant information by considering the user's geographical location when collecting information on home appliances. For example, if the user lives in a specific area, the data collection unit will prioritize the collection of information on home appliances available in that area. Furthermore, if the user is traveling, the data collection unit can prioritize the collection of information on home appliances available at their travel destination. Additionally, if the user is planning to move, the data collection unit can prioritize the collection of information on home appliances available at their new location. This allows for the efficient collection of highly relevant information by considering the user's geographical location. Some or all of the above processing in the data collection unit may be performed using AI, or without AI. For example, the data collection unit can input the user's geographical location information into a generating AI, which can then collect highly relevant information.

[0097] The data collection unit can analyze the user's social media activity and collect relevant information when collecting information on home appliances. For example, if a user mentions a specific home appliance on social media, the data collection unit will prioritize collecting information related to that appliance. Furthermore, if a user follows a specific home appliance brand on social media, the data collection unit can prioritize collecting information related to that brand. Additionally, if a user frequently posts about a specific home appliance on social media, the data collection unit can prioritize collecting information related to that appliance. This allows for the efficient collection of relevant information by analyzing the user's social media activity. Some or all of the above processing in the data collection unit may be performed using AI or not. For example, the data collection unit can input the user's social media data into a generating AI and have the generating AI collect relevant information.

[0098] The management unit can estimate the user's emotions and adjust the management method based on the estimated emotions. For example, if the user is stressed, the management unit can provide a simple and intuitive management method. If the user is relaxed, the management unit can also provide detailed management options. Furthermore, if the user is excited, the management unit can provide a visually appealing management method. In this way, by adjusting the management method based on the user's emotions, a user-friendly management method can be provided. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the management unit may be performed using AI or not. For example, the management unit can input user emotion data into a generative AI and have the generative AI adjust the management method.

[0099] The management department can adjust the level of detail in its management based on the importance of the appliance information. For example, the management department can prioritize the management of important appliance information (e.g., warranty period, troubleshooting information). It can also manage information on appliances that are frequently used in detail. Furthermore, it can manage information on appliances that experience frequent problems in detail. In this way, by adjusting the level of detail in management based on the importance of the appliance information, important information can be prioritized. Some or all of the above processes in the management department may be performed using AI or not. For example, the management department can input importance data of appliance information into a generating AI and have the generating AI perform the process of adjusting the level of detail in management.

[0100] The management unit can apply different management algorithms depending on the category of home appliance. For example, the management unit can apply a management algorithm that emphasizes usage frequency and maintenance information to kitchen appliances. It can also apply a management algorithm that emphasizes software update information to entertainment appliances. Furthermore, it can apply a management algorithm that emphasizes consumable replacement information to household appliances. By applying different management algorithms depending on the category of home appliance, optimal management can be achieved for each category. Some or all of the above processing in the management unit may be performed using AI, or not. For example, the management unit can input home appliance category data into a generating AI and have the generating AI apply the management algorithm.

[0101] The management unit can estimate the user's emotions and determine management priorities based on the estimated emotions. For example, if the user is stressed, the management unit will prioritize managing information on home appliances that help reduce stress. Similarly, if the user is relaxed, the management unit can prioritize managing information on home appliances that promote relaxation. Furthermore, if the user is agitated, the management unit can prioritize managing information on home appliances that calm agitation. This allows for management tailored to the user's needs by determining management priorities based on their emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processes in the management unit may be performed using AI or not. For example, the management unit can input user emotion data into a generative AI and have the generative AI perform the process of determining management priorities.

[0102] The management department can determine management priorities based on the timing of appliance information submission. For example, the management department may prioritize the management of recently submitted appliance information. It may also prioritize the management of older appliance information. Furthermore, it may prioritize the management of appliance information whose submission dates are concentrated within a specific period. This allows for the prioritization of the latest information by determining management priorities based on the timing of appliance information submission. Some or all of the above processes in the management department may be performed using AI or not. For example, the management department may input appliance information submission timing data into a generating AI and have the generating AI perform the process of determining management priorities.

[0103] The management unit can adjust the order of management based on the relevance of home appliance information. For example, the management unit can prioritize the management of highly relevant home appliance information. It can also postpone the management of less relevant home appliance information. Furthermore, the management unit can group and manage relevant home appliance information. This allows for the prioritization of highly relevant information by adjusting the order of management based on the relevance of home appliance information. Some or all of the above processes in the management unit may be performed using AI or not. For example, the management unit can input relevance data of home appliance information into a generating AI and have the generating AI perform the process of adjusting the order of management.

[0104] The troubleshooting unit can estimate the user's emotions and adjust the troubleshooting method based on the estimated emotions. For example, if the user is stressed, the troubleshooting unit can provide a simple and intuitive troubleshooting method. If the user is relaxed, the troubleshooting unit can also provide detailed troubleshooting options. Furthermore, if the user is agitated, the troubleshooting unit can provide a visually appealing troubleshooting method. In this way, by adjusting the troubleshooting method based on the user's emotions, a user-friendly troubleshooting method can be provided. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the troubleshooting unit may be performed using AI or not. For example, the troubleshooting unit can input user emotion data into a generative AI and have the generative AI adjust the troubleshooting method.

[0105] The troubleshooting unit can select the optimal solution by referring to the appliance's past trouble history during troubleshooting. For example, the troubleshooting unit can analyze the appliance's past trouble history and suggest solutions for similar problems that may occur. The troubleshooting unit can also identify the cause of a problem from the appliance's past trouble history and suggest the optimal solution. Furthermore, the troubleshooting unit can suggest measures to prevent the recurrence of problems based on the appliance's past trouble history. In this way, by referring to the appliance's past trouble history, the optimal solution can be selected and problems can be resolved efficiently. Some or all of the above processes in the troubleshooting unit may be performed using AI, or they may not be performed using AI. For example, the troubleshooting unit can input the appliance's past trouble history data into a generating AI and have the generating AI select the optimal solution.

[0106] The troubleshooting unit can apply different troubleshooting methods depending on the category of home appliance. For example, the troubleshooting unit can apply troubleshooting methods that prioritize usage frequency and maintenance information to kitchen appliances. It can also apply troubleshooting methods that prioritize software update information to entertainment appliances. Furthermore, it can apply troubleshooting methods that prioritize consumable replacement information to household appliances. By applying different troubleshooting methods to each category of appliance, the troubleshooting unit can perform troubleshooting that is optimal for each category. Some or all of the above processes in the troubleshooting unit may be performed using AI or not. For example, the troubleshooting unit can input appliance category data into a generating AI and have the generating AI apply the troubleshooting methods.

[0107] The troubleshooting unit can estimate the user's emotions and determine troubleshooting priorities based on those estimated emotions. For example, if the user is stressed, the troubleshooting unit will prioritize troubleshooting that helps reduce stress. If the user is relaxed, the troubleshooting unit can also prioritize troubleshooting that promotes relaxation. Furthermore, if the user is agitated, the troubleshooting unit can prioritize troubleshooting that calms the agitation. This allows for troubleshooting tailored to the user's needs by prioritizing troubleshooting based on their emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processes in the troubleshooting unit may be performed using AI or not. For example, the troubleshooting unit can input user emotion data into a generative AI and have the generative AI perform the process of determining troubleshooting priorities.

[0108] The troubleshooting unit can select the optimal solution during troubleshooting by considering the geographical distribution of home appliances. For example, if a home appliance is widely used in a particular region, the troubleshooting unit can propose a solution that takes into account the characteristics of that region. Furthermore, if a home appliance is used under specific climatic conditions, the troubleshooting unit can propose a solution that takes those conditions into account. Additionally, if a home appliance is used in a particular cultural area, the troubleshooting unit can propose a solution that takes into account the characteristics of that cultural area. This allows for the selection of the optimal solution tailored to regional characteristics by considering the geographical distribution of home appliances. Some or all of the above-described processes in the troubleshooting unit may be performed using AI, or they may not. For example, the troubleshooting unit can input geographical distribution data of home appliances into a generating AI and have the generating AI select the optimal solution.

[0109] The troubleshooting unit can propose solutions by referring to relevant documentation for home appliances during troubleshooting. For example, the troubleshooting unit can propose the best solution by referring to the home appliance's instruction manual. It can also propose the best solution by referring to the home appliance manufacturer's Q&A site. Furthermore, the troubleshooting unit can propose the best solution by analyzing relevant documentation for home appliances. This allows for efficient troubleshooting by proposing the best solution through the referencing of relevant documentation for home appliances. Some or all of the above processes in the troubleshooting unit may be performed using AI or not. For example, the troubleshooting unit can input data on relevant documentation for home appliances into a generating AI and have the generating AI propose solutions.

[0110] The warranty management unit can estimate the user's emotions and adjust the warranty management method based on the estimated emotions. For example, if the user is stressed, the warranty management unit can provide a simple and intuitive warranty management method. If the user is relaxed, the warranty management unit can also provide detailed warranty management options. Furthermore, if the user is excited, the warranty management unit can provide a visually appealing warranty management method. By adjusting the warranty management method based on the user's emotions, a user-friendly warranty management method can be provided. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the warranty management unit may be performed using AI or not. For example, the warranty management unit can input user emotion data into a generative AI and have the generative AI adjust the warranty management method.

[0111] The warranty management department can select the optimal management method by referring to the past warranty history of home appliances when managing the warranty period. For example, the warranty management department can analyze the past warranty history of home appliances and propose the optimal method for managing the warranty period of similar home appliances. The warranty management department can also propose how to handle cases where the warranty period needs to be extended based on the past warranty history of home appliances. Furthermore, the warranty management department can propose measures to prevent recurrence of warranty period issues based on the past warranty history of home appliances. In this way, by referring to the past warranty history of home appliances, the optimal warranty period management method can be selected and managed efficiently. Some or all of the above processes in the warranty management department may be performed using AI or not. For example, the warranty management department can input past warranty history data of home appliances into a generating AI and have the generating AI select the optimal management method.

[0112] The warranty management unit can estimate the user's emotions and determine the priority of warranty periods based on the estimated emotions. For example, if the user is stressed, the warranty management unit will prioritize managing warranty periods that help reduce stress. Similarly, if the user is relaxed, the warranty management unit can prioritize managing warranty periods that promote relaxation. Furthermore, if the user is agitated, the warranty management unit can prioritize managing warranty periods that calm the agitation. This allows for warranty period management tailored to the user's needs by prioritizing warranty periods based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the warranty management unit may be performed using AI or not. For example, the warranty management unit can input user emotion data into a generative AI and have the generative AI perform the process of determining the priority of warranty periods.

[0113] The warranty management department can select the optimal management method when managing the warranty period, taking into account the geographical location information of the home appliance. For example, if the home appliance is widely used in a particular region, the warranty management department can propose a warranty period management method that takes into account the characteristics of that region. Furthermore, if the home appliance is used under specific climatic conditions, the warranty management department can also propose a warranty period management method that takes into account those climatic conditions. In addition, if the home appliance is used in a particular cultural area, the warranty management department can propose a warranty period management method that takes into account the characteristics of that cultural area. In this way, by considering the geographical location information of the home appliance, the optimal warranty period management method according to regional characteristics can be selected. Some or all of the above processing in the warranty management department may be performed using AI, or it may be performed without using AI. For example, the warranty management department can input the geographical location information data of the home appliance into a generating AI and have the generating AI select the optimal management method.

[0114] The suggestion unit can estimate the user's emotions and adjust the way suggestions are presented based on those emotions. For example, if the user is stressed, the suggestion unit can provide simple and intuitive suggestions. If the user is relaxed, it can also provide detailed suggestion options. Furthermore, if the user is excited, it can provide visually appealing suggestions. By adjusting the way suggestions are presented based on the user's emotions, a user-friendly suggestion method can be provided. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the suggestion unit may be performed using AI or not. For example, the suggestion unit can input user emotion data into a generative AI and have the generative AI adjust the way suggestions are presented.

[0115] The proposal unit can adjust the level of detail in its proposals based on the importance of the appliances. For example, the proposal unit can provide detailed proposals for important appliances (e.g., refrigerators, washing machines). It can also provide detailed proposals for appliances that are frequently used. Furthermore, it can provide detailed proposals for appliances that are prone to problems. By adjusting the level of detail in proposals based on the importance of the appliances, it is possible to provide detailed proposals for important appliances. Some or all of the above processing in the proposal unit may be performed using AI or not. For example, the proposal unit can input appliance importance data into a generating AI and have the generating AI perform the process of adjusting the level of detail in the proposals.

[0116] The suggestion unit can apply different suggestion algorithms depending on the category of the home appliance when making suggestions. For example, the suggestion unit can apply a suggestion algorithm that emphasizes usage frequency and maintenance information to kitchen appliances. It can also apply a suggestion algorithm that emphasizes software update information to entertainment appliances. Furthermore, it can apply a suggestion algorithm that emphasizes consumable replacement information to household appliances. By applying different suggestion algorithms depending on the category of the home appliance, the unit can make optimal suggestions for each category. Some or all of the above processing in the suggestion unit may be performed using AI or not. For example, the suggestion unit can input home appliance category data into a generating AI and have the generating AI apply the suggestion algorithm.

[0117] The suggestion unit can estimate the user's emotions and adjust the length of the suggestions based on those emotions. For example, if the user is in a hurry, the suggestion unit can provide short, concise suggestions. If the user is relaxed, the suggestion unit can provide longer suggestions with more detailed explanations. Furthermore, if the user is excited, the suggestion unit can provide suggestions with visually stimulating effects. By adjusting the length of suggestions based on the user's emotions, suggestions can be tailored to the user's needs. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the suggestion unit may be performed using AI or not. For example, the suggestion unit can input user emotion data into a generative AI and have the generative AI perform the process of adjusting the length of the suggestions.

[0118] The proposal department can determine the priority of proposals based on the submission timing of home appliances. For example, the proposal department can make proposals based on recently submitted home appliance information. It can also make proposals based on older home appliance information. Furthermore, it can make proposals based on home appliance information where submission timings are concentrated within a specific period. By determining the priority of proposals based on the submission timing of home appliances, it is possible to make proposals based on the latest information. Some or all of the above processing in the proposal department may be performed using AI or not. For example, the proposal department can input home appliance submission timing data into a generating AI and have the generating AI perform the process of determining the priority of proposals.

[0119] The suggestion unit can adjust the order of suggestions based on the relevance of the home appliances. For example, the suggestion unit can make suggestions based on highly relevant home appliance information. It can also make suggestions based on less relevant home appliance information. Furthermore, the suggestion unit can group relevant home appliance information and make suggestions based on that group. By adjusting the order of suggestions based on the relevance of the home appliances, it is possible to make suggestions based on highly relevant information. Some or all of the above processing in the suggestion unit may be performed using AI or not. For example, the suggestion unit can input relevance data of home appliance information into a generating AI and have the generating AI perform the process of adjusting the order of suggestions.

[0120] The maintenance unit can estimate the user's emotions and adjust the maintenance schedule based on those emotions. For example, if the user is stressed, the maintenance unit can provide a simple and intuitive maintenance schedule. If the user is relaxed, the maintenance unit can also provide a detailed maintenance schedule. Furthermore, if the user is excited, the maintenance unit can provide a visually appealing maintenance schedule. By adjusting the maintenance schedule based on the user's emotions, a user-friendly maintenance schedule can be provided. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the maintenance unit may be performed using AI or not. For example, the maintenance unit can input user emotion data into a generative AI and have the generative AI adjust the maintenance schedule.

[0121] The maintenance department can create an optimal maintenance schedule by referring to the appliance's past maintenance history. For example, the maintenance department can analyze the appliance's past maintenance history and propose an optimal maintenance schedule for similar appliances. The maintenance department can also propose methods to optimize maintenance frequency based on the appliance's past maintenance history. Furthermore, the maintenance department can propose measures to prevent recurrence of maintenance issues based on the appliance's past maintenance history. This allows for the creation of an optimal maintenance schedule and efficient maintenance by referring to the appliance's past maintenance history. Some or all of the above processes in the maintenance department may be performed using AI or not. For example, the maintenance department can input the appliance's past maintenance history data into a generating AI and have the generating AI create an optimal schedule.

[0122] The maintenance unit can apply different maintenance methods to each category of home appliance when creating maintenance schedules. For example, the maintenance unit can apply maintenance methods that prioritize usage frequency and maintenance information to kitchen appliances. It can also apply maintenance methods that prioritize software update information to entertainment appliances. Furthermore, it can apply maintenance methods that prioritize consumable replacement information to household appliances. By applying different maintenance methods to each category of appliance, optimal maintenance can be performed for each category. Some or all of the above processing in the maintenance unit may be performed using AI or not. For example, the maintenance unit can input appliance category data into a generating AI and have the generating AI apply the maintenance methods.

[0123] The maintenance unit can estimate the user's emotions and determine the priority of the maintenance schedule based on the estimated emotions. For example, if the user is stressed, the maintenance unit will prioritize maintenance that helps reduce stress. Similarly, if the user is relaxed, the maintenance unit can prioritize maintenance that promotes relaxation. Furthermore, if the user is agitated, the maintenance unit can prioritize maintenance that calms the agitation. This allows for maintenance tailored to the user's needs by prioritizing the maintenance schedule based on their emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processes in the maintenance unit may be performed using AI or not. For example, the maintenance unit can input user emotion data into a generative AI and have the generative AI perform the process of determining the priority of the maintenance schedule.

[0124] The maintenance department can create an optimal maintenance schedule by considering the geographical location of the home appliances. For example, if a home appliance is widely used in a particular region, the maintenance department can propose a maintenance schedule that takes into account the characteristics of that region. Furthermore, if a home appliance is used under specific climatic conditions, the maintenance department can propose a maintenance schedule that takes those conditions into account. Additionally, if a home appliance is used in a particular cultural area, the maintenance department can propose a maintenance schedule that takes into account the characteristics of that cultural area. This allows for the creation of an optimal maintenance schedule tailored to regional characteristics by considering the geographical location of the home appliances. Some or all of the above processes in the maintenance department may be performed using AI, or they may not. For example, the maintenance department can input geographical location data of the home appliances into a generating AI and have the generating AI create the optimal schedule.

[0125] The maintenance department can propose a maintenance schedule by referring to relevant documentation for home appliances when creating the schedule. For example, the maintenance department can propose an optimal maintenance schedule by referring to the instruction manual for the home appliance. It can also propose an optimal maintenance schedule by referring to the home appliance manufacturer's Q&A site. Furthermore, the maintenance department can propose an optimal maintenance schedule by analyzing relevant documentation for home appliances. This allows for efficient maintenance by proposing an optimal maintenance schedule through the referencing of relevant documentation for home appliances. Some or all of the above processes in the maintenance department may be performed using AI or not. For example, the maintenance department can input data on relevant documentation for home appliances into a generating AI and have the generating AI propose a schedule.

[0126] The parts suggestion unit can estimate the user's emotions and adjust its parts suggestion method based on those emotions. For example, if the user is stressed, the parts suggestion unit can provide a simple and intuitive parts suggestion method. If the user is relaxed, it can also provide detailed parts suggestion options. Furthermore, if the user is excited, it can provide a visually appealing parts suggestion method. By adjusting the parts suggestion method based on the user's emotions, it is possible to provide a parts suggestion method that is easy for the user to use. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the parts suggestion unit may be performed using AI or not. For example, the parts suggestion unit can input user emotion data into a generative AI and have the generative AI adjust the parts suggestion method.

[0127] The parts proposal department can select the optimal proposal method when proposing parts by referring to the past parts usage history of home appliances. For example, the parts proposal department can analyze the past parts usage history of home appliances and propose the optimal parts proposal method for similar home appliances. The parts proposal department can also propose methods to optimize the frequency of parts replacement based on the past parts usage history of home appliances. Furthermore, the parts proposal department can propose measures to prevent recurrence of parts problems based on the past parts usage history of home appliances. In this way, by referring to the past parts usage history of home appliances, the optimal parts proposal method can be selected and parts can be proposed efficiently. Some or all of the above processes in the parts proposal department may be performed using AI or not. For example, the parts proposal department can input past parts usage history data of home appliances into a generating AI and have the generating AI select the optimal proposal method.

[0128] The parts suggestion department can apply different suggestion methods to each category of home appliance when suggesting parts. For example, for kitchen appliances, the parts suggestion department can apply a parts suggestion method that emphasizes usage frequency and maintenance information. For entertainment appliances, the parts suggestion department can also apply a parts suggestion method that emphasizes software update information. Furthermore, for household appliances, the parts suggestion department can apply a parts suggestion method that emphasizes consumable replacement information. By applying different suggestion methods to each category of home appliance, the department can provide optimal parts suggestions for each category. Some or all of the above processing in the parts suggestion department may be performed using AI, or not. For example, the parts suggestion department can input home appliance category data into a generating AI and have the generating AI apply the suggestion method.

[0129] The parts suggestion unit can estimate the user's emotions and determine the priority of parts suggestions based on the estimated emotions. For example, if the user is feeling stressed, the parts suggestion unit will prioritize parts suggestions that help reduce stress. It can also prioritize parts suggestions that promote relaxation if the user is relaxed. Furthermore, if the user is excited, it can prioritize parts suggestions that calm the excitement. This allows for parts suggestions tailored to the user's needs by prioritizing them based on their emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processes in the parts suggestion unit may be performed using AI or not. For example, the parts suggestion unit can input user emotion data into a generative AI and have the generative AI perform the process of determining the priority of parts suggestions.

[0130] The parts proposal department can select the optimal proposal method when proposing parts, taking into account the geographical location information of the home appliance. For example, if a home appliance is widely used in a particular region, the parts proposal department can propose a parts proposal method that takes into account the characteristics of that region. Furthermore, if a home appliance is used under specific climatic conditions, the parts proposal department can propose a parts proposal method that takes those conditions into account. In addition, if a home appliance is used in a particular cultural area, the parts proposal department can propose a parts proposal method that takes into account the characteristics of that cultural area. In this way, by considering the geographical location information of the home appliance, the optimal parts proposal method can be selected according to regional characteristics. Some or all of the above processing in the parts proposal department may be performed using AI, or it may be performed without using AI. For example, the parts proposal department can input geographical location information data of the home appliance into a generating AI and have the generating AI select the optimal proposal method.

[0131] The parts proposal department can propose a parts proposal method by referring to relevant literature on home appliances when proposing parts. For example, the parts proposal department can propose the optimal parts proposal method by referring to the instruction manual of a home appliance. It can also propose the optimal parts proposal method by referring to the Q&A site of the home appliance manufacturer. Furthermore, the parts proposal department can propose the optimal parts proposal method by analyzing relevant literature on home appliances. In this way, by referring to relevant literature on home appliances, the optimal parts proposal method can be proposed, and parts can be proposed efficiently. Some or all of the above processing in the parts proposal department may be performed using AI or not. For example, the parts proposal department can input data on relevant literature on home appliances into a generating AI and have the generating AI propose a proposal method.

[0132] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0133] The smart home appliance concierge can estimate the user's emotions and adjust the troubleshooting method for home appliances based on those emotions. For example, if the user is stressed, it can provide a simple and intuitive troubleshooting method. If the user is relaxed, it can also provide detailed troubleshooting options. Furthermore, if the user is agitated, it can provide a visually appealing troubleshooting method. By adjusting the troubleshooting method based on the user's emotions, it can provide a user-friendly troubleshooting method. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the troubleshooting unit may be performed using AI or not. For example, the troubleshooting unit can input user emotion data into a generative AI and have the generative AI adjust the troubleshooting method.

[0134] The smart home appliance concierge can monitor the usage of home appliances in real time and automatically initiate troubleshooting if an abnormality is detected. For example, if the temperature of a refrigerator rises rapidly, the system will automatically detect the abnormality, notify the user, and suggest appropriate countermeasures. Similarly, if a washing machine detects abnormal vibrations, the system can automatically initiate troubleshooting and guide the user on how to resolve the issue. Furthermore, if an air conditioner filter is clogged, the system can automatically detect the abnormality and send a notification prompting the user to clean the filter. This allows for early detection of abnormalities and the provision of appropriate countermeasures by monitoring the usage of home appliances in real time. Some or all of the above processes in the troubleshooting unit may be performed using AI or not. For example, the troubleshooting unit can input home appliance usage data into a generating AI, which can then detect abnormalities and suggest countermeasures.

[0135] The smart home appliance concierge can estimate the user's emotions and adjust the appliance maintenance schedule based on those emotions. For example, if the user is stressed, it can provide a simple and intuitive maintenance schedule. If the user is relaxed, it can provide a detailed maintenance schedule. Furthermore, if the user is excited, it can provide a visually appealing maintenance schedule. By adjusting the maintenance schedule based on the user's emotions, it is possible to provide a user-friendly maintenance schedule. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the maintenance unit may be performed using AI or not. For example, the maintenance unit can input user emotion data into the generative AI and have the generative AI adjust the maintenance schedule.

[0136] The smart home appliance concierge can analyze the purchase history of home appliances and select the optimal data collection method. For example, it can analyze the brands and models of home appliances that the user has purchased in the past and prioritize the collection of information on the same brands and models. It can also analyze the frequency of use of home appliances that the user has purchased in the past and prioritize the collection of information on appliances that are used frequently. Furthermore, it can analyze the trouble history of home appliances that the user has purchased in the past and prioritize the collection of information on appliances that have had many problems. In this way, by analyzing the purchase history of home appliances, the optimal data collection method can be selected and information can be collected efficiently. Some or all of the above processing in the collection unit may be performed using AI or not. For example, the collection unit can input home appliance purchase history data into a generating AI and have the generating AI select the optimal data collection method.

[0137] The smart home appliance concierge can estimate the user's emotions and adjust the timing of appliance information collection based on those emotions. For example, if the user is stressed, the collection timing can be delayed to collect information when the user is relaxed. If the user is busy, the collection timing can be adjusted to collect information when the user has free time. Furthermore, if the user is excited, the collection timing can be advanced to collect information when the user has calmed down. In this way, by adjusting the timing of appliance information collection based on the user's emotions, the user's stress can be reduced and information can be collected at the appropriate time. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to these examples. Some or all of the above processing in the collection unit may be performed using AI or not using AI. For example, the collection unit can input user emotion data into the generative AI and have the generative AI adjust the collection timing.

[0138] The smart home appliance concierge can prioritize collecting highly relevant information by considering the user's geographical location when gathering information about home appliances. For example, if the user lives in a specific area, it can prioritize collecting information about home appliances available in that area. Furthermore, if the user is traveling, it can prioritize collecting information about home appliances available at their travel destination. Additionally, if the user is planning to move, it can prioritize collecting information about home appliances available at their new address. This allows for the efficient collection of highly relevant information by considering the user's geographical location. Some or all of the above processing in the collection unit may be performed using AI, or without AI. For example, the collection unit can input the user's geographical location information into a generating AI, which can then collect highly relevant information.

[0139] The smart home appliance concierge can estimate the user's emotions and prioritize the appliance information to collect based on those emotions. For example, if the user is stressed, it can prioritize collecting appliance information that helps reduce stress. If the user is relaxed, it can prioritize collecting appliance information that promotes relaxation. Furthermore, if the user is excited, it can prioritize collecting appliance information that calms excitement. In this way, by prioritizing appliance information based on the user's emotions, it can prioritize the collection of information that meets the user's needs. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the collection unit may be performed using AI or not. For example, the collection unit can input the user's emotion data into the generative AI and have the generative AI perform the processing of determining the priority of information.

[0140] The smart home appliance concierge can collect relevant information by analyzing the user's social media activity when gathering information about home appliances. For example, if a user mentions a specific home appliance on social media, it can prioritize collecting information related to that appliance. Similarly, if a user follows a specific home appliance brand on social media, it can prioritize collecting information related to that brand. Furthermore, if a user frequently posts about a specific home appliance on social media, it can prioritize collecting information related to that appliance. This allows for the efficient collection of relevant information by analyzing the user's social media activity. Some or all of the above processing in the collection unit may be performed using AI or not. For example, the collection unit can input the user's social media data into a generating AI, which can then collect the relevant information.

[0141] The smart home appliance concierge can estimate the user's emotions and adjust the way suggestions are presented based on those emotions. For example, if the user is stressed, it can provide simple and intuitive suggestions. If the user is relaxed, it can provide more detailed suggestion options. Furthermore, if the user is excited, it can provide visually appealing suggestions. By adjusting the way suggestions are presented based on the user's emotions, it can provide suggestions that are easy for the user to use. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the processing described above in the suggestion unit may be performed using AI or not. For example, the suggestion unit can input user emotion data into the generative AI and have the generative AI adjust the way suggestions are presented.

[0142] The smart home appliance concierge can apply different management algorithms to each category of home appliance. For example, kitchen appliances can be managed using an algorithm that prioritizes usage frequency and maintenance information. Entertainment appliances can be managed using an algorithm that prioritizes software update information. Furthermore, household appliances can be managed using an algorithm that prioritizes consumable replacement information. By applying different management algorithms to each category of appliance, optimal management can be achieved for each category. Some or all of the above processing in the management unit may be performed using AI, or not. For example, the management unit can input appliance category data into a generating AI and have the generating AI apply the management algorithm.

[0143] The following briefly describes the processing flow for example form 2.

[0144] Step 1: The data collection unit collects information about home appliances. For example, the data collection unit can collect information about home appliances by taking a picture of the instruction manual cover with a smartphone camera. The data collection unit can also collect information such as the model name, manufacturer, purchase date, and warranty period of the home appliance. In addition, the data collection unit can automatically collect information about home appliances using AI. Step 2: The management department digitally manages the information collected by the collection department. For example, the management department can digitize instruction manuals so that they can be accessed immediately when needed. For example, the management department can use cloud storage to securely store information about home appliances. The management department can also use AI to efficiently manage information about home appliances. Step 3: The troubleshooting department performs troubleshooting based on information managed by the management department. For example, when a user describes the problem by voice, the troubleshooting department can search for appropriate solutions from the instruction manual or the manufacturer's Q&A site and provide voice guidance. For example, the troubleshooting department can use AI to identify the cause of the problem and suggest appropriate solutions. The troubleshooting department can also use AI to automatically guide users through resolving the problem. Step 4: The warranty management department manages the warranty period based on the information managed by the management department. The warranty management department can, for example, send notifications before the warranty period expires. The warranty management department can, for example, use AI to efficiently manage the warranty period. The warranty management department can also use AI to automatically extend or renew the warranty period. Step 5: The Proposal Department makes replacement suggestions based on information managed by the Management Department. For example, the Proposal Department can analyze appliance usage and market information to suggest replacement timing and recommended new products. For example, the Proposal Department can use AI to provide users with optimal replacement suggestions. The Proposal Department can also use AI to automatically notify users when it is time to replace their appliances. Step 6: The maintenance department manages the maintenance schedule based on the information managed by the management department. The maintenance department can, for example, automatically create maintenance schedules for home appliances and notify them of the timing of regular maintenance. The maintenance department can, for example, use AI to perform maintenance on home appliances efficiently. The maintenance department can also use AI to manage the maintenance history of home appliances. Step 7: The parts proposal department provides support for proposing and purchasing spare parts based on information managed by the management department. For example, the parts proposal department can analyze the storage volume and usage status of spare parts for home appliances and propose the necessary spare parts. For example, the parts proposal department can use AI to provide support such as finding suppliers for spare parts and comparing prices. The parts proposal department can also use AI to automatically manage spare parts inventory.

[0145] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0146] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0147] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0148] Each of the multiple elements described above, including the data collection unit, management unit, troubleshooting unit, warranty management unit, proposal unit, maintenance unit, and parts proposal unit, is implemented, for example, by at least one of the smart device 14 and the data processing unit 12. For example, the data collection unit collects information about home appliances using the camera 42 of the smart device 14 and processes that information with the control unit 46A. The management unit digitally manages the collected information with the specific processing unit 290 of the data processing unit 12 and stores it securely using cloud storage. The troubleshooting unit identifies the cause of a problem with the specific processing unit 290 of the data processing unit 12 and proposes an appropriate solution. The warranty management unit manages the warranty period with the specific processing unit 290 of the data processing unit 12 and sends a notification before it expires. The proposal unit analyzes the usage status and market information of home appliances with the specific processing unit 290 of the data processing unit 12 and proposes replacement timing and new products. The maintenance unit creates a maintenance schedule with the specific processing unit 290 of the data processing unit 12 and notifies the user of the timing of regular maintenance. The parts suggestion unit, for example, analyzes the storage quantity and usage status of spare parts using the specific processing unit 290 of the data processing device 12, and suggests the necessary spare parts. The correspondence between each unit and the device or control unit is not limited to the example described above, and various modifications are possible.

[0149] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0150] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0151] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0152] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0153] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0154] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0155] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0156] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0157] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0158] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0159] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0160] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0161] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0162] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0163] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0164] Each of the multiple elements described above, including the data collection unit, management unit, troubleshooting unit, warranty management unit, proposal unit, maintenance unit, and parts proposal unit, is implemented, for example, by at least one of the smart glasses 214 and the data processing unit 12. For example, the data collection unit collects information about home appliances using the camera 42 of the smart glasses 214 and processes that information with the control unit 46A. The management unit digitally manages the collected information with the specific processing unit 290 of the data processing unit 12 and stores it securely using cloud storage. The troubleshooting unit identifies the cause of a problem with the specific processing unit 290 of the data processing unit 12 and proposes an appropriate solution. The warranty management unit manages the warranty period with the specific processing unit 290 of the data processing unit 12 and sends a notification before it expires. The proposal unit analyzes the usage status and market information of home appliances with the specific processing unit 290 of the data processing unit 12 and proposes replacement timing and new products. The maintenance unit creates a maintenance schedule with the specific processing unit 290 of the data processing unit 12 and notifies the user of the timing of regular maintenance. The parts suggestion unit, for example, analyzes the storage quantity and usage status of spare parts using the specific processing unit 290 of the data processing device 12, and suggests the necessary spare parts. The correspondence between each unit and the device or control unit is not limited to the example described above, and various modifications are possible.

[0165] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0166] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0167] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0168] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0169] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0170] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0171] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0172] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0173] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0174] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0175] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0176] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0177] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0178] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0179] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0180] Each of the multiple elements described above, including the data collection unit, management unit, troubleshooting unit, warranty management unit, proposal unit, maintenance unit, and parts proposal unit, is implemented by, for example, at least one of the headset terminal 314 and the data processing unit 12. For example, the data collection unit collects information on home appliances using the camera 42 of the headset terminal 314 and processes that information with the control unit 46A. The management unit digitally manages the collected information with, for example, the specific processing unit 290 of the data processing unit 12 and stores it securely using cloud storage. The troubleshooting unit identifies the cause of a problem with, for example, the specific processing unit 290 of the data processing unit 12 and proposes an appropriate solution. The warranty management unit manages the warranty period with, for example, the specific processing unit 290 of the data processing unit 12 and sends a notification before it expires. The proposal unit analyzes the usage status and market information of home appliances with, for example, the specific processing unit 290 of the data processing unit 12 and proposes replacement timing and new products. The maintenance unit creates a maintenance schedule with, for example, the specific processing unit 290 of the data processing unit 12 and notifies the user of the timing of regular maintenance. The parts suggestion unit, for example, analyzes the storage quantity and usage status of spare parts using the specific processing unit 290 of the data processing device 12, and suggests the necessary spare parts. The correspondence between each unit and the device or control unit is not limited to the example described above, and various modifications are possible.

[0181] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0182] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0183] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0184] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0185] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0186] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0187] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0188] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0189] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0190] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0191] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0192] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0193] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0194] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0195] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0196] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0197] Each of the multiple elements described above, including the collection unit, management unit, troubleshooting unit, warranty management unit, proposal unit, maintenance unit, and parts proposal unit, is implemented by, for example, at least one of the robot 414 and the data processing unit 12. For example, the collection unit collects information on home appliances using the camera 42 of the robot 414 and processes that information with the control unit 46A. The management unit digitally manages the collected information with, for example, the specific processing unit 290 of the data processing unit 12 and stores it securely using cloud storage. The troubleshooting unit identifies the cause of a problem with, for example, the specific processing unit 290 of the data processing unit 12 and proposes an appropriate solution. The warranty management unit manages the warranty period with, for example, the specific processing unit 290 of the data processing unit 12 and sends a notification before it expires. The proposal unit analyzes the usage status and market information of home appliances with, for example, the specific processing unit 290 of the data processing unit 12 and proposes replacement timing and new products. The maintenance unit creates a maintenance schedule with, for example, the specific processing unit 290 of the data processing unit 12 and notifies the user of the timing of regular maintenance. The parts suggestion unit, for example, analyzes the storage quantity and usage status of spare parts using the specific processing unit 290 of the data processing device 12, and suggests the necessary spare parts. The correspondence between each unit and the device or control unit is not limited to the example described above, and various modifications are possible.

[0198] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0199] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0200] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0201] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0202] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0203] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0204] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0205] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0206] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0207] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0208] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0209] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0210] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0211] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0212] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0213] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0214] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0215] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0216] (Note 1) A collection unit that collects information on home appliances, A management unit digitally manages the information collected by the aforementioned collection unit, A troubleshooting unit that performs troubleshooting based on information managed by the aforementioned management unit, A warranty management department manages the warranty period based on the information managed by the aforementioned management department, A proposal department makes replacement proposals based on information managed by the aforementioned management department, A maintenance department manages the maintenance schedule based on the information managed by the aforementioned management department, The system includes a parts proposal unit that proposes and provides support for purchasing spare parts based on information managed by the aforementioned management unit. A system characterized by the following features. (Note 2) The aforementioned collection unit is Information about home appliances is collected by taking a picture of the instruction manual cover with a smartphone camera. The system described in Appendix 1, characterized by the features described herein. (Note 3) The troubleshooting unit described above, When a user describes a problem by voice, the system searches for appropriate solutions in the instruction manual or the manufacturer's Q&A site and provides voice guidance. The system described in Appendix 1, characterized by the features described herein. (Note 4) The aforementioned assurance management department, Send a notification before the warranty period expires. The system described in Appendix 1, characterized by the features described herein. (Note 5) The aforementioned proposal section is, We analyze home appliance usage patterns and market information to suggest the best time to replace appliances and recommend new products. The system described in Appendix 1, characterized by the features described herein. (Note 6) The aforementioned maintenance unit is Automatically creates maintenance schedules for home appliances and notifies users of scheduled maintenance times. The system described in Appendix 1, characterized by the features described herein. (Note 7) The aforementioned parts proposal unit, We analyze the storage volume and usage of spare parts for home appliances, propose necessary spare parts, and provide support such as purchasing options and price comparisons. The system described in Appendix 1, characterized by the features described herein. (Note 8) The aforementioned collection unit is The system estimates the user's emotions and adjusts the timing of collecting home appliance information based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 9) The aforementioned collection unit is We analyze the purchase history of home appliances and select the optimal data collection method. The system described in Appendix 1, characterized by the features described herein. (Note 10) The aforementioned collection unit is When collecting information on home appliances, filtering is performed based on the user's current lifestyle and areas of interest. The system described in Appendix 1, characterized by the features described herein. (Note 11) The aforementioned collection unit is It estimates the user's emotions and determines the priority of home appliance information to collect based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 12) The aforementioned collection unit is When collecting information on home appliances, the system prioritizes collecting highly relevant information by considering the user's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 13) The aforementioned collection unit is When collecting information on home appliances, we analyze users' social media activity and collect relevant information. The system described in Appendix 1, characterized by the features described herein. (Note 14) The aforementioned management department, We estimate the user's emotions and adjust management methods based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 15) The aforementioned management department, Adjust the level of detail in management based on the importance of the home appliance information. The system described in Appendix 1, characterized by the features described herein. (Note 16) The aforementioned management department, Apply different management algorithms depending on the category of home appliance. The system described in Appendix 1, characterized by the features described herein. (Note 17) The aforementioned management department, It estimates user sentiment and determines management priorities based on the estimated user sentiment. The system described in Appendix 1, characterized by the features described herein. (Note 18) The aforementioned management department, Prioritizing management based on when home appliance information is submitted. The system described in Appendix 1, characterized by the features described herein. (Note 19) The aforementioned management department, Adjust the order of management based on the relevance of home appliance information. The system described in Appendix 1, characterized by the features described herein. (Note 20) The troubleshooting unit described above, It estimates the user's emotions and adjusts troubleshooting methods based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 21) The troubleshooting unit described above, During troubleshooting, the system refers to the appliance's past trouble history to select the most appropriate solution. The system described in Appendix 1, characterized by the features described herein. (Note 22) The troubleshooting unit described above, When troubleshooting, apply different troubleshooting methods depending on the category of home appliance. The system described in Appendix 1, characterized by the features described herein. (Note 23) The troubleshooting unit described above, It estimates the user's emotions and prioritizes troubleshooting based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 24) The troubleshooting unit described above, When troubleshooting, the geographical distribution of home appliances should be considered to select the most appropriate solution. The system described in Appendix 1, characterized by the features described herein. (Note 25) The troubleshooting unit described above, When troubleshooting, we refer to relevant literature on home appliances to suggest solutions. The system described in Appendix 1, characterized by the features described herein. (Note 26) The aforementioned assurance management department, We estimate user sentiment and adjust warranty management methods based on the estimated user sentiment. The system described in Appendix 1, characterized by the features described herein. (Note 27) The aforementioned assurance management department, When managing warranty periods, refer to the appliance's past warranty history to select the most suitable management method. The system described in Appendix 1, characterized by the features described herein. (Note 28) The aforementioned assurance management department, We estimate the user's emotions and determine the priority of the warranty period based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 29) The aforementioned assurance management department, When managing the warranty period, the optimal management method is selected by considering the geographical location information of the home appliance. The system described in Appendix 1, characterized by the features described herein. (Note 30) The aforementioned proposal section is, It estimates the user's emotions and adjusts the way suggestions are presented based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 31) The aforementioned proposal section is, When making a proposal, adjust the level of detail based on the importance of the home appliances. The system described in Appendix 1, characterized by the features described herein. (Note 32) The aforementioned proposal section is, When making suggestions, different suggestion algorithms are applied depending on the category of home appliance. The system described in Appendix 1, characterized by the features described herein. (Note 33) The aforementioned proposal section is, It estimates the user's emotions and adjusts the length of the suggestion based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 34) The aforementioned proposal section is, When submitting proposals, prioritize them based on when the home appliances will be submitted. The system described in Appendix 1, characterized by the features described herein. (Note 35) The aforementioned proposal section is, When making a proposal, adjust the order of suggestions based on the relevance of the home appliances. The system described in Appendix 1, characterized by the features described herein. (Note 36) The aforementioned maintenance unit is The system estimates user sentiment and adjusts the maintenance schedule based on the estimated sentiment. The system described in Appendix 1, characterized by the features described herein. (Note 37) The aforementioned maintenance unit is When creating a maintenance schedule, refer to the appliance's past maintenance history to create the optimal schedule. The system described in Appendix 1, characterized by the features described herein. (Note 38) The aforementioned maintenance unit is When creating a maintenance schedule, apply different maintenance methods to each category of home appliance. The system described in Appendix 1, characterized by the features described herein. (Note 39) The aforementioned maintenance unit is The system estimates user sentiment and prioritizes maintenance schedules based on the estimated sentiment. The system described in Appendix 1, characterized by the features described herein. (Note 40) The aforementioned maintenance unit is When creating a maintenance schedule, we take into account the geographical location of the home appliances to create the optimal schedule. The system described in Appendix 1, characterized by the features described herein. (Note 41) The aforementioned maintenance unit is When creating a maintenance schedule, we will propose a schedule by referring to relevant literature on home appliances. The system described in Appendix 1, characterized by the features described herein. (Note 42) The aforementioned parts proposal unit, The system estimates the user's emotions and adjusts the part suggestion method based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 43) The aforementioned parts proposal unit, When proposing parts, the optimal proposal method is selected by referring to the past parts usage history of home appliances. The system described in Appendix 1, characterized by the features described herein. (Note 44) The aforementioned parts proposal unit, When proposing components, different proposal methods are applied depending on the category of home appliance. The system described in Appendix 1, characterized by the features described herein. (Note 45) The aforementioned parts proposal unit, The system estimates the user's emotions and prioritizes component suggestions based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 46) The aforementioned parts proposal unit, When proposing components, the optimal proposal method is selected by considering the geographical location information of the home appliance. The system described in Appendix 1, characterized by the features described herein. (Note 47) The component proposal department proposes a proposal method by referring to relevant literature of home appliances when proposing components The system according to supplementary note 1, characterized in that.

Explanation of symbols

[0217] 10, 210, 310, 410 Data processing system 12 Data processing device 14 Smart device 214 Smart glasses 314 Headset-type terminal 414 Robot

Claims

1. A collection unit that collects information on home appliances, A management unit digitally manages the information collected by the aforementioned collection unit, A troubleshooting unit that performs troubleshooting based on information managed by the aforementioned management unit, A warranty management department manages the warranty period based on the information managed by the aforementioned management department, A proposal department makes replacement proposals based on information managed by the aforementioned management department, A maintenance department manages the maintenance schedule based on the information managed by the aforementioned management department, The system includes a parts proposal unit that proposes and provides support for purchasing spare parts based on information managed by the aforementioned management unit. A system characterized by the following features.

2. The aforementioned collection unit is Information about home appliances is collected by taking a picture of the instruction manual cover with a smartphone camera. The system according to feature 1.

3. The troubleshooting unit described above, When a user describes a problem by voice, the system searches for appropriate solutions in the instruction manual or the manufacturer's Q&A site and provides voice guidance. The system according to feature 1.

4. The aforementioned assurance management department, Send a notification before the warranty period expires. The system according to feature 1.

5. The aforementioned proposal section is, We analyze home appliance usage patterns and market information to suggest the best time to replace appliances and recommend new products. The system according to feature 1.

6. The aforementioned maintenance unit is Automatically creates maintenance schedules for home appliances and notifies users of scheduled maintenance times. The system according to feature 1.

7. The aforementioned parts proposal unit, We analyze the storage volume and usage of spare parts for home appliances, propose necessary spare parts, and provide support such as purchasing options and price comparisons. The system according to feature 1.

8. The aforementioned collection unit is The system estimates the user's emotions and adjusts the timing of collecting home appliance information based on those estimated emotions. The system according to feature 1.

Images