System

Abstract

A system is provided.SOLUTION: This system is provided with a means for allowing a user to upload the warranty certificate of a home electric appliance or a long-term guarantee document, a means for analyzing the uploaded document by using an image recognizing technology, and for extracting information related with guarantee, a means for managing the extracted information for each user, and for providing it as necessary, and a means for monitoring the use circumstances of the home electric appliance or housing environments, and for proposing a proper maintenance time and cleaning method. This system includes a means for performing analysis, and for proposing a self-solution and an inquiry destination, and a means for predicting the optimal replacement time based on the use period and performance deterioration data of a home electric appliance, and for proposing recommended merchandise and the cheapest selling store.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The objective of this project is to reduce the time and effort required for users to manually respond when home appliances break down, by quickly responding to maintenance issues that arise particularly in condominiums and detached houses, and by managing home appliance warranties and proposing optimal replacement timing. It also aims to solve the problem of warranty and long-term compensation documents, which tend to become complicated to manage, making it difficult to quickly obtain the necessary information. Furthermore, due to the lack of regular maintenance and support in the event of a breakdown, a system is needed to centrally manage these and support users' lives. [Means for solving the problem]

[0005] The present invention provides a means for users to upload their home appliance warranties or extended warranty documents, enabling centralized management of warranty and warranty document information. It also includes a means for analyzing uploaded documents using image recognition technology and extracting warranty-related information. Furthermore, it includes a means for managing the extracted information for each user and providing it when needed, allowing users to quickly obtain the information they need. It also includes a means for monitoring the usage status and home environment of the home appliance and suggesting appropriate maintenance times and cleaning methods. This allows users to easily perform daily maintenance. Furthermore, it provides a means for accepting and analyzing malfunction reports from users in the event of a malfunction, and suggesting self-solutions and contact points. This allows for quick and efficient malfunction response. Finally, it includes a means for predicting the optimal replacement time based on the appliance's usage period and performance degradation data, and suggesting recommended products and the lowest-priced retailers. This allows users to replace their home appliances at the appropriate time and make economical choices.

[0006] "User" refers to an individual or household who uses this system to manage home appliances.

[0007] "Home appliances" refers to all electrical appliances that users use in their daily lives.

[0008] A "warranty" is a document provided when purchasing an appliance, which contains information such as the warranty period, conditions, and serial number.

[0009] "Long-term warranty documents" are documents related to additional warranty services provided by electronics retailers and other entities, and include coverage that extends beyond the normal warranty period.

[0010] "Upload" refers to the act of sending digital data of warranty cards and long-term compensation documents held by the user to the system.

[0011] "Image recognition technology" refers to technology for extracting information from digital images, using machine learning and computer vision techniques.

[0012] "Analysis" refers to the process of extracting and structuring the necessary information from uploaded image data.

[0013] "Database" refers to a system for efficiently storing, managing, and retrieving collected information.

[0014] "Monitoring" refers to the act of continuously observing and recording the usage of home appliances and the living environment.

[0015] "Maintenance period" refers to the timing of regular maintenance and cleaning required to keep the home appliance operating properly.

[0016] "Fault reporting" refers to the act of a user notifying the system of a fault in a home appliance.

[0017] "Contact information" refers to the contact information for the manufacturer or retailer's support center that should be contacted if the home appliance breaks down.

[0018] "Period of use" refers to the period from when the appliance was first used to the present.

[0019] "Performance degradation data" refers to data that shows how much the performance of a home appliance has deteriorated over time.

[0020] "Replacement time" refers to the optimal time to purchase a new appliance, taking into account the deterioration and frequency of breakdowns of the appliance.

[0021] "Recommendation" refers to the act of selecting and recommending a specific home appliance to a user.

[0022] "Cheapest store" refers to the store where you can purchase a particular home appliance at the lowest price. [Brief explanation of the drawings]

[0023] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0024] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0025] First, the terms used in the following description will be explained.

[0026] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0027] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0028] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0029] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0030] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0031] [First embodiment]

[0032] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0033] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0034] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0035] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0036] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0037] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0038] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0039] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0040] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0041] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0042] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0043] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0044] As an embodiment of the present invention, a system for managing warranties and long-term warranty documents for home appliances and improving user convenience will be described. This system allows users to upload their warranties and long-term warranty documents for home appliances, and through a series of processes that automatically analyze and manage this information, it supports maintenance and responses in the event of a breakdown and suggests the optimal time to replace the appliance.

[0045] Management of warranties and long-term compensation documents

[0046] First, users upload their appliance warranty or extended warranty documents to the app using their smartphone or tablet. The device receives the uploaded image and analyzes it using image recognition technology (such as OCR). Specifically, it uses an image processing library to extract text information (such as warranty period, serial number, and store information) and generates it as structured data. The extracted data is registered in a database managed by the server. This allows users to check the warranty information at any time through the app.

[0047] Maintenance Notifications and Suggestions

[0048] The device periodically (e.g., daily or weekly) collects usage data from the appliance. In the case of smart appliances, the device communicates with the appliance and automatically acquires the data. The server analyzes this data and predicts when maintenance will be required based on the maintenance cycle of each appliance. When the predicted maintenance period approaches, the server generates a notification and sends it to the device. The notification provides the user with specific instructions and tools for the appropriate maintenance work (e.g., cleaning the filter).

[0049] Support for inquiries in case of breakdowns

[0050] When a home appliance breaks down, the user opens the support function within the app and enters the details of the malfunction into the "Report a Malfunction" form. The device then presents questions to the user and collects details of the malfunction, such as "Is an error message displayed?" The server then analyzes the collected malfunction data and suggests possible self-solutions to the user (e.g., "The drain may be clogged with dust"). If the self-solutions are ineffective, the server also provides contact information for the most appropriate point of contact (the manufacturer's support center or the support desk of the store where the appliance was purchased). This allows the user to quickly and efficiently deal with the malfunction.

[0051] Replacement timing prediction and recommendations

[0052] The server monitors the length of time an appliance has been in use based on its start date, operational data, and failure history. It analyzes usage and error logs to detect signs of performance degradation. Based on the analyzed data, the server applies an algorithm to predict when it is time to replace the appliance. For example, it references statistical data such as "Model X washing machines typically begin to deteriorate after about seven years." When the optimal time to replace the appliance approaches, the server generates a notification and sends it to the device. The notification provides the user with a list of recommended products and retailer information. The product list includes price comparison data and performance evaluations. This allows users to replace their appliances at the appropriate time and make economical choices.

[0053] As described above, the present invention provides a system that efficiently manages warranties and long-term warranty documents for home appliances, supports maintenance and troubleshooting, and suggests the optimal time to replace them. This system allows users to use home appliances with peace of mind, greatly improving the convenience of their lives.

[0054] The processing flow will be explained below.

[0055] Management of warranties and long-term compensation documents

[0056] Step 1:

[0057] Users take a photo of the home appliance warranty or long-term warranty document with their smartphone or tablet and upload it to the app.

[0058] Step 2:

[0059] The device receives the uploaded image and begins analyzing it using OCR (Optical Character Recognition) technology, recognizing text information using an image processing library.

[0060] Step 3:

[0061] The device analyzes the text information extracted from the image (warranty period, serial number, purchase store information, etc.) and converts it into structured data.

[0062] Step 4:

[0063] The server receives the structured data sent from the terminal and registers it in a database for each user.

[0064] Step 5:

[0065] Whenever necessary, the user opens the app and checks the registered warranty information.

[0066] Maintenance Notifications and Suggestions

[0067] Step 1:

[0068] The device periodically (e.g., daily or weekly) collects usage data from the home appliance. In the case of smart home appliances, the device automatically acquires data through communication with the appliance.

[0069] Step 2:

[0070] The device sends collected usage data to a server, including operating time, number of uses, error logs, etc.

[0071] Step 3:

[0072] The server analyzes the data it receives and predicts the next maintenance date based on the maintenance cycle for each appliance.

[0073] Step 4:

[0074] When the server approaches the predicted maintenance time, it generates a maintenance notice and delivers it to the terminal.

[0075] Step 5:

[0076] The device displays maintenance notifications to the user, providing specific work instructions (e.g., cleaning the filter) and the necessary tools.

[0077] Support for inquiries in case of breakdowns

[0078] Step 1:

[0079] When the user recognizes that their home appliance is malfunctioning, they open the support function within the app and enter details of the malfunction into the "Report a Malfunction" form.

[0080] Step 2:

[0081] The device presents the user with questions related to the malfunction and collects specific malfunction information, such as "Is an error message displayed?"

[0082] Step 3:

[0083] The terminal transmits the collected failure data to the server.

[0084] Step 4:

[0085] The server analyzes the failure data and suggests a self-solution to the user (e.g., "It may be due to dust clogging the drain").

[0086] Step 5:

[0087] If the server's self-help efforts are not effective, it will provide contact information for the appropriate point of contact (such as the manufacturer's support center or the support desk at the store where the product was purchased).

[0088] Replacement timing prediction and recommendations

[0089] Step 1:

[0090] The server monitors the period of use of the appliance based on the date of use, operation data, and failure history.

[0091] Step 2:

[0092] The server analyzes the monitoring data to detect signs of performance degradation and also refers to statistical data to predict performance degradation.

[0093] Step 3:

[0094] When the time for replacement approaches, the server generates a notification for the user and sends it to the terminal.

[0095] Step 4:

[0096] The terminal displays a replacement notice to the user and provides a list of recommended products and information on the cheapest retailer.

[0097] Step 5:

[0098] Based on the information presented, the user selects the most suitable home appliance and considers purchasing it.

[0099] The above is a detailed flow of the program processing of this system.

[0100] Example 1

[0101] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0102] Modern homes use many electrical appliances, and it is necessary to properly store their warranties and long-term warranty documents, and efficiently manage their usage, maintenance, and troubleshooting. However, doing this on your own is cumbersome, and missing appropriate maintenance intervals increases the risk of breakdowns, making it difficult to determine the optimal time to replace appliances. For this reason, a system is needed that increases user convenience and streamlines the lifecycle management of electrical appliances.

[0103] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0104] In this invention, the server includes: means for users to upload warranty or extended warranty documents for electrical appliances; means for analyzing the uploaded documents using optical character recognition technology and extracting warranty-related information; means for managing the extracted information for each user and providing it when needed; means for monitoring the usage status and living environment of electrical appliances and suggesting appropriate maintenance times and cleaning methods; means for receiving and analyzing malfunction reports from users in the event of a malfunction and suggesting self-resolved solutions and contact information using a machine learning model; and means for predicting the optimal replacement time based on the duration of use and performance degradation data of the electrical appliances and suggesting recommended products and optimal retailers. This makes it easier for users to manage their warranty and warranty documents, enabling them to perform maintenance and replacement at the appropriate time.

[0105] "Upload" refers to the operation of a user sending a digital image of an electrical product warranty or extended warranty document to the system through the application.

[0106] "Optical character recognition" is a technology for extracting text information from images, specifically OCR (Optical Character Recognition).

[0107] "Warranty information" refers to important data such as the warranty period, serial number, and purchase store information listed on the warranty card or compensation document for an electrical product.

[0108] "Structured data" refers to a data format that is organized so that extracted information can be stored in a table or database.

[0109] "Monitoring" refers to the act of continuously observing the usage of electrical appliances and living environments and collecting necessary data.

[0110] "Maintenance period" refers to the time when an electrical appliance should be inspected, cleaned, or repaired to keep it functioning properly.

[0111] "Fault reporting" refers to the act of a user notifying the system of detailed information about abnormal operation or malfunction of an electrical appliance.

[0112] A "machine learning model" refers to an algorithm that analyzes collected data, finds patterns, and predicts future trends.

[0113] "Performance degradation data" refers to information regarding malfunctions or performance degradation that occur during the use of an electrical product.

[0114] The "optimal replacement time" refers to the point at which the performance of an electrical appliance deteriorates and it becomes economically advantageous to replace it with a new product rather than continuing to repair and maintain it.

[0115] "Recommended products" refer to products that the system recommends as having the same or better performance as the user's current electrical products.

[0116] "Best seller" refers to the store or online shop where you can purchase the recommended products at the best conditions.

[0117] As an embodiment of the present invention, a system for managing warranty certificates and long-term warranty documents for electrical appliances and improving user convenience will be described. This system allows users to upload warranty certificates and long-term warranty documents for electrical appliances, and through a series of processes that automatically analyze and manage this information, it supports maintenance and responses in the event of a breakdown and suggests the optimal time to replace the appliance.

[0118] Management of warranties and long-term compensation documents

[0119] First, users upload the warranty card or extended warranty document for their electrical appliances to the app using a smartphone or tablet. The device receives the uploaded image and analyzes it using image recognition technology (such as OCR). Specifically, an image processing library is used to extract text information (such as warranty period, serial number, and store information) and generate it as structured data. The extracted data is then registered in a database managed by the server. This allows users to check the warranty information at any time through the app.

[0120] Maintenance Notifications and Suggestions

[0121] The device periodically (e.g., daily or weekly) collects appliance usage data. In the case of smart home appliances, the device communicates with the appliance to automatically obtain the data. The server analyzes this data and predicts when maintenance is required based on the maintenance cycle of each appliance. When the predicted maintenance period approaches, the server generates a notification and sends it to the device. The notification provides the user with specific instructions and tools for the appropriate maintenance work (e.g., cleaning the filter).

[0122] Support for inquiries in case of breakdowns

[0123] When an electrical appliance breaks down, the user opens the support function within the app and enters the details of the malfunction into the "Report a malfunction" form. The device then presents questions to the user and collects details of the malfunction, such as "Is an error message displayed?" The server then analyzes the collected malfunction data and suggests possible self-solutions to the user (e.g., "The drain may be clogged with dust"). If the self-solutions are ineffective, the server also provides contact information for the most appropriate point of contact (the manufacturer's support center or the support desk of the store where the appliance was purchased). This allows the user to quickly and efficiently resolve the malfunction.

[0124] Replacement timing prediction and recommendations

[0125] The server monitors the length of time an appliance has been in use based on its start date, operational data, and fault history. It analyzes usage and error logs to detect signs of performance degradation. Based on the analyzed data, the server applies an algorithm to predict when it's time to replace it. For example, it references statistical data such as "Model X refrigerators typically begin to deteriorate after about seven years." When the optimal time to replace an appliance approaches, the server generates a notification and sends it to the device. The notification provides the user with a list of recommended products and retailer information. The product list includes price comparison data and performance ratings. This allows users to replace appliances at the appropriate time and make economical choices.

[0126] Specific examples

[0127] A user uploads a refrigerator warranty, and the device extracts that the warranty period is "May 1, 2023 to May 1, 2025." The server predicts when the washing machine filter needs cleaning and notifies the user, "Please clean the filter." When a user reports error code "E05," the server suggests a self-solution, saying, "Check the drain hose." When seven years have passed since the refrigerator was first used, the server notifies the user, "It's time to replace it," and displays a list of recommended new refrigerators.

[0128] Prompt Sentence Examples

[0129] "Just take a picture of your appliance's warranty card with your camera and upload it to the app. The warranty period and serial number will be automatically extracted."

[0130] "It collects data every week and notifies you when it's time to maintain your appliances. It lets you know when it's time to clean your filters."

[0131] "If your appliance breaks down, please report the problem through the app and we will suggest a solution for you."

[0132] "It analyzes the usage period of home appliances and notifies you of the optimal time to replace them. It also provides a list of recommended products."

[0133] As described above, the present invention provides a system that efficiently manages warranties and long-term warranty documents for electrical appliances, supports maintenance and troubleshooting, and suggests the optimal time to replace them. This allows users to use electrical appliances with peace of mind, greatly improving the convenience of their lives.

[0134] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0135] Step 1:

[0136] User uploads warranty card

[0137] A user uploads an image of the warranty or extended warranty document for an electrical appliance using an app on their smartphone or tablet. Specifically, the user taps the camera button in the app to launch the camera and take a photo of the warranty or select an existing image.

[0138] Input: Warranty card image taken or selected by the user

[0139] Output: Uploaded warranty image

[0140] Step 2:

[0141] The device receives the image

[0142] The device receives the image uploaded by the user and prepares it for image recognition processing.

[0143] Input: Uploaded warranty image

[0144] Output: warranty card image ready for analysis

[0145] Step 3:

[0146] The device analyzes the image

[0147] The device performs preprocessing using an image processing library (e.g., OpenCV). This includes adjusting the resolution, removing noise, and identifying text regions. It then uses an OCR engine (e.g., Tesseract) to extract text from the image. This allows information such as the warranty period, serial number, and store information to be obtained.

[0148] Input: warranty card image ready for analysis

[0149] Data processing: image resolution adjustment, noise removal, text area identification

[0150] Data Calculation: Text Extraction Using an OCR Engine

[0151] Output: Extracted text data

[0152] Step 4:

[0153] The device structures the data

[0154] The text data extracted by the device is converted into structured data such as JSON format. Specifically, the warranty period, serial number, and purchase store information are stored in the appropriate fields.

[0155] Input: Extracted text data

[0156] Data processing: structuring text data into JSON format

[0157] Output: Structured data in JSON format

[0158] Step 5:

[0159] The device sends data to the server

[0160] The device sends the structured data to the server using HTTPS as the communication protocol.

[0161] Input: Structured data in JSON format

[0162] Output: Structured data sent to the server

[0163] Step 6:

[0164] The server stores the data

[0165] The server stores the received structured data in a database management system (e.g. MySQL). The server first checks the data for errors, and if there are no errors, stores it in the database in the appropriate format.

[0166] Input: Structured data sent to the server

[0167] Data processing: Checking data for errors and converting it to the appropriate format

[0168] Output: Data stored in the database

[0169] Step 7:

[0170] Device collects usage data

[0171] The device periodically collects usage data from the appliance. In the case of smart appliances, the device communicates with the appliance to automatically obtain data, including operating time and error logs.

[0172] Input: Usage data collected from electronic devices

[0173] Output: Usage data stored on the device

[0174] Step 8:

[0175] Server maintenance forecast

[0176] The server analyzes the usage data sent from the device, calculates the maintenance cycle using an analysis tool (e.g., Pandas), and calculates the predicted maintenance time.

[0177] Input: Usage data stored on the device

[0178] Data calculation: Usage data analysis, maintenance cycle calculation

[0179] Output: Estimated maintenance time

[0180] Step 9:

[0181] Server generates maintenance notice

[0182] The server generates a notification based on the predicted maintenance time, specifically a message containing the maintenance procedure and required tools.

[0183] Input: Estimated maintenance time

[0184] Data processing: Notification message generation

[0185] Output: The generated notification message

[0186] Step 10:

[0187] The server sends a notification

[0188] The server generates a notification message and sends it to the device, which receives it and displays it to the user as a push notification.

[0189] Input: The generated notification message

[0190] Output: Notification message sent to the terminal

[0191] Step 11:

[0192] User reports a malfunction

[0193] The user accesses the support function within the app and enters detailed information about the problem into the "Report a Problem" form.

[0194] Input: The details of the malfunction entered in the "Malfunction Report" form

[0195] Output: The entered fault report data

[0196] Step 12:

[0197] The device collects failure data

[0198] The device asks the user additional questions to gather more detailed information about the failure, such as "Is an error message displayed?"

[0199] Input: Fault report data and responses to additional questions

[0200] Output: Collected detailed failure data

[0201] Step 13:

[0202] The server proposes a self-solution

[0203] The device sends the collected failure data to a server, which then analyzes the data using a machine learning model and proposes a self-solution to the user.

[0204] Input: Collected detailed failure data

[0205] Data calculation: failure data analysis, self-resolved solution generation

[0206] Output: Self-solution proposal

[0207] Step 14:

[0208] The server provides contact information

[0209] If self-help measures are not effective, the server provides contact information for the best place to contact (the manufacturer's support center or the support desk at the store where you purchased the product).

[0210] Input: Failure data and self-solving results

[0211] Output: Best contact information

[0212] Step 15:

[0213] The server monitors the usage period

[0214] The server monitors the electrical appliances based on their start date of use, operation data, and failure history.

[0215] Input: Start date of use, operation data, failure history

[0216] Output: Monitoring result data

[0217] Step 16:

[0218] Server replacement time prediction

[0219] The server analyzes the monitoring results data and uses data analysis tools and algorithms to predict when it is time to replace the device.

[0220] Input: Monitoring result data

[0221] Data calculation: Performance degradation analysis, replacement timing prediction

[0222] Output: Predicted replacement time

[0223] Step 17:

[0224] The server generates a recommendation notification

[0225] When the time for replacement approaches, the server generates a notification message including a list of recommended products and information on the best retailer.

[0226] Input: Predicted replacement time

[0227] Data processing: Generation of recommended product list and retailer information

[0228] Output: Recommendation notification message

[0229] Step 18:

[0230] The server sends a notification

[0231] The server generates a recommendation notification and sends it to the device, which receives it and displays it to the user as a push notification.

[0232] Input: Generated recommendation notification message

[0233] Output: Recommendation notification message sent to the device

[0234] (Application example 1)

[0235] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0236] Users often have difficulty managing or lose their home appliance warranties and long-term warranty documents, making it difficult to quickly check the warranty details when needed. Furthermore, notifications regarding home appliance maintenance schedules and troubleshooting are not provided in a timely manner, reducing user convenience. Furthermore, it is difficult for users to properly determine when to replace their home appliances, which can lead to financial losses if they miss the optimal timing. These issues need to be resolved.

[0237] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0238] In this invention, the server includes: means for users to upload warranty or extended warranty documents for electrical appliances; means for analyzing the uploaded documents using image recognition technology and extracting warranty-related information; means for managing the extracted information for each user and providing it as needed; means for monitoring the usage status and installation environment of the electrical appliance and suggesting appropriate maintenance times and cleaning methods; means for receiving and analyzing malfunction reports from users and suggesting self-resolved solutions and contact information; means for predicting the optimal replacement time based on the electrical appliance's usage period and performance degradation data and suggesting recommended products and the lowest-priced retailers; means for extracting warranty information from warranty images using a smartphone and storing it in a database; means for providing maintenance notifications and support in the event of a malfunction; and means for predicting the replacement time based on the start date of use and usage data and providing a list of recommended products. This allows users to efficiently manage warranty and warranty documents, receive timely notifications for maintenance and malfunction response, and use electrical appliances economically and efficiently by knowing the optimal replacement time.

[0239] "Electrical appliances" is a general term for devices and equipment that require electrical power for use.

[0240] A "warranty" is a document stating that the manufacturer or seller guarantees the quality and performance of a product for a certain period of time.

[0241] An "extended warranty document" is a contract that guarantees the repair or replacement of a product for a period beyond the normal warranty period.

[0242] "User" means any person or entity that uses a system or device.

[0243] "Image recognition technology" is a general term for technology that analyzes and extracts specific information from images, and generally includes OCR (optical character recognition).

[0244] "Uploading" is the act of sending data from a local device to a network or server.

[0245] "Warranty information" is data about warranty details such as the warranty period, serial number, and purchase store information, which are written on the warranty card.

[0246] "Extraction" refers to the process of extracting necessary information from images or text.

[0247] "Management" is the act of controlling the organization, storage, access, and updating of information.

[0248] "Providing" refers to supplying information or services required by users.

[0249] "Monitoring" is the process of continuously observing and recording the state of a system or device.

[0250] "Maintenance period" refers to the period when a device or equipment requires regular maintenance, inspection, or repair.

[0251] "Cleaning method" refers to the selection of procedures and materials for cleaning equipment or devices.

[0252] "Fault reporting" is the act of a user notifying a system operator of a fault or problem in a device or system.

[0253] "Analysis" is the process of examining and evaluating data to understand its structure and meaning.

[0254] "Self-help" is a method by which a user can solve a problem on their own without professional support.

[0255] "Contact Information" refers to the location or person that a user should contact when seeking problem resolution or support.

[0256] "Period of use" refers to the period during which a device or system is actually used.

[0257] "Performance degradation data" is data that records how the performance of a device or system deteriorates over time.

[0258] "Replacement time" is the best time to replace an existing device or system with a new one.

[0259] A "recommended product" is a product that the system determines to be most appropriate in a particular situation or under a particular condition, based on specific user needs.

[0260] "Best Dealer" refers to the dealer that offers the lowest price for a particular product.

[0261] A "smartphone" is a type of mobile phone, a handheld device that combines advanced computing power and communication capabilities.

[0262] A database is a system that organizes and stores information and allows it to be retrieved in any format required.

[0263] "Date of Use" means the date on which a device or system is first used.

[0264] As an embodiment of this invention, we will explain the "Home Appliance Security Support" system. This system allows users to manage warranties and long-term warranty documents for electrical appliances, and efficiently recommends maintenance, troubleshooting, and optimal replacement timing. The main hardware and software that make up this system are described below.

[0265] Hardware

[0266] Smartphone

[0267] server

[0268] software

[0269] Python

[0270] OpenCV

[0271] Tesseract OCR

[0272] SQLite

[0273] Requests

[0274] Processing flow

[0275] 1. Upload and analyze the warranty card

[0276] Users use their smartphones to take photos of electrical appliance warranties and long-term warranty documents and upload them to the app. The device receives the uploaded images and analyzes them using image recognition technology (OCR).

[0277] Specifically, OpenCV and Tesseract OCR are used to extract text information (warranty period, serial number, purchase store information, etc.) from uploaded images and generate it as structured data.

[0278] The extracted data is registered in a database (SQLite) managed by the server, allowing users to check warranty information at any time via their smartphone.

[0279] 2. Maintenance Notifications and Suggestions

[0280] The server continuously monitors the usage status and installation environment of electrical appliances to predict maintenance times and cleaning methods.

[0281] When maintenance is due, the server generates a notification and sends it to the smartphone, including specific instructions for the appropriate maintenance task (e.g., cleaning the filter) and the tools required.

[0282] 3. Support in case of breakdown

[0283] When an electrical appliance breaks down, users can use the support function in the smartphone app to "report the malfunction."

[0284] The device asks the user questions to gather details about the failure, such as "Are you seeing any error messages?"

[0285] The server analyzes the collected failure data and suggests possible solutions to the user (for example, "The drain may be clogged with dust.") If the solutions are ineffective, the server provides the user with the best contact point (the manufacturer's support center).

[0286] 4. Replacement timing prediction and recommendations

[0287] The server monitors the period of use of the appliance based on the date of use, operation data, and failure history.

[0288] The system analyzes usage and error logs to detect signs of performance degradation and applies an algorithm to predict when it's time to replace the device. Users are notified of the optimal time to replace the device via email or in-app notifications.

[0289] The notification will also include a list of recommended products, including performance evaluations and price comparison data, as well as retailer information.

[0290] Examples and prompts

[0291] Specific examples

[0292] Users simply take a photo of their washing machine's warranty card with their smartphone and upload it. The app analyzes the warranty information and registers it in a database. When seven years have passed since the purchase date, the app displays a replacement notification and provides a list of recommended products.

[0293] Prompt Sentence Examples

[0294] "Please implement a program to extract warranty information from uploaded images and register the warranty period and serial number in a database. After a certain period of time, send a maintenance notification or suggest a replacement."

[0295] In this way, the "Home Appliance Security Support" system aims to enable users to efficiently manage warranties and compensation documents, receive timely notifications about maintenance and malfunctions, and know the optimal time to replace their electrical appliances, thereby enabling them to use their appliances economically and efficiently.

[0296] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0297] Step 1:

[0298] Warranty upload and analysis

[0299] A user uses a smartphone to take a photo of an electronic product's warranty or extended warranty document and upload it to the app. The image taken by the user is provided to the app as input. The device receives this and uses OpenCV and Tesseract OCR to extract text information from the image (warranty period, serial number, purchase store information, etc.). The output is structured data extracted from the warranty, which is sent to the server.

[0300] Step 2:

[0301] Saving to a database

[0302] The server stores the structured data received from step 1 (warranty period, serial number, purchase store information, etc.) in an SQLite database. As input, the structured data is given to the server, and an insert operation is performed into the database. The output is the warranty information registered in the database.

[0303] Step 3:

[0304] Maintenance forecast and notification

[0305] The server predicts when maintenance is required based on the appliance's usage and installation environment. Warranty information and usage data stored in the database are used as input. Based on this, the server predicts when the next maintenance task (e.g., filter cleaning) will be performed. When the maintenance deadline approaches, the server generates a notification and distributes it to the device. The output is a maintenance notification that is displayed on the device.

[0306] Step 4:

[0307] Fault reporting and support suggestions

[0308] When a user reports a malfunction of an electrical appliance, they use the support function within the smartphone app to make a "fault report." The user's report is provided to the app as input. The device then presents questions to the user and collects detailed information about the malfunction. Based on the information entered, the server performs an analysis and suggests a self-solution (e.g., a possible dust clog in the drain). If the self-solution is insufficient, it also provides information on the most appropriate contact point (the manufacturer's support center). The output is the self-solution presented to the user and the contact information.

[0309] Step 5:

[0310] Replacement timing prediction and recommendations

[0311] The server predicts when to replace a home appliance based on the date it was first used, its operation data, and its failure history. The inputs include warranty information, usage status, and error logs stored in a database. The server analyzes these to detect signs of performance degradation and determine the optimal time to replace the appliance. When the time to replace the appliance approaches, the server generates a notification and sends it to the device along with a list of recommended products. The output is a replacement notification and a list of recommended products to the user.

[0312] This series of processes enables users to efficiently manage warranties and compensation documents, receive timely notifications for maintenance and malfunctions, and know the optimal time to replace their electrical appliances, enabling them to use them economically and efficiently.

[0313] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0314] This invention is a system that manages home appliance warranties and long-term warranty documents to improve user convenience, and also combines an emotion engine to provide support tailored to the user's emotional state. This system allows users to upload their home appliance warranties and long-term warranty documents, and through a series of processes that automatically analyze and manage this information, it supports maintenance and responses in the event of a breakdown, and suggests the optimal time to replace the appliance. Furthermore, by using an emotion engine that recognizes the user's emotions, it provides more appropriate support.

[0315] Management of warranties and long-term compensation documents

[0316] First, users upload their appliance warranty or extended warranty documents to the app using their smartphone or tablet. The device receives the uploaded image and analyzes it using OCR (Optical Character Recognition) technology. Specifically, it uses an image processing library to extract text information (such as warranty period, serial number, and purchase store information) and generates it as structured data. The extracted data is registered in a database managed by the server. This allows users to easily check warranty information at any time through the app.

[0317] Maintenance Notifications and Suggestions

[0318] The device periodically (e.g., daily or weekly) collects usage data from the appliance. In the case of smart appliances, the device communicates with the appliance and automatically acquires the data. The server analyzes this data and predicts when maintenance will be required based on the maintenance cycle of each appliance. When the predicted maintenance period approaches, the server generates a notification and sends it to the device. The notification provides the user with specific instructions and tools for the appropriate maintenance work (e.g., cleaning the filter).

[0319] Support for inquiries in case of breakdowns

[0320] When a home appliance breaks down, the user opens the support function within the app and enters the details of the malfunction into the "Report a Malfunction" form. The device then presents questions to the user and collects details of the malfunction, such as "Is an error message displayed?" The device then sends the collected malfunction data to the server. The server analyzes the malfunction data and suggests possible self-solutions to the user (e.g., "The drain may be clogged with dust"). If the self-solutions are ineffective, the server also provides contact information for the best point of contact (the manufacturer's support center or the support desk of the store where the appliance was purchased). This allows the user to quickly and efficiently handle the malfunction.

[0321] Replacement timing prediction and recommendations

[0322] The server monitors the length of time an appliance has been in use based on its start date, operational data, and failure history. It analyzes usage and error logs to detect signs of performance degradation. Based on the analyzed data, the server applies an algorithm to predict when it is time to replace the appliance. For example, it references statistical data such as "Model X washing machines typically begin to deteriorate after about seven years." When the optimal time to replace the appliance approaches, the server generates a notification and sends it to the device. The notification provides the user with a list of recommended products and retailer information. The product list includes price comparison data and performance evaluations. This allows users to replace their appliances at the appropriate time and make economical choices.

[0323] User support using an emotion engine

[0324] If an emotion engine is built into the system, it can recognize the user's emotional state and provide more effective support. For example, when a user uses the in-app support feature after a home appliance breaks down, the emotion engine can determine the user's level of stress or anxiety from their facial expression, voice, or input.

[0325] Examples:

[0326] One day, the washing machine stops working properly, so the user uses the app to report the malfunction. The device analyzes the user's facial expressions and tone of voice and uses an emotion engine to determine that the user's stress level is high. In this case, the server quickly sets the support priority high and offers the user a self-solution using gentle language or the fastest support option. The emotion engine also takes the user's emotions into consideration when sending maintenance notifications, adjusting them to be sent at the appropriate time. This reduces the user's psychological burden and enables thorough support.

[0327] As described above, this invention is a system that efficiently manages warranties and long-term warranty documents for home appliances, supports maintenance and troubleshooting, and suggests the optimal time to replace the appliance. Furthermore, by utilizing an emotion engine, it provides support according to the user's emotional state. This system allows users to use home appliances with peace of mind, improving their quality of life.

[0328] The processing flow will be explained below.

[0329] Management of warranties and long-term compensation documents

[0330] Step 1:

[0331] Users take a photo of the home appliance warranty or long-term warranty document with their smartphone or tablet and upload it to the app.

[0332] Step 2:

[0333] The device receives the uploaded image and begins analyzing it using OCR (Optical Character Recognition) technology, recognizing text information using an image processing library.

[0334] Step 3:

[0335] The device analyzes the text information extracted from the image (warranty period, serial number, purchase store information, etc.) and converts it into structured data.

[0336] Step 4:

[0337] The server receives the structured data sent from the terminal and registers it in a database for each user.

[0338] Step 5:

[0339] Whenever necessary, the user opens the app and checks the registered warranty information.

[0340] Maintenance Notifications and Suggestions

[0341] Step 1:

[0342] The device periodically (e.g., daily or weekly) collects usage data from the home appliance. In the case of smart home appliances, the device automatically acquires data through communication with the appliance.

[0343] Step 2:

[0344] The device sends collected usage data to a server, including operating time, number of uses, error logs, etc.

[0345] Step 3:

[0346] The server analyzes the data it receives and predicts the next maintenance date based on the maintenance cycle for each appliance.

[0347] Step 4:

[0348] When the server approaches the predicted maintenance time, it generates a maintenance notice and delivers it to the terminal.

[0349] Step 5:

[0350] The device displays maintenance notifications to the user, providing specific work instructions (e.g., cleaning the filter) and the necessary tools.

[0351] Support for inquiries in case of breakdowns

[0352] Step 1:

[0353] When the user recognizes that their home appliance is malfunctioning, they open the support function within the app and enter details of the malfunction into the "Report a Malfunction" form.

[0354] Step 2:

[0355] The device asks the user questions related to the malfunction and collects specific malfunction information, such as "Is an error message displayed?"

[0356] Step 3:

[0357] The terminal transmits the collected failure data to the server.

[0358] Step 4:

[0359] The server analyzes the failure data and suggests a self-solution to the user (e.g., "It may be due to dust clogging the drain").

[0360] Step 5:

[0361] If the server's self-help efforts are not effective, it will provide contact information for the appropriate point of contact (such as the manufacturer's support center or the support desk at the store where the product was purchased).

[0362] Replacement timing prediction and recommendations

[0363] Step 1:

[0364] The server monitors the period of use of the appliance based on the date of use, operation data, and failure history.

[0365] Step 2:

[0366] The server analyzes the monitoring data to detect signs of performance degradation and also refers to statistical data to predict performance degradation.

[0367] Step 3:

[0368] When the time for replacement approaches, the server generates a notification for the user and sends it to the terminal.

[0369] Step 4:

[0370] The terminal displays a replacement notice to the user and provides a list of recommended products and information on the cheapest retailer.

[0371] Step 5:

[0372] Based on the information presented, the user selects the most suitable home appliance and considers purchasing it.

[0373] User support using an emotion engine

[0374] Step 1:

[0375] When a user reports a malfunction of a home appliance within the app, the device captures the user's facial expressions, voice, or input and sends it to the emotion engine.

[0376] Step 2:

[0377] The emotion engine analyzes the user's emotional data to determine their stress and anxiety levels.

[0378] Step 3:

[0379] The server receives emotional data from the emotion engine and adjusts the response based on the user's emotional state: if stress levels are high, prompt support is prioritized.

[0380] Step 4:

[0381] The server suggests ways to respond based on the user's emotional state. For example, if the user is under high stress, it will respond with gentle words and provide a quick solution.

[0382] Step 5:

[0383] The terminal displays the response methods and specific solutions received from the server to the user, and the user solves the problem according to the most appropriate response method.

[0384] Examples:

[0385] Example 1:

[0386] Step 1: The user takes a photo of the washing machine warranty card with their smartphone and uploads it to the app.

[0387] Step 2: The device analyzes the image and extracts the warranty period and serial number.

[0388] Step 3: The server stores the extracted data in the database, and the user can check the warranty information when needed.

[0389] Example 2:

[0390] Step 1: When the washing machine breaks down, the user reports the problem through the app. The device sends the user's tone of voice to the emotion engine.

[0391] Step 2: The emotion engine determines that the user's stress level is high.

[0392] Step 3: The server provides prompt support and uses kind words to ease stress.

[0393] Step 4: The device will display the self-help solution and contact information for the support center to the user.

[0394] Example 2

[0395] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0396] In recent years, many household electrical appliances have been used, but managing the warranties and long-term warranty documents for these appliances has become cumbersome, making it difficult to know when to respond to breakdowns and when to perform maintenance. As a result, users are often unable to take appropriate action within the warranty period, resulting in unnecessary expenses and stress. Furthermore, not knowing when to replace an appliance can lead to problems such as a decline in efficiency due to the extended use of degraded appliances. Furthermore, there is a need to reduce the stress users feel when an appliance breaks down and provide effective support, but no system exists to achieve this.

[0397] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0398] In this invention, the server includes: a means for users to upload warranty or extended warranty documents for household electrical appliances; a means for analyzing the uploaded documents using optical character recognition technology and extracting warranty-related information; a means for managing the extracted information for each user and providing it as needed; a means for monitoring the usage status of the household electrical appliances and suggesting appropriate maintenance and cleaning methods; a means for receiving and analyzing malfunction reports from users and suggesting self-resolved solutions and contact information; a means for predicting the optimal replacement timing based on the usage period and performance degradation data of the household electrical appliances and suggesting recommended products and retailers; and a means for recognizing the user's emotional state using emotion analysis technology, adjusting support priorities, and providing support tailored to the user. This allows users to efficiently manage their warranty and warranty documents, ensure appropriate maintenance and malfunction response, and know the optimal replacement timing. Furthermore, providing support tailored to the user's emotions can reduce stress and help them live a more comfortable life.

[0399] "User" means a person or entity that owns a household electrical appliance and provides information to manage warranty and extended warranty documents within the system.

[0400] "Household electrical appliances" refers to electrical appliances used in the home, such as washing machines, refrigerators, vacuum cleaners, etc.

[0401] A "warranty" is a document issued when purchasing household electrical appliances, which lists the product's warranty period and scope.

[0402] "Extended warranty documents" refer to documents that provide additional warranty services or extended warranties in addition to the warranty certificate.

[0403] "Means for uploading" refers to a function that allows a user to use a terminal to send image data of a warranty or long-term compensation document to the system.

[0404] "Optical character recognition technology (OCR)" refers to the technology that analyzes character information in an image and converts it into text data.

[0405] "Means of extraction" refers to the process of extracting necessary information (e.g., warranty period, serial number) from the analyzed data and registering it in the database.

[0406] "Monitoring means" refers to the function of regularly collecting and analyzing usage and performance data of household electrical appliances.

[0407] "Means to suggest maintenance timing" refers to a function that predicts when maintenance of household electrical appliances is necessary based on collected data and notifies the user.

[0408] The "means for receiving a fault report" refers to an input interface that allows a user to report a fault in a household electrical appliance to the system.

[0409] "Means for proposing self-resolving solutions" refers to the process by which the system analyzes failure data and provides the user with a method for self-resolving the problem.

[0410] "A means of predicting the time to replace an appliance based on the period of use and performance degradation data" refers to a function that analyzes the usage history and performance data of household electrical appliances and calculates the optimal time to replace them.

[0411] "Means for suggesting recommended products and retailers" refers to a function that provides the user with appropriate purchasing options and retailer information when it is time to replace their product.

[0412] "Emotion analysis technology" refers to technology that recognizes a user's emotional state from their facial expression, voice, or input content.

[0413] "Means for adjusting support priority" refers to a process for adjusting the urgency and method of support response based on the user's emotional state.

[0414] This system efficiently manages warranties and long-term warranty documents for users' home electrical appliances, and suggests appropriate maintenance, troubleshooting, and replacement timing. Furthermore, by using emotion analysis technology, it can provide customized support according to the user's emotional state.

[0415] Warranty and long-term coverage document management

[0416] Users use their smartphones or tablets to upload warranty and extended warranty documents for home appliances to the application. The device receives the uploaded image and analyzes the text information in the image using OCR (optical character recognition) technology (e.g., Google Cloud Vision API). At this time, an image processing library (e.g., OpenCV) is used to preprocess the image. The analyzed data (e.g., warranty period, serial number, and purchase store information) is structured and sent from the device to the server. The server connects to the database and inserts the received data into the appropriate tables. This allows users to check the warranty information at any time through the app.

[0417] Maintenance Notification

[0418] The device periodically collects usage data for home appliances. For example, in the case of smart home appliances, the device automatically collects data using Wi-Fi or Bluetooth and sends it to a server. The server analyzes the collected data and predicts when maintenance is required based on the appliance's maintenance cycle. When the predicted maintenance period approaches, the server generates a notification and sends it to the device. The notification includes specific maintenance procedures (e.g., filter cleaning) and information on tools.

[0419] Support for inquiries in case of breakdowns

[0420] When a user reports a malfunction of a home electrical appliance, they use the support function within the app to enter the details of the malfunction into the "Malfunction Report" form. The device asks the user questions (e.g., "Is an error message displayed?") and collects detailed information about the malfunction. The collected information is sent from the device to the server. The server analyzes the malfunction data and suggests a self-solution to the user (e.g., "The drain may be clogged with dust"). If the self-solution is ineffective, the server provides contact information for the best point of contact (e.g., the manufacturer's support center or the support desk of the store where the device was purchased).

[0421] Replacement timing prediction and recommendations

[0422] The server monitors the length of use of home electrical appliances based on their start date of use, operation data, and failure history. It analyzes usage status and error logs to detect signs of performance degradation and applies an algorithm (e.g., regression analysis) to predict when to replace the appliance. When the optimal time to replace the appliance approaches, the server generates a notification and sends it to the device. The notification provides a list of recommended products and retailer information (e.g., price comparison data and performance evaluation).

[0423] User support using emotion analysis technology

[0424] If emotion analysis technology (e.g., Affectiva) is built into the device, the device analyzes the user's facial expressions and tone of voice to recognize the user's emotional state. Specifically, the device uses a camera and microphone to record the user's facial expressions and voice and transmits them to the emotion analysis engine. Based on this information, the server adjusts support priorities and provides friendly self-help solutions or the fastest support options.

[0425] Examples of concrete examples and prompts

[0426] Examples:

[0427] One day, the washing machine stops working properly, so the user reports the malfunction using the app. The device analyzes the user's facial expressions and tone of voice, and uses emotion analysis technology to determine that the user's stress level is high. In this case, the server quickly sets the support priority high and offers the user a self-help solution using kind words or the fastest support option. Emotion analysis technology also adjusts maintenance notifications to be sent at the appropriate time, taking the user's emotions into account.

[0428] Example prompt sentence:

[0429] "When your washing machine isn't working, check the error message first."

[0430] "The filter needs to be cleaned. Please refer to the following steps."

[0431] "Your washing machine drain may be clogged. Try a self-fix solution."

[0432] In this way, the present invention efficiently manages warranties and long-term warranty documents for home electrical appliances, enabling appropriate maintenance and troubleshooting. It also helps users determine the optimal timing for replacement and provides support tailored to their emotions, helping to reduce stress and support a more comfortable life.

[0433] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0434] Step 1:

[0435] Upload warranty and extended warranty documents

[0436] Users use their smartphones or tablets to upload warranty and long-term warranty documents for home electrical appliances to the application.

[0437] Input: Image data of warranty or extended warranty documents taken by the user.

[0438] Output: Image file saved on your device.

[0439] Specific operation: The user taps the "Upload warranty card" button in the app, takes a photo of the warranty card using the camera function, and uploads it.

[0440] Step 2:

[0441] OCR text analysis

[0442] The device receives the uploaded image and analyzes the text information in the image using OCR technology (e.g., Google Cloud Vision API).

[0443] Input: An image file saved on your device.

[0444] Output: Parsed text data (e.g. warranty period, serial number, store information).

[0445] What happens: The device uses an image processing library (e.g., OpenCV) to preprocess the image and perform character recognition.

[0446] Step 3:

[0447] Structuring and transferring text data

[0448] The analyzed text data is structured and sent from the terminal to the server.

[0449] Input: Text data generated by OCR analysis.

[0450] Output: The structured data is sent to the server and stored in a database.

[0451] What it does: The device converts text data into a structured format, such as JSON, and sends it to the server, which then connects to a database and inserts the received data into the appropriate table.

[0452] Step 4:

[0453] Maintenance data collection and analysis

[0454] The terminal periodically collects usage data of the home electrical appliances and sends it to the server.

[0455] Input: Usage data obtained from home appliances.

[0456] Output: Analyzed maintenance timing forecast data.

[0457] Specific operation: The device automatically collects operational data from home appliances using Wi-Fi or Bluetooth and sends it to a server. The server analyzes the collected data and predicts when maintenance will be required based on the maintenance cycle of the home appliance.

[0458] Step 5:

[0459] Maintenance notification generation and distribution

[0460] When the server approaches the predicted maintenance period, it generates a notification and delivers it to the terminal.

[0461] Input: Maintenance forecast information based on analytical data.

[0462] Output: Maintenance notification displayed on the terminal.

[0463] Specific operation: The server creates a notification message and sends it to the device as a push notification. The notification contains information about the filter cleaning procedure and the necessary tools.

[0464] Step 6:

[0465] Receiving and analyzing malfunction reports

[0466] When a user reports a malfunction within the app, the device collects detailed information and sends it to the server.

[0467] Input: User-entered fault description and answers to follow-up questions.

[0468] Output: Self-help solutions and contact information as a result of analyzing the failure information.

[0469] Specific operation: The device acquires additional information from the user in the form of interactive questions and sends it to the server. The server then refers to a database, generates a self-solution based on past troubleshooting data, and proposes it to the user.

[0470] Step 7:

[0471] Replacement timing prediction and recommendations

[0472] The server predicts when it is time to replace household electrical appliances based on usage data.

[0473] Input: Date the appliance was first used, operation data, and failure history.

[0474] Output: Notification of replacement time and recommended product list.

[0475] Specific operation: The server uses statistical models (e.g., regression analysis) to predict signs of deterioration and the recommended replacement time for home appliances. When the optimal replacement time approaches, it creates a notification and sends it to the device, including a list of recommended products and retailer information.

[0476] Step 8:

[0477] Optimizing support through sentiment analysis

[0478] Emotion analysis technology recognizes the user's emotional state and adjusts support priorities.

[0479] Input: The user's facial expression and tone of voice.

[0480] Output: Adjusted support priority and customized support message.

[0481] How it works: The device uses a camera and microphone to record the user's facial expressions and voice, and sends them to an emotion analysis engine. The server analyzes the user's emotional data and generates a customized message based on the user's stress level, which is then provided to the user.

[0482] By explaining the specific operations and processing flow in this step-by-step manner, this system can efficiently manage home appliance warranties, perform maintenance, handle malfunctions, and suggest replacements, thereby improving user convenience.

[0483] (Application example 2)

[0484] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0485] Conventional management systems for home appliance warranties and long-term warranty documents are inconvenient for many users and lack the functionality to provide fast and effective support in the event of breakdowns or maintenance. Furthermore, support does not take into account the psychological state of the user, making it difficult to provide appropriate support, especially during busy periods or emergencies. Furthermore, when it comes to security devices, warranty management and fast response in the event of breakdowns are required.

[0486] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0487] In this invention, the server includes: means for users to upload their home appliance warranties or extended warranty documents; means for analyzing the uploaded documents using recognition technology and extracting warranty-related information; means for managing the extracted information for each user and providing it as needed; means for monitoring the usage and environment of the home appliance and suggesting appropriate maintenance times and work methods; means for receiving and analyzing reports from users when a malfunction occurs and suggesting self-resolved solutions and contact information; means for predicting the optimal replacement timing based on the duration of use and performance degradation data of the home appliance and suggesting recommended products and optimal retailers; means for managing the device warranties, recognizing the user's emotional state, and providing appropriate support; means for setting priorities according to the user's emotional state in the event of a security device malfunction and providing a prompt response; and means for providing security services using an emotion engine. This allows users to efficiently manage their warranties and extended warranty documents and receive prompt and appropriate support in the event of a malfunction or maintenance, particularly for security devices, enabling warranty management and prompt response in the event of a malfunction.

[0488] "User" refers to an individual or organization that uses the system.

[0489] "Home appliances" refers to electrical appliances used in ordinary households, such as refrigerators, washing machines, air conditioners, and televisions.

[0490] A "warranty" is a document that guarantees the quality and performance of a product and states that if a product breaks or is damaged within a certain period of time, it will be repaired or replaced free of charge.

[0491] "Extended warranty documents" refer to contracts or certificates that extend the product warranty for a certain period of time.

[0492] "Means of uploading" refers to the function of sending images and data to a cloud server, etc. using a smartphone or tablet.

[0493] "Recognition technology" refers to technology for analyzing images and text data, such as OCR (optical character recognition) and image analysis technology.

[0494] "Means for extracting" refers to the function of extracting necessary information from an image using recognition technology.

[0495] "Means for managing information for each user" refers to the function of linking extracted information to each user and storing and managing it in a database.

[0496] "Means for providing" refers to the function of immediately displaying or providing stored information when a user needs the information.

[0497] "Usage status" refers to data on the actual usage of home appliances, such as operating time and frequency of use.

[0498] "Environment" refers to data on external factors, including the location where the home appliance is placed and the conditions of use.

[0499] "Means to suggest maintenance timing" refers to a function that predicts and notifies appropriate maintenance timing based on usage and environmental data.

[0500] "Means for suggesting work methods" refers to a function that provides users with specific procedures and tools for maintenance.

[0501] "Means for accepting reports" refers to the functionality that allows users to input and transmit faults and problems into the system.

[0502] "Means of analysis" refers to the function of analyzing the reported information and identifying the cause of the failure and how to deal with it.

[0503] "Means for suggesting self-solutions" refers to a function that suggests ways for users to solve malfunctions or problems on their own.

[0504] "Means to suggest contact points" refers to a function that provides contact information for the most appropriate support desk when you are unable to resolve the issue yourself.

[0505] "Usage period" refers to data on the period of time that a home appliance is actually used after it is purchased.

[0506] "Performance degradation data" refers to data that indicates a decline in performance, such as the usage time or malfunction history of a home appliance.

[0507] "Means for predicting replacement timing" refers to a function that predicts the optimal time to replace a product based on the length of use and performance degradation data.

[0508] "Means for suggesting recommended products" refers to the function of listing and providing products that meet the user's needs when replacement is required.

[0509] "Means to suggest the best retailer" refers to the function of providing information on stores that sell recommended products at the most reasonable prices.

[0510] "Emotional state" refers to the psychological state of the user as judged from facial expressions, tone of voice, input content, etc.

[0511] "Emotion engine" refers to technology that recognizes the user's emotional state and adjusts support priorities and response methods.

[0512] "Security devices" refers to devices used to secure a home or business, such as security cameras and door locks.

[0513] An embodiment of the present invention will be described.

[0514] This invention is a system that allows users to efficiently manage warranties or long-term warranty documents for home appliances and perform maintenance or troubleshooting in a timely manner. It also has the feature of recognizing the user's emotional state and providing prompt and appropriate support. The system of the present invention includes the following means.

[0515] 1. Upload and analyze the warranty card

[0516] First, users upload their appliance warranty or extended warranty documents using their smartphone or tablet. The uploaded documents are then analyzed using OCR technology (e.g., Tesseract OCR), and warranty-related information, such as the warranty period, serial number, and purchase store information, is extracted as text. The extracted information is managed for each user and securely stored in a cloud database. This allows users to instantly check warranty information whenever needed.

[0517] 2. Maintenance Notifications and Suggestions

[0518] The server monitors the usage status and environmental data of the home appliances to detect signs of deterioration or failure. This involves analyzing the log data and usage frequency emitted by the appliances. For example, it monitors the operating time of a refrigerator compressor and the number of times a washing machine is used. The server analyzes the collected data and predicts the appropriate time for maintenance. When the predicted maintenance time approaches, it notifies the user using a notification service such as Firebase and suggests specific work methods.

[0519] 3. Support for inquiries in the event of a malfunction

[0520] When a malfunction occurs, the user uses the malfunction reporting function within the app and enters details of the problem. The server receives this information, analyzes it, and proposes a self-solution as quickly as possible. If self-solution is difficult, the server suggests the most appropriate point of contact (for example, contact information for the manufacturer's support center or the store where the device was purchased). Furthermore, an emotion engine (for example, Google Cloud AutoML) that recognizes the user's emotional state is used to determine the user's level of stress or anxiety. Depending on the user's emotional state, the server quickly raises the priority of support and takes appropriate action.

[0521] 4. Replacement timing prediction and recommendations

[0522] The server predicts the optimal time to replace the appliance based on its usage period and performance degradation data. It analyzes statistical data and the usage of the appliance to detect signs of deterioration. Based on this information, when the optimal time to replace the appliance approaches, it provides the user with a list of recommended products and information on the best retailers. The list includes product performance evaluations and price comparison data, allowing the user to make the most economical choice.

[0523] 5. Security Devices and Emotion Engines

[0524] The present invention is also effective in managing warranty certificates for security devices. When a user reports a malfunction of a security device, the emotion engine analyzes the user's facial expression and tone of voice to determine their emotional state. If the engine determines that the user's stress level is high, it increases the priority of support and provides a prompt response.

[0525] As a concrete example, consider a situation where a washing machine breaks down. The user opens the app and fills in the breakdown report form with details. The server analyzes the user's facial expressions and tone of voice, and uses an emotion engine to determine that the user's stress level is high. In this case, the server sets the support priority high and offers a kind self-help solution or the fastest support option.

[0526] An example of a prompt sentence is, "Analyze the user's current emotional state based on facial expressions and voice data, and if the stress level is high, set a high priority for support, and provide the user with troubleshooting procedures and the fastest support options in user-friendly language."

[0527] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0528] Step 1:

[0529] Users scan the warranty or extended warranty document of their home appliance with a smartphone or tablet and upload it to the system. The input data is an image file of the warranty or extended warranty document. The output is an image file that can be analyzed.

[0530] Step 2:

[0531] The server receives the uploaded image and analyzes the text information in the image using OCR (Optical Character Recognition) technology. The input data is the image file. The output is text data such as the extracted warranty period, serial number, and purchase store information. Specifically, the server uses Tesseract OCR to extract the text.

[0532] Step 3:

[0533] The server structures the extracted text data and stores it in a database. The input data is the text information extracted by OCR. The output is the structured information registered in the database. Specifically, the text information is inserted into a MySQL database.

[0534] Step 4:

[0535] The server periodically monitors the usage status and environmental data of the home appliances. The input data is usage data and sensor data obtained from the home appliances. The output is information on predicted maintenance times. Specifically, the server communicates with the home appliances, collects log data, and analyzes it.

[0536] Step 5:

[0537] Based on the analyzed data, the server predicts the appropriate maintenance time and notifies the user. The input data is the analysis results of usage status and environmental data. The output is a maintenance notification sent to the user. Specifically, the notification is sent to the user's smartphone using a notification service such as Firebase.

[0538] Step 6:

[0539] When a malfunction occurs, the user enters details into a malfunction report form within the app. The input data is text information about the malfunction. The output is the malfunction data that is sent to the server. Specifically, the user opens the app and submits the malfunction report.

[0540] Step 7:

[0541] The server receives the failure data and uses an emotion engine to analyze the user's emotional state. The input data is the failure report and the user's emotional data (facial expressions and tone of voice). The output is the result of the user's emotional state assessment. Specifically, emotion analysis is performed using Google Cloud AutoML.

[0542] Step 8:

[0543] The server sets support priorities according to the user's emotional state and responds promptly and appropriately. The input data is the user's emotional state assessment result and fault data. The output is the support priority setting and specific countermeasures. Specific operations include suggesting a self-solution to resolve the fault, and if that does not resolve the issue, directing the user to the most appropriate support desk.

[0544] Step 9:

[0545] The server predicts the optimal time to replace appliances based on their usage period and performance degradation data. The input data is usage period data and performance degradation data. The output is the replacement period and a list of recommended products. Specifically, it references statistical data and applies a prediction algorithm.

[0546] Step 10:

[0547] The server provides users with a list of recommended products and information on the best retailers. The input data is the predicted replacement timing and market data. The output is a list of recommended products and retailer information sent to the user. Specifically, it presents product performance evaluations and price comparison data.

[0548] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0549] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0550] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0551] [Second embodiment]

[0552] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0553] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0554] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0555] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0556] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0557] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0558] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0559] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0560] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0561] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0562] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0563] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0564] As an embodiment of the present invention, a system for managing warranties and long-term warranty documents for home appliances and improving user convenience will be described. This system allows users to upload their warranties and long-term warranty documents for home appliances, and through a series of processes that automatically analyze and manage this information, it supports maintenance and responses in the event of a breakdown and suggests the optimal time to replace the appliance.

[0565] Management of warranties and long-term compensation documents

[0566] First, users upload their appliance warranty or extended warranty documents to the app using their smartphone or tablet. The device receives the uploaded image and analyzes it using image recognition technology (such as OCR). Specifically, it uses an image processing library to extract text information (such as warranty period, serial number, and store information) and generates it as structured data. The extracted data is registered in a database managed by the server. This allows users to check the warranty information at any time through the app.

[0567] Maintenance Notifications and Suggestions

[0568] The device periodically (e.g., daily or weekly) collects usage data from the appliance. In the case of smart appliances, the device communicates with the appliance and automatically acquires the data. The server analyzes this data and predicts when maintenance will be required based on the maintenance cycle of each appliance. When the predicted maintenance period approaches, the server generates a notification and sends it to the device. The notification provides the user with specific instructions and tools for the appropriate maintenance work (e.g., cleaning the filter).

[0569] Support for inquiries in case of breakdowns

[0570] When a home appliance breaks down, the user opens the support function within the app and enters the details of the malfunction into the "Report a Malfunction" form. The device then presents questions to the user and collects details of the malfunction, such as "Is an error message displayed?" The server then analyzes the collected malfunction data and suggests possible self-solutions to the user (e.g., "The drain may be clogged with dust"). If the self-solutions are ineffective, the server also provides contact information for the most appropriate point of contact (the manufacturer's support center or the support desk of the store where the appliance was purchased). This allows the user to quickly and efficiently deal with the malfunction.

[0571] Replacement timing prediction and recommendations

[0572] The server monitors the length of time an appliance has been in use based on its start date, operational data, and failure history. It analyzes usage and error logs to detect signs of performance degradation. Based on the analyzed data, the server applies an algorithm to predict when it is time to replace the appliance. For example, it references statistical data such as "Model X washing machines typically begin to deteriorate after about seven years." When the optimal time to replace the appliance approaches, the server generates a notification and sends it to the device. The notification provides the user with a list of recommended products and retailer information. The product list includes price comparison data and performance evaluations. This allows users to replace their appliances at the appropriate time and make economical choices.

[0573] As described above, the present invention provides a system that efficiently manages warranties and long-term warranty documents for home appliances, supports maintenance and troubleshooting, and suggests the optimal time to replace them. This system allows users to use home appliances with peace of mind, greatly improving the convenience of their lives.

[0574] The processing flow will be explained below.

[0575] Management of warranties and long-term compensation documents

[0576] Step 1:

[0577] Users take a photo of the home appliance warranty or long-term warranty document with their smartphone or tablet and upload it to the app.

[0578] Step 2:

[0579] The device receives the uploaded image and begins analyzing it using OCR (Optical Character Recognition) technology, recognizing text information using an image processing library.

[0580] Step 3:

[0581] The device analyzes the text information extracted from the image (warranty period, serial number, purchase store information, etc.) and converts it into structured data.

[0582] Step 4:

[0583] The server receives the structured data sent from the terminal and registers it in a database for each user.

[0584] Step 5:

[0585] Whenever necessary, the user opens the app and checks the registered warranty information.

[0586] Maintenance Notifications and Suggestions

[0587] Step 1:

[0588] The device periodically (e.g., daily or weekly) collects usage data from the home appliance. In the case of smart home appliances, the device automatically acquires data through communication with the appliance.

[0589] Step 2:

[0590] The device sends collected usage data to a server, including operating time, number of uses, error logs, etc.

[0591] Step 3:

[0592] The server analyzes the data it receives and predicts the next maintenance date based on the maintenance cycle for each appliance.

[0593] Step 4:

[0594] When the server approaches the predicted maintenance time, it generates a maintenance notice and delivers it to the terminal.

[0595] Step 5:

[0596] The device displays maintenance notifications to the user, providing specific work instructions (e.g., cleaning the filter) and the necessary tools.

[0597] Support for inquiries in case of breakdowns

[0598] Step 1:

[0599] When the user recognizes that their home appliance is malfunctioning, they open the support function within the app and enter details of the malfunction into the "Report a Malfunction" form.

[0600] Step 2:

[0601] The device presents the user with questions related to the malfunction and collects specific malfunction information, such as "Is an error message displayed?"

[0602] Step 3:

[0603] The terminal transmits the collected failure data to the server.

[0604] Step 4:

[0605] The server analyzes the failure data and suggests a self-solution to the user (e.g., "It may be due to dust clogging the drain").

[0606] Step 5:

[0607] If the server's self-help efforts are not effective, it will provide contact information for the appropriate point of contact (such as the manufacturer's support center or the support desk at the store where the product was purchased).

[0608] Replacement timing prediction and recommendations

[0609] Step 1:

[0610] The server monitors the period of use of the appliance based on the date of use, operation data, and failure history.

[0611] Step 2:

[0612] The server analyzes the monitoring data to detect signs of performance degradation and also refers to statistical data to predict performance degradation.

[0613] Step 3:

[0614] When the time for replacement approaches, the server generates a notification for the user and sends it to the terminal.

[0615] Step 4:

[0616] The terminal displays a replacement notice to the user and provides a list of recommended products and information on the cheapest retailer.

[0617] Step 5:

[0618] Based on the information presented, the user selects the most suitable home appliance and considers purchasing it.

[0619] The above is a detailed flow of the program processing of this system.

[0620] Example 1

[0621] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0622] Modern homes use many electrical appliances, and it is necessary to properly store their warranties and long-term warranty documents, and efficiently manage their usage, maintenance, and troubleshooting. However, doing this on your own is cumbersome, and missing appropriate maintenance intervals increases the risk of breakdowns, making it difficult to determine the optimal time to replace appliances. For this reason, a system is needed that increases user convenience and streamlines the lifecycle management of electrical appliances.

[0623] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0624] In this invention, the server includes: means for users to upload warranty or extended warranty documents for electrical appliances; means for analyzing the uploaded documents using optical character recognition technology and extracting warranty-related information; means for managing the extracted information for each user and providing it when needed; means for monitoring the usage status and living environment of electrical appliances and suggesting appropriate maintenance times and cleaning methods; means for receiving and analyzing malfunction reports from users in the event of a malfunction and suggesting self-resolved solutions and contact information using a machine learning model; and means for predicting the optimal replacement time based on the duration of use and performance degradation data of the electrical appliances and suggesting recommended products and optimal retailers. This makes it easier for users to manage their warranty and warranty documents, enabling them to perform maintenance and replacement at the appropriate time.

[0625] "Upload" refers to the operation of a user sending a digital image of an electrical product warranty or extended warranty document to the system through the application.

[0626] "Optical character recognition" is a technology for extracting text information from images, specifically OCR (Optical Character Recognition).

[0627] "Warranty information" refers to important data such as the warranty period, serial number, and purchase store information listed on the warranty card or compensation document for an electrical product.

[0628] "Structured data" refers to a data format that is organized so that extracted information can be stored in a table or database.

[0629] "Monitoring" refers to the act of continuously observing the usage of electrical appliances and living environments and collecting necessary data.

[0630] "Maintenance period" refers to the time when an electrical appliance should be inspected, cleaned, or repaired to keep it functioning properly.

[0631] "Fault reporting" refers to the act of a user notifying the system of detailed information about abnormal operation or malfunction of an electrical appliance.

[0632] A "machine learning model" refers to an algorithm that analyzes collected data, finds patterns, and predicts future trends.

[0633] "Performance degradation data" refers to information regarding malfunctions or performance degradation that occur during the use of an electrical product.

[0634] The "optimal replacement time" refers to the point at which the performance of an electrical appliance deteriorates and it becomes economically advantageous to replace it with a new product rather than continuing to repair and maintain it.

[0635] "Recommended products" refer to products that the system recommends as having the same or better performance as the user's current electrical products.

[0636] "Best seller" refers to the store or online shop where you can purchase the recommended products at the best conditions.

[0637] As an embodiment of the present invention, a system for managing warranty certificates and long-term warranty documents for electrical appliances and improving user convenience will be described. This system allows users to upload warranty certificates and long-term warranty documents for electrical appliances, and through a series of processes that automatically analyze and manage this information, it supports maintenance and responses in the event of a breakdown and suggests the optimal time to replace the appliance.

[0638] Management of warranties and long-term compensation documents

[0639] First, users upload the warranty card or extended warranty document for their electrical appliances to the app using a smartphone or tablet. The device receives the uploaded image and analyzes it using image recognition technology (such as OCR). Specifically, an image processing library is used to extract text information (such as warranty period, serial number, and store information) and generate it as structured data. The extracted data is then registered in a database managed by the server. This allows users to check the warranty information at any time through the app.

[0640] Maintenance Notifications and Suggestions

[0641] The device periodically (e.g., daily or weekly) collects appliance usage data. In the case of smart home appliances, the device communicates with the appliance to automatically obtain the data. The server analyzes this data and predicts when maintenance is required based on the maintenance cycle of each appliance. When the predicted maintenance period approaches, the server generates a notification and sends it to the device. The notification provides the user with specific instructions and tools for the appropriate maintenance work (e.g., cleaning the filter).

[0642] Support for inquiries in case of breakdowns

[0643] When an electrical appliance breaks down, the user opens the support function within the app and enters the details of the malfunction into the "Report a malfunction" form. The device then presents questions to the user and collects details of the malfunction, such as "Is an error message displayed?" The server then analyzes the collected malfunction data and suggests possible self-solutions to the user (e.g., "The drain may be clogged with dust"). If the self-solutions are ineffective, the server also provides contact information for the most appropriate point of contact (the manufacturer's support center or the support desk of the store where the appliance was purchased). This allows the user to quickly and efficiently resolve the malfunction.

[0644] Replacement timing prediction and recommendations

[0645] The server monitors the length of time an appliance has been in use based on its start date, operational data, and fault history. It analyzes usage and error logs to detect signs of performance degradation. Based on the analyzed data, the server applies an algorithm to predict when it's time to replace it. For example, it references statistical data such as "Model X refrigerators typically begin to deteriorate after about seven years." When the optimal time to replace an appliance approaches, the server generates a notification and sends it to the device. The notification provides the user with a list of recommended products and retailer information. The product list includes price comparison data and performance ratings. This allows users to replace appliances at the appropriate time and make economical choices.

[0646] Specific examples

[0647] A user uploads a refrigerator warranty, and the device extracts that the warranty period is "May 1, 2023 to May 1, 2025." The server predicts when the washing machine filter needs cleaning and notifies the user, "Please clean the filter." When a user reports error code "E05," the server suggests a self-solution, saying, "Check the drain hose." When seven years have passed since the refrigerator was first used, the server notifies the user, "It's time to replace it," and displays a list of recommended new refrigerators.

[0648] Prompt Sentence Examples

[0649] "Just take a picture of your appliance's warranty card with your camera and upload it to the app. The warranty period and serial number will be automatically extracted."

[0650] "It collects data every week and notifies you when it's time to maintain your appliances. It lets you know when it's time to clean your filters."

[0651] "If your appliance breaks down, please report the problem through the app and we will suggest a solution for you."

[0652] "It analyzes the usage period of home appliances and notifies you of the optimal time to replace them. It also provides a list of recommended products."

[0653] As described above, the present invention provides a system that efficiently manages warranties and long-term warranty documents for electrical appliances, supports maintenance and troubleshooting, and suggests the optimal time to replace them. This allows users to use electrical appliances with peace of mind, greatly improving the convenience of their lives.

[0654] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0655] Step 1:

[0656] User uploads warranty card

[0657] A user uploads an image of the warranty or extended warranty document for an electrical appliance using an app on their smartphone or tablet. Specifically, the user taps the camera button in the app to launch the camera and take a photo of the warranty or select an existing image.

[0658] Input: Warranty card image taken or selected by the user

[0659] Output: Uploaded warranty image

[0660] Step 2:

[0661] The device receives the image

[0662] The device receives the image uploaded by the user and prepares it for image recognition processing.

[0663] Input: Uploaded warranty image

[0664] Output: warranty card image ready for analysis

[0665] Step 3:

[0666] The device analyzes the image

[0667] The device performs preprocessing using an image processing library (e.g., OpenCV). This includes adjusting the resolution, removing noise, and identifying text regions. It then uses an OCR engine (e.g., Tesseract) to extract text from the image. This allows information such as the warranty period, serial number, and store information to be obtained.

[0668] Input: warranty card image ready for analysis

[0669] Data processing: image resolution adjustment, noise removal, text area identification

[0670] Data Calculation: Text Extraction Using an OCR Engine

[0671] Output: Extracted text data

[0672] Step 4:

[0673] The device structures the data

[0674] The text data extracted by the device is converted into structured data such as JSON format. Specifically, the warranty period, serial number, and purchase store information are stored in the appropriate fields.

[0675] Input: Extracted text data

[0676] Data processing: structuring text data into JSON format

[0677] Output: Structured data in JSON format

[0678] Step 5:

[0679] The device sends data to the server

[0680] The device sends the structured data to the server using HTTPS as the communication protocol.

[0681] Input: Structured data in JSON format

[0682] Output: Structured data sent to the server

[0683] Step 6:

[0684] The server stores the data

[0685] The server stores the received structured data in a database management system (e.g. MySQL). The server first checks the data for errors, and if there are no errors, stores it in the database in the appropriate format.

[0686] Input: Structured data sent to the server

[0687] Data processing: Checking data for errors and converting it to the appropriate format

[0688] Output: Data stored in the database

[0689] Step 7:

[0690] Device collects usage data

[0691] The device periodically collects usage data from the appliance. In the case of smart appliances, the device communicates with the appliance to automatically obtain data, including operating time and error logs.

[0692] Input: Usage data collected from electronic devices

[0693] Output: Usage data stored on the device

[0694] Step 8:

[0695] Server maintenance forecast

[0696] The server analyzes the usage data sent from the device, calculates the maintenance cycle using an analysis tool (e.g., Pandas), and calculates the predicted maintenance time.

[0697] Input: Usage data stored on the device

[0698] Data calculation: Usage data analysis, maintenance cycle calculation

[0699] Output: Estimated maintenance time

[0700] Step 9:

[0701] Server generates maintenance notice

[0702] The server generates a notification based on the predicted maintenance time, specifically a message containing the maintenance procedure and required tools.

[0703] Input: Estimated maintenance time

[0704] Data processing: Notification message generation

[0705] Output: The generated notification message

[0706] Step 10:

[0707] The server sends a notification

[0708] The server generates a notification message and sends it to the device, which receives it and displays it to the user as a push notification.

[0709] Input: The generated notification message

[0710] Output: Notification message sent to the terminal

[0711] Step 11:

[0712] User reports a malfunction

[0713] The user accesses the support function within the app and enters detailed information about the problem into the "Report a Problem" form.

[0714] Input: The details of the malfunction entered in the "Malfunction Report" form

[0715] Output: The entered fault report data

[0716] Step 12:

[0717] The device collects failure data

[0718] The device asks the user additional questions to gather more detailed information about the failure, such as "Is an error message displayed?"

[0719] Input: Fault report data and responses to additional questions

[0720] Output: Collected detailed failure data

[0721] Step 13:

[0722] The server proposes a self-solution

[0723] The device sends the collected failure data to a server, which then analyzes the data using a machine learning model and proposes a self-solution to the user.

[0724] Input: Collected detailed failure data

[0725] Data calculation: failure data analysis, self-resolved solution generation

[0726] Output: Self-solution proposal

[0727] Step 14:

[0728] The server provides contact information

[0729] If self-help measures are not effective, the server provides contact information for the best place to contact (the manufacturer's support center or the support desk at the store where you purchased the product).

[0730] Input: Failure data and self-solving results

[0731] Output: Best contact information

[0732] Step 15:

[0733] The server monitors the usage period

[0734] The server monitors the electrical appliances based on their start date of use, operation data, and failure history.

[0735] Input: Start date of use, operation data, failure history

[0736] Output: Monitoring result data

[0737] Step 16:

[0738] Server replacement time prediction

[0739] The server analyzes the monitoring results data and uses data analysis tools and algorithms to predict when it is time to replace the device.

[0740] Input: Monitoring result data

[0741] Data calculation: Performance degradation analysis, replacement timing prediction

[0742] Output: Predicted replacement time

[0743] Step 17:

[0744] The server generates a recommendation notification

[0745] When the time for replacement approaches, the server generates a notification message including a list of recommended products and information on the best retailer.

[0746] Input: Predicted replacement time

[0747] Data processing: Generation of recommended product list and retailer information

[0748] Output: Recommendation notification message

[0749] Step 18:

[0750] The server sends a notification

[0751] The server generates a recommendation notification and sends it to the device, which receives it and displays it to the user as a push notification.

[0752] Input: Generated recommendation notification message

[0753] Output: Recommendation notification message sent to the device

[0754] (Application example 1)

[0755] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0756] Users often have difficulty managing or lose their home appliance warranties and long-term warranty documents, making it difficult to quickly check the warranty details when needed. Furthermore, notifications regarding home appliance maintenance schedules and troubleshooting are not provided in a timely manner, reducing user convenience. Furthermore, it is difficult for users to properly determine when to replace their home appliances, which can lead to financial losses if they miss the optimal timing. These issues need to be resolved.

[0757] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0758] In this invention, the server includes: means for users to upload warranty or extended warranty documents for electrical appliances; means for analyzing the uploaded documents using image recognition technology and extracting warranty-related information; means for managing the extracted information for each user and providing it as needed; means for monitoring the usage status and installation environment of the electrical appliance and suggesting appropriate maintenance times and cleaning methods; means for receiving and analyzing malfunction reports from users and suggesting self-resolved solutions and contact information; means for predicting the optimal replacement time based on the electrical appliance's usage period and performance degradation data and suggesting recommended products and the lowest-priced retailers; means for extracting warranty information from warranty images using a smartphone and storing it in a database; means for providing maintenance notifications and support in the event of a malfunction; and means for predicting the replacement time based on the start date of use and usage data and providing a list of recommended products. This allows users to efficiently manage warranty and warranty documents, receive timely notifications for maintenance and malfunction response, and use electrical appliances economically and efficiently by knowing the optimal replacement time.

[0759] "Electrical appliances" is a general term for devices and equipment that require electrical power for use.

[0760] A "warranty" is a document stating that the manufacturer or seller guarantees the quality and performance of a product for a certain period of time.

[0761] An "extended warranty document" is a contract that guarantees the repair or replacement of a product for a period beyond the normal warranty period.

[0762] "User" means any person or entity that uses a system or device.

[0763] "Image recognition technology" is a general term for technology that analyzes and extracts specific information from images, and generally includes OCR (optical character recognition).

[0764] "Uploading" is the act of sending data from a local device to a network or server.

[0765] "Warranty information" is data about warranty details such as the warranty period, serial number, and purchase store information, which are written on the warranty card.

[0766] "Extraction" refers to the process of extracting necessary information from images or text.

[0767] "Management" is the act of controlling the organization, storage, access, and updating of information.

[0768] "Providing" refers to supplying information or services required by users.

[0769] "Monitoring" is the process of continuously observing and recording the state of a system or device.

[0770] "Maintenance period" refers to the period when a device or equipment requires regular maintenance, inspection, or repair.

[0771] "Cleaning method" refers to the selection of procedures and materials for cleaning equipment or devices.

[0772] "Fault reporting" is the act of a user notifying a system operator of a fault or problem in a device or system.

[0773] "Analysis" is the process of examining and evaluating data to understand its structure and meaning.

[0774] "Self-help" is a method by which a user can solve a problem on their own without professional support.

[0775] "Contact Information" refers to the location or person that a user should contact when seeking problem resolution or support.

[0776] "Period of use" refers to the period during which a device or system is actually used.

[0777] "Performance degradation data" is data that records how the performance of a device or system deteriorates over time.

[0778] "Replacement time" is the best time to replace an existing device or system with a new one.

[0779] A "recommended product" is a product that the system determines to be most appropriate in a particular situation or under a particular condition, based on specific user needs.

[0780] "Best Dealer" refers to the dealer that offers the lowest price for a particular product.

[0781] A "smartphone" is a type of mobile phone, a handheld device that combines advanced computing power and communication capabilities.

[0782] A database is a system that organizes and stores information and allows it to be retrieved in any format required.

[0783] "Date of Use" means the date on which a device or system is first used.

[0784] As an embodiment of this invention, we will explain the "Home Appliance Security Support" system. This system allows users to manage warranties and long-term warranty documents for electrical appliances, and efficiently recommends maintenance, troubleshooting, and optimal replacement timing. The main hardware and software that make up this system are described below.

[0785] Hardware

[0786] Smartphone

[0787] server

[0788] software

[0789] Python

[0790] OpenCV

[0791] Tesseract OCR

[0792] SQLite

[0793] Requests

[0794] Processing flow

[0795] 1. Upload and analyze the warranty card

[0796] Users use their smartphones to take photos of electrical appliance warranties and long-term warranty documents and upload them to the app. The device receives the uploaded images and analyzes them using image recognition technology (OCR).

[0797] Specifically, OpenCV and Tesseract OCR are used to extract text information (warranty period, serial number, purchase store information, etc.) from uploaded images and generate it as structured data.

[0798] The extracted data is registered in a database (SQLite) managed by the server, allowing users to check warranty information at any time via their smartphone.

[0799] 2. Maintenance Notifications and Suggestions

[0800] The server continuously monitors the usage status and installation environment of electrical appliances to predict maintenance times and cleaning methods.

[0801] When maintenance is due, the server generates a notification and sends it to the smartphone, including specific instructions for the appropriate maintenance task (e.g., cleaning the filter) and the tools required.

[0802] 3. Support in case of breakdown

[0803] When an electrical appliance breaks down, users can use the support function in the smartphone app to "report the malfunction."

[0804] The device asks the user questions to gather details about the failure, such as "Are you seeing any error messages?"

[0805] The server analyzes the collected failure data and suggests possible solutions to the user (for example, "The drain may be clogged with dust.") If the solutions are ineffective, the server provides the user with the best contact point (the manufacturer's support center).

[0806] 4. Replacement timing prediction and recommendations

[0807] The server monitors the period of use of the appliance based on the date of use, operation data, and failure history.

[0808] The system analyzes usage and error logs to detect signs of performance degradation and applies an algorithm to predict when it's time to replace the device. Users are notified of the optimal time to replace the device via email or in-app notifications.

[0809] The notification will also include a list of recommended products, including performance evaluations and price comparison data, as well as retailer information.

[0810] Examples and prompts

[0811] Specific examples

[0812] Users simply take a photo of their washing machine's warranty card with their smartphone and upload it. The app analyzes the warranty information and registers it in a database. When seven years have passed since the purchase date, the app displays a replacement notification and provides a list of recommended products.

[0813] Prompt Sentence Examples

[0814] "Please implement a program to extract warranty information from uploaded images and register the warranty period and serial number in a database. After a certain period of time, send a maintenance notification or suggest a replacement."

[0815] In this way, the "Home Appliance Security Support" system aims to enable users to efficiently manage warranties and compensation documents, receive timely notifications about maintenance and malfunctions, and know the optimal time to replace their electrical appliances, thereby enabling them to use their appliances economically and efficiently.

[0816] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0817] Step 1:

[0818] Warranty upload and analysis

[0819] A user uses a smartphone to take a photo of an electronic product's warranty or extended warranty document and upload it to the app. The image taken by the user is provided to the app as input. The device receives this and uses OpenCV and Tesseract OCR to extract text information from the image (warranty period, serial number, purchase store information, etc.). The output is structured data extracted from the warranty, which is sent to the server.

[0820] Step 2:

[0821] Saving to a database

[0822] The server stores the structured data received from step 1 (warranty period, serial number, purchase store information, etc.) in an SQLite database. As input, the structured data is given to the server, and an insert operation is performed into the database. The output is the warranty information registered in the database.

[0823] Step 3:

[0824] Maintenance forecast and notification

[0825] The server predicts when maintenance is required based on the appliance's usage and installation environment. Warranty information and usage data stored in the database are used as input. Based on this, the server predicts when the next maintenance task (e.g., filter cleaning) will be performed. When the maintenance deadline approaches, the server generates a notification and distributes it to the device. The output is a maintenance notification that is displayed on the device.

[0826] Step 4:

[0827] Fault reporting and support suggestions

[0828] When a user reports a malfunction of an electrical appliance, they use the support function within the smartphone app to make a "fault report." The user's report is provided to the app as input. The device then presents questions to the user and collects detailed information about the malfunction. Based on the information entered, the server performs an analysis and suggests a self-solution (e.g., a possible dust clog in the drain). If the self-solution is insufficient, it also provides information on the most appropriate contact point (the manufacturer's support center). The output is the self-solution presented to the user and the contact information.

[0829] Step 5:

[0830] Replacement timing prediction and recommendations

[0831] The server predicts when to replace a home appliance based on the date it was first used, its operation data, and its failure history. The inputs include warranty information, usage status, and error logs stored in a database. The server analyzes these to detect signs of performance degradation and determine the optimal time to replace the appliance. When the time to replace the appliance approaches, the server generates a notification and sends it to the device along with a list of recommended products. The output is a replacement notification and a list of recommended products to the user.

[0832] This series of processes enables users to efficiently manage warranties and compensation documents, receive timely notifications for maintenance and malfunctions, and know the optimal time to replace their electrical appliances, enabling them to use them economically and efficiently.

[0833] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0834] This invention is a system that manages home appliance warranties and long-term warranty documents to improve user convenience, and also combines an emotion engine to provide support tailored to the user's emotional state. This system allows users to upload their home appliance warranties and long-term warranty documents, and through a series of processes that automatically analyze and manage this information, it supports maintenance and responses in the event of a breakdown, and suggests the optimal time to replace the appliance. Furthermore, by using an emotion engine that recognizes the user's emotions, it provides more appropriate support.

[0835] Management of warranties and long-term compensation documents

[0836] First, users upload their appliance warranty or extended warranty documents to the app using their smartphone or tablet. The device receives the uploaded image and analyzes it using OCR (Optical Character Recognition) technology. Specifically, it uses an image processing library to extract text information (such as warranty period, serial number, and purchase store information) and generates it as structured data. The extracted data is registered in a database managed by the server. This allows users to easily check warranty information at any time through the app.

[0837] Maintenance Notifications and Suggestions

[0838] The device periodically (e.g., daily or weekly) collects usage data from the appliance. In the case of smart appliances, the device communicates with the appliance and automatically acquires the data. The server analyzes this data and predicts when maintenance will be required based on the maintenance cycle of each appliance. When the predicted maintenance period approaches, the server generates a notification and sends it to the device. The notification provides the user with specific instructions and tools for the appropriate maintenance work (e.g., cleaning the filter).

[0839] Support for inquiries in case of breakdowns

[0840] When a home appliance breaks down, the user opens the support function within the app and enters the details of the malfunction into the "Report a Malfunction" form. The device then presents questions to the user and collects details of the malfunction, such as "Is an error message displayed?" The device then sends the collected malfunction data to the server. The server analyzes the malfunction data and suggests possible self-solutions to the user (e.g., "The drain may be clogged with dust"). If the self-solutions are ineffective, the server also provides contact information for the best point of contact (the manufacturer's support center or the support desk of the store where the appliance was purchased). This allows the user to quickly and efficiently handle the malfunction.

[0841] Replacement timing prediction and recommendations

[0842] The server monitors the length of time an appliance has been in use based on its start date, operational data, and failure history. It analyzes usage and error logs to detect signs of performance degradation. Based on the analyzed data, the server applies an algorithm to predict when it is time to replace the appliance. For example, it references statistical data such as "Model X washing machines typically begin to deteriorate after about seven years." When the optimal time to replace the appliance approaches, the server generates a notification and sends it to the device. The notification provides the user with a list of recommended products and retailer information. The product list includes price comparison data and performance evaluations. This allows users to replace their appliances at the appropriate time and make economical choices.

[0843] User support using an emotion engine

[0844] If an emotion engine is built into the system, it can recognize the user's emotional state and provide more effective support. For example, when a user uses the in-app support feature after a home appliance breaks down, the emotion engine can determine the user's level of stress or anxiety from their facial expression, voice, or input.

[0845] Examples:

[0846] One day, the washing machine stops working properly, so the user uses the app to report the malfunction. The device analyzes the user's facial expressions and tone of voice and uses an emotion engine to determine that the user's stress level is high. In this case, the server quickly sets the support priority high and offers the user a self-solution using gentle language or the fastest support option. The emotion engine also takes the user's emotions into consideration when sending maintenance notifications, adjusting them to be sent at the appropriate time. This reduces the user's psychological burden and enables thorough support.

[0847] As described above, this invention is a system that efficiently manages warranties and long-term warranty documents for home appliances, supports maintenance and troubleshooting, and suggests the optimal time to replace the appliance. Furthermore, by utilizing an emotion engine, it provides support according to the user's emotional state. This system allows users to use home appliances with peace of mind, improving their quality of life.

[0848] The processing flow will be explained below.

[0849] Management of warranties and long-term compensation documents

[0850] Step 1:

[0851] Users take a photo of the home appliance warranty or long-term warranty document with their smartphone or tablet and upload it to the app.

[0852] Step 2:

[0853] The device receives the uploaded image and begins analyzing it using OCR (Optical Character Recognition) technology, recognizing text information using an image processing library.

[0854] Step 3:

[0855] The device analyzes the text information extracted from the image (warranty period, serial number, purchase store information, etc.) and converts it into structured data.

[0856] Step 4:

[0857] The server receives the structured data sent from the terminal and registers it in a database for each user.

[0858] Step 5:

[0859] Whenever necessary, the user opens the app and checks the registered warranty information.

[0860] Maintenance Notifications and Suggestions

[0861] Step 1:

[0862] The device periodically (e.g., daily or weekly) collects usage data from the home appliance. In the case of smart home appliances, the device automatically acquires data through communication with the appliance.

[0863] Step 2:

[0864] The device sends collected usage data to a server, including operating time, number of uses, error logs, etc.

[0865] Step 3:

[0866] The server analyzes the data it receives and predicts the next maintenance date based on the maintenance cycle for each appliance.

[0867] Step 4:

[0868] When the server approaches the predicted maintenance time, it generates a maintenance notice and delivers it to the terminal.

[0869] Step 5:

[0870] The device displays maintenance notifications to the user, providing specific work instructions (e.g., cleaning the filter) and the necessary tools.

[0871] Support for inquiries in case of breakdowns

[0872] Step 1:

[0873] When the user recognizes that their home appliance is malfunctioning, they open the support function within the app and enter details of the malfunction into the "Report a Malfunction" form.

[0874] Step 2:

[0875] The device presents the user with questions related to the malfunction and collects specific malfunction information, such as "Is an error message displayed?"

[0876] Step 3:

[0877] The terminal transmits the collected failure data to the server.

[0878] Step 4:

[0879] The server analyzes the failure data and suggests a self-solution to the user (e.g., "It may be due to dust clogging the drain").

[0880] Step 5:

[0881] If the server's self-help efforts are not effective, it will provide contact information for the appropriate point of contact (such as the manufacturer's support center or the support desk at the store where the product was purchased).

[0882] Replacement timing prediction and recommendations

[0883] Step 1:

[0884] The server monitors the period of use of the appliance based on the date of use, operation data, and failure history.

[0885] Step 2:

[0886] The server analyzes the monitoring data to detect signs of performance degradation and also refers to statistical data to predict performance degradation.

[0887] Step 3:

[0888] When the time for replacement approaches, the server generates a notification for the user and sends it to the terminal.

[0889] Step 4:

[0890] The terminal displays a replacement notice to the user and provides a list of recommended products and information on the cheapest retailer.

[0891] Step 5:

[0892] Based on the information presented, the user selects the most suitable home appliance and considers purchasing it.

[0893] User support using an emotion engine

[0894] Step 1:

[0895] When a user reports a malfunction of a home appliance within the app, the device captures the user's facial expressions, voice, or input and sends it to the emotion engine.

[0896] Step 2:

[0897] The emotion engine analyzes the user's emotional data to determine their stress and anxiety levels.

[0898] Step 3:

[0899] The server receives emotional data from the emotion engine and tailors its response based on the user's emotional state: if stress levels are high, it prioritizes providing immediate support.

[0900] Step 4:

[0901] The server suggests ways to respond based on the user's emotional state. For example, if the user is under high stress, it will respond with gentle words and provide a quick solution.

[0902] Step 5:

[0903] The terminal displays the response methods and specific solutions received from the server to the user, and the user solves the problem according to the most appropriate response method.

[0904] Examples:

[0905] Example 1:

[0906] Step 1: The user takes a photo of the washing machine warranty card with their smartphone and uploads it to the app.

[0907] Step 2: The device analyzes the image and extracts the warranty period and serial number.

[0908] Step 3: The server stores the extracted data in the database, and the user can check the warranty information when needed.

[0909] Example 2:

[0910] Step 1: When the washing machine breaks down, the user reports the problem through the app. The device sends the user's tone of voice to the emotion engine.

[0911] Step 2: The emotion engine determines that the user's stress level is high.

[0912] Step 3: The server provides prompt support and uses kind words to ease stress.

[0913] Step 4: The device displays the self-help solution and contact information for the support center to the user.

[0914] Example 2

[0915] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0916] In recent years, many household electrical appliances have been used, but managing the warranties and long-term warranty documents for these appliances has become cumbersome, making it difficult to know when to respond to breakdowns and when to perform maintenance. As a result, users are often unable to take appropriate action within the warranty period, resulting in unnecessary expenses and stress. Furthermore, not knowing when to replace an appliance can lead to problems such as a decline in efficiency due to the extended use of degraded appliances. Furthermore, there is a need to reduce the stress users feel when an appliance breaks down and provide effective support, but no system exists to achieve this.

[0917] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0918] In this invention, the server includes: a means for users to upload warranty or extended warranty documents for household electrical appliances; a means for analyzing the uploaded documents using optical character recognition technology and extracting warranty-related information; a means for managing the extracted information for each user and providing it as needed; a means for monitoring the usage status of the household electrical appliances and suggesting appropriate maintenance and cleaning methods; a means for receiving and analyzing malfunction reports from users and suggesting self-resolved solutions and contact information; a means for predicting the optimal replacement timing based on the usage period and performance degradation data of the household electrical appliances and suggesting recommended products and retailers; and a means for recognizing the user's emotional state using emotion analysis technology, adjusting support priorities, and providing support tailored to the user. This allows users to efficiently manage their warranty and warranty documents, ensure appropriate maintenance and malfunction response, and know the optimal replacement timing. Furthermore, providing support tailored to the user's emotions can reduce stress and help them live a more comfortable life.

[0919] "User" means a person or entity that owns a household electrical appliance and provides information to manage warranty and extended warranty documents within the system.

[0920] "Household electrical appliances" refers to electrical appliances used in the home, such as washing machines, refrigerators, vacuum cleaners, etc.

[0921] A "warranty" is a document issued when purchasing household electrical appliances, which lists the product's warranty period and scope.

[0922] "Extended warranty documents" refer to documents that provide additional warranty services or extended warranties in addition to the warranty certificate.

[0923] "Means for uploading" refers to a function that allows a user to use a terminal to send image data of a warranty or long-term compensation document to the system.

[0924] "Optical character recognition technology (OCR)" refers to the technology that analyzes character information in an image and converts it into text data.

[0925] "Means of extraction" refers to the process of extracting necessary information (e.g., warranty period, serial number) from the analyzed data and registering it in the database.

[0926] "Monitoring means" refers to the function of regularly collecting and analyzing usage and performance data of household electrical appliances.

[0927] "Means to suggest maintenance timing" refers to a function that predicts when maintenance of household electrical appliances is necessary based on collected data and notifies the user.

[0928] The "means for receiving a fault report" refers to an input interface that allows a user to report a fault in a household electrical appliance to the system.

[0929] "Means for proposing self-resolving solutions" refers to the process by which the system analyzes failure data and provides the user with a method for self-resolving the problem.

[0930] "A means of predicting the time to replace an appliance based on the period of use and performance degradation data" refers to a function that analyzes the usage history and performance data of household electrical appliances and calculates the optimal time to replace them.

[0931] "Means for suggesting recommended products and retailers" refers to a function that provides the user with appropriate purchasing options and retailer information when it is time to replace their product.

[0932] "Emotion analysis technology" refers to technology that recognizes a user's emotional state from their facial expression, voice, or input content.

[0933] "Means for adjusting support priority" refers to a process for adjusting the urgency and method of support response based on the user's emotional state.

[0934] This system efficiently manages warranties and long-term warranty documents for users' home electrical appliances, and suggests appropriate maintenance, troubleshooting, and replacement timing. Furthermore, by using emotion analysis technology, it can provide customized support according to the user's emotional state.

[0935] Warranty and long-term coverage document management

[0936] Users use their smartphones or tablets to upload warranty and extended warranty documents for home appliances to the application. The device receives the uploaded image and analyzes the text information in the image using OCR (optical character recognition) technology (e.g., Google Cloud Vision API). At this time, an image processing library (e.g., OpenCV) is used to preprocess the image. The analyzed data (e.g., warranty period, serial number, and purchase store information) is structured and sent from the device to the server. The server connects to the database and inserts the received data into the appropriate tables. This allows users to check the warranty information at any time through the app.

[0937] Maintenance Notification

[0938] The device periodically collects usage data for home appliances. For example, in the case of smart home appliances, the device automatically collects data using Wi-Fi or Bluetooth and sends it to a server. The server analyzes the collected data and predicts when maintenance is required based on the appliance's maintenance cycle. When the predicted maintenance period approaches, the server generates a notification and sends it to the device. The notification includes specific maintenance procedures (e.g., filter cleaning) and information on tools.

[0939] Support for inquiries in case of breakdowns

[0940] When a user reports a malfunction of a home electrical appliance, they use the support function within the app to enter the details of the malfunction into the "Malfunction Report" form. The device asks the user questions (e.g., "Is an error message displayed?") and collects detailed information about the malfunction. The collected information is sent from the device to the server. The server analyzes the malfunction data and suggests a self-solution to the user (e.g., "The drain may be clogged with dust"). If the self-solution is ineffective, the server provides contact information for the best point of contact (e.g., the manufacturer's support center or the support desk of the store where the device was purchased).

[0941] Replacement timing prediction and recommendations

[0942] The server monitors the length of use of home electrical appliances based on their start date of use, operation data, and failure history. It analyzes usage status and error logs to detect signs of performance degradation and applies an algorithm (e.g., regression analysis) to predict when to replace the appliance. When the optimal time to replace the appliance approaches, the server generates a notification and sends it to the device. The notification provides a list of recommended products and retailer information (e.g., price comparison data and performance evaluation).

[0943] User support using emotion analysis technology

[0944] If emotion analysis technology (e.g., Affectiva) is built into the device, the device analyzes the user's facial expressions and tone of voice to recognize the user's emotional state. Specifically, the device uses a camera and microphone to record the user's facial expressions and voice and transmits them to the emotion analysis engine. Based on this information, the server adjusts support priorities and provides friendly self-help solutions or the fastest support options.

[0945] Examples of concrete examples and prompts

[0946] Examples:

[0947] One day, the washing machine stops working properly, so the user reports the malfunction using the app. The device analyzes the user's facial expressions and tone of voice, and uses emotion analysis technology to determine that the user's stress level is high. In this case, the server quickly sets the support priority high and offers the user a self-help solution using kind words or the fastest support option. Emotion analysis technology also adjusts maintenance notifications to be sent at the appropriate time, taking the user's emotions into account.

[0948] Example prompt sentence:

[0949] "When your washing machine isn't working, check the error message first."

[0950] "The filter needs to be cleaned. Please refer to the following steps."

[0951] "Your washing machine drain may be clogged. Try a self-fix solution."

[0952] In this way, the present invention efficiently manages warranties and long-term warranty documents for home electrical appliances, enabling appropriate maintenance and troubleshooting. It also helps users determine the optimal timing for replacement and provides support tailored to their emotions, helping to reduce stress and support a more comfortable life.

[0953] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0954] Step 1:

[0955] Upload warranty and extended warranty documents

[0956] Users use their smartphones or tablets to upload warranty and long-term warranty documents for home electrical appliances to the application.

[0957] Input: Image data of warranty or extended warranty documents taken by the user.

[0958] Output: Image file saved on your device.

[0959] Specific operation: The user taps the "Upload warranty card" button in the app, takes a photo of the warranty card using the camera function, and uploads it.

[0960] Step 2:

[0961] OCR text analysis

[0962] The device receives the uploaded image and analyzes the text information in the image using OCR technology (e.g., Google Cloud Vision API).

[0963] Input: An image file saved on your device.

[0964] Output: Parsed text data (e.g. warranty period, serial number, store information).

[0965] What happens: The device uses an image processing library (e.g., OpenCV) to preprocess the image and perform character recognition.

[0966] Step 3:

[0967] Structuring and transferring text data

[0968] The analyzed text data is structured and sent from the terminal to the server.

[0969] Input: Text data generated by OCR analysis.

[0970] Output: The structured data is sent to the server and stored in a database.

[0971] What it does: The device converts text data into a structured format, such as JSON, and sends it to the server, which then connects to a database and inserts the received data into the appropriate table.

[0972] Step 4:

[0973] Maintenance data collection and analysis

[0974] The terminal periodically collects usage data of the home electrical appliances and sends it to the server.

[0975] Input: Usage data obtained from home appliances.

[0976] Output: Analyzed maintenance timing forecast data.

[0977] Specific operation: The device automatically collects operational data from home appliances using Wi-Fi or Bluetooth and sends it to a server. The server analyzes the collected data and predicts when maintenance will be required based on the maintenance cycle of the home appliance.

[0978] Step 5:

[0979] Maintenance notification generation and distribution

[0980] When the server approaches the predicted maintenance period, it generates a notification and delivers it to the terminal.

[0981] Input: Maintenance forecast information based on analytical data.

[0982] Output: Maintenance notification displayed on the terminal.

[0983] Specific operation: The server creates a notification message and sends it to the device as a push notification. The notification contains information about the filter cleaning procedure and the necessary tools.

[0984] Step 6:

[0985] Receiving and analyzing malfunction reports

[0986] When a user reports a malfunction within the app, the device collects detailed information and sends it to the server.

[0987] Input: User-entered fault description and answers to follow-up questions.

[0988] Output: Self-help solutions and contact information as a result of analyzing the failure information.

[0989] Specific operation: The device acquires additional information from the user in the form of interactive questions and sends it to the server. The server then refers to a database, generates a self-solution based on past troubleshooting data, and proposes it to the user.

[0990] Step 7:

[0991] Replacement timing prediction and recommendations

[0992] The server predicts when it is time to replace household electrical appliances based on usage data.

[0993] Input: Date the appliance was first used, operation data, and failure history.

[0994] Output: Notification of replacement time and recommended product list.

[0995] Specific operation: The server uses statistical models (e.g., regression analysis) to predict signs of deterioration and the recommended replacement time for home appliances. When the optimal replacement time approaches, it creates a notification and sends it to the device, including a list of recommended products and retailer information.

[0996] Step 8:

[0997] Optimizing support through sentiment analysis

[0998] Emotion analysis technology recognizes the user's emotional state and adjusts support priorities.

[0999] Input: The user's facial expression and tone of voice.

[1000] Output: Adjusted support priority and customized support message.

[1001] How it works: The device uses a camera and microphone to record the user's facial expressions and voice, and sends them to an emotion analysis engine. The server analyzes the user's emotional data and generates a customized message based on the user's stress level, which is then provided to the user.

[1002] By explaining the specific operations and processing flow in this step-by-step manner, this system can efficiently manage home appliance warranties, perform maintenance, handle malfunctions, and suggest replacements, thereby improving user convenience.

[1003] (Application example 2)

[1004] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1005] Conventional management systems for home appliance warranties and long-term warranty documents are inconvenient for many users and lack the functionality to provide fast and effective support in the event of breakdowns or maintenance. Furthermore, support does not take into account the psychological state of the user, making it difficult to provide appropriate support, especially during busy periods or emergencies. Furthermore, when it comes to security devices, warranty management and fast response in the event of breakdowns are required.

[1006] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1007] In this invention, the server includes: means for users to upload their home appliance warranties or extended warranty documents; means for analyzing the uploaded documents using recognition technology and extracting warranty-related information; means for managing the extracted information for each user and providing it as needed; means for monitoring the usage and environment of the home appliance and suggesting appropriate maintenance times and work methods; means for receiving and analyzing reports from users when a malfunction occurs and suggesting self-resolved solutions and contact information; means for predicting the optimal replacement timing based on the duration of use and performance degradation data of the home appliance and suggesting recommended products and optimal retailers; means for managing the device warranties, recognizing the user's emotional state, and providing appropriate support; means for setting priorities according to the user's emotional state in the event of a security device malfunction and providing a prompt response; and means for providing security services using an emotion engine. This allows users to efficiently manage their warranties and extended warranty documents and receive prompt and appropriate support in the event of a malfunction or maintenance, particularly for security devices, enabling warranty management and prompt response in the event of a malfunction.

[1008] "User" refers to an individual or organization that uses the system.

[1009] "Home appliances" refers to electrical appliances used in ordinary households, such as refrigerators, washing machines, air conditioners, and televisions.

[1010] A "warranty" is a document that guarantees the quality and performance of a product and states that if a product breaks or is damaged within a certain period of time, it will be repaired or replaced free of charge.

[1011] "Extended warranty documents" refer to contracts or certificates that extend the product warranty for a certain period of time.

[1012] "Means of uploading" refers to the function of sending images and data to a cloud server, etc. using a smartphone or tablet.

[1013] "Recognition technology" refers to technology for analyzing images and text data, such as OCR (optical character recognition) and image analysis technology.

[1014] "Means for extracting" refers to the function of extracting necessary information from an image using recognition technology.

[1015] "Means for managing information for each user" refers to the function of linking extracted information to each user and storing and managing it in a database.

[1016] "Means for providing" refers to the function of immediately displaying or providing stored information when a user needs the information.

[1017] "Usage status" refers to data on the actual usage of home appliances, such as operating time and frequency of use.

[1018] "Environment" refers to data on external factors, including the location where the home appliance is placed and the conditions of use.

[1019] "Means to suggest maintenance timing" refers to a function that predicts and notifies appropriate maintenance timing based on usage and environmental data.

[1020] "Means for suggesting work methods" refers to a function that provides users with specific procedures and tools for maintenance.

[1021] "Means for accepting reports" refers to the functionality that allows users to input and transmit faults and problems into the system.

[1022] "Means of analysis" refers to the function of analyzing the reported information and identifying the cause of the failure and how to deal with it.

[1023] "Means for suggesting self-solutions" refers to a function that suggests ways for users to solve malfunctions or problems on their own.

[1024] "Means to suggest contact points" refers to a function that provides contact information for the most appropriate support desk when you are unable to resolve the issue yourself.

[1025] "Usage period" refers to data on the period of time that a home appliance is actually used after it is purchased.

[1026] "Performance degradation data" refers to data that indicates a decline in performance, such as the usage time or malfunction history of a home appliance.

[1027] "Means for predicting replacement timing" refers to a function that predicts the optimal time to replace a product based on the length of use and performance degradation data.

[1028] "Means for suggesting recommended products" refers to the function of listing and providing products that meet the user's needs when replacement is required.

[1029] "Means to suggest the best retailer" refers to the function of providing information on stores that sell recommended products at the most reasonable prices.

[1030] "Emotional state" refers to the psychological state of the user as judged from facial expressions, tone of voice, input content, etc.

[1031] "Emotion engine" refers to technology that recognizes the user's emotional state and adjusts support priorities and response methods.

[1032] "Security devices" refers to devices used to secure a home or business, such as security cameras and door locks.

[1033] An embodiment of the present invention will be described.

[1034] This invention is a system that allows users to efficiently manage warranties or long-term warranty documents for home appliances and perform maintenance or troubleshooting in a timely manner. It also has the feature of recognizing the user's emotional state and providing prompt and appropriate support. The system of the present invention includes the following means.

[1035] 1. Upload and analyze the warranty card

[1036] First, users upload their appliance warranty or extended warranty documents using their smartphone or tablet. The uploaded documents are then analyzed using OCR technology (e.g., Tesseract OCR), and warranty-related information, such as the warranty period, serial number, and purchase store information, is extracted as text. The extracted information is managed for each user and securely stored in a cloud database. This allows users to instantly check warranty information whenever needed.

[1037] 2. Maintenance Notifications and Suggestions

[1038] The server monitors the usage status and environmental data of the home appliances to detect signs of deterioration or failure. This involves analyzing the log data and usage frequency emitted by the appliances. For example, it monitors the operating time of a refrigerator compressor and the number of times a washing machine is used. The server analyzes the collected data and predicts the appropriate time for maintenance. When the predicted maintenance time approaches, it notifies the user using a notification service such as Firebase and suggests specific work methods.

[1039] 3. Support for inquiries in the event of a breakdown

[1040] When a malfunction occurs, the user uses the malfunction reporting function within the app and enters details of the problem. The server receives this information, analyzes it, and proposes a self-solution as quickly as possible. If self-solution is difficult, the server suggests the most appropriate point of contact (for example, contact information for the manufacturer's support center or the store where the device was purchased). Furthermore, an emotion engine (for example, Google Cloud AutoML) that recognizes the user's emotional state is used to determine the user's level of stress or anxiety. Depending on the user's emotional state, the server quickly raises the priority of support and takes appropriate action.

[1041] 4. Replacement timing prediction and recommendations

[1042] The server predicts the optimal time to replace the appliance based on its usage period and performance degradation data. It analyzes statistical data and the usage of the appliance to detect signs of deterioration. Based on this information, when the optimal time to replace the appliance approaches, it provides the user with a list of recommended products and information on the best retailers. The list includes product performance evaluations and price comparison data, allowing the user to make the most economical choice.

[1043] 5. Security Devices and Emotion Engines

[1044] The present invention is also effective in managing warranty certificates for security devices. When a user reports a malfunction of a security device, the emotion engine analyzes the user's facial expression and tone of voice to determine their emotional state. If the engine determines that the user's stress level is high, it increases the priority of support and provides a prompt response.

[1045] As a concrete example, consider a situation where a washing machine breaks down. The user opens the app and fills in the breakdown report form with details. The server analyzes the user's facial expressions and tone of voice, and uses an emotion engine to determine that the user's stress level is high. In this case, the server sets the support priority high and offers a kind self-help solution or the fastest support option.

[1046] An example of a prompt sentence is, "Analyze the user's current emotional state based on facial expressions and voice data, and if the stress level is high, set a high priority for support, and provide the user with troubleshooting procedures and the fastest support options in user-friendly language."

[1047] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1048] Step 1:

[1049] Users scan the warranty or extended warranty document of their home appliance with a smartphone or tablet and upload it to the system. The input data is an image file of the warranty or extended warranty document. The output is an image file that can be analyzed.

[1050] Step 2:

[1051] The server receives the uploaded image and analyzes the text information in the image using OCR (Optical Character Recognition) technology. The input data is the image file. The output is text data such as the extracted warranty period, serial number, and purchase store information. Specifically, the server uses Tesseract OCR to extract the text.

[1052] Step 3:

[1053] The server structures the extracted text data and stores it in a database. The input data is the text information extracted by OCR. The output is the structured information registered in the database. Specifically, the text information is inserted into a MySQL database.

[1054] Step 4:

[1055] The server periodically monitors the usage status and environmental data of the home appliances. The input data is usage data and sensor data obtained from the home appliances. The output is information on predicted maintenance times. Specifically, the server communicates with the home appliances, collects log data, and analyzes it.

[1056] Step 5:

[1057] Based on the analyzed data, the server predicts the appropriate maintenance time and notifies the user. The input data is the analysis results of usage status and environmental data. The output is a maintenance notification sent to the user. Specifically, the notification is sent to the user's smartphone using a notification service such as Firebase.

[1058] Step 6:

[1059] When a malfunction occurs, the user enters details into a malfunction report form within the app. The input data is text information about the malfunction. The output is the malfunction data that is sent to the server. Specifically, the user opens the app and submits the malfunction report.

[1060] Step 7:

[1061] The server receives the failure data and uses an emotion engine to analyze the user's emotional state. The input data is the failure report and the user's emotional data (facial expressions and tone of voice). The output is the result of the user's emotional state assessment. Specifically, emotion analysis is performed using Google Cloud AutoML.

[1062] Step 8:

[1063] The server sets support priorities according to the user's emotional state and responds promptly and appropriately. The input data is the user's emotional state assessment result and fault data. The output is the support priority setting and specific countermeasures. Specific operations include suggesting a self-solution to resolve the fault, and if that does not resolve the issue, directing the user to the most appropriate support desk.

[1064] Step 9:

[1065] The server predicts the optimal time to replace appliances based on their usage period and performance degradation data. The input data is usage period data and performance degradation data. The output is the replacement period and a list of recommended products. Specifically, it references statistical data and applies a prediction algorithm.

[1066] Step 10:

[1067] The server provides users with a list of recommended products and information on the best retailers. The input data is the predicted replacement timing and market data. The output is a list of recommended products and retailer information sent to the user. Specifically, it presents product performance evaluations and price comparison data.

[1068] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1069] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1070] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1071] [Third embodiment]

[1072] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1073] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1074] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1075] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1076] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1077] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1078] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1079] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1080] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1081] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1082] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1083] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1084] As an embodiment of the present invention, a system for managing warranties and long-term warranty documents for home appliances and improving user convenience will be described. This system allows users to upload their warranties and long-term warranty documents for home appliances, and through a series of processes that automatically analyze and manage this information, it supports maintenance and responses in the event of a breakdown and suggests the optimal time to replace the appliance.

[1085] Management of warranties and long-term compensation documents

[1086] First, users upload their appliance warranty or extended warranty documents to the app using their smartphone or tablet. The device receives the uploaded image and analyzes it using image recognition technology (such as OCR). Specifically, it uses an image processing library to extract text information (such as warranty period, serial number, and store information) and generates it as structured data. The extracted data is registered in a database managed by the server. This allows users to check the warranty information at any time through the app.

[1087] Maintenance Notifications and Suggestions

[1088] The device periodically (e.g., daily or weekly) collects usage data from the appliance. In the case of smart appliances, the device communicates with the appliance and automatically acquires the data. The server analyzes this data and predicts when maintenance will be required based on the maintenance cycle of each appliance. When the predicted maintenance period approaches, the server generates a notification and sends it to the device. The notification provides the user with specific instructions and tools for the appropriate maintenance work (e.g., cleaning the filter).

[1089] Support for inquiries in case of breakdowns

[1090] When a home appliance breaks down, the user opens the support function within the app and enters the details of the malfunction into the "Report a Malfunction" form. The device then presents questions to the user and collects details of the malfunction, such as "Is an error message displayed?" The server then analyzes the collected malfunction data and suggests possible self-solutions to the user (e.g., "The drain may be clogged with dust"). If the self-solutions are ineffective, the server also provides contact information for the most appropriate point of contact (the manufacturer's support center or the support desk of the store where the appliance was purchased). This allows the user to quickly and efficiently deal with the malfunction.

[1091] Replacement timing prediction and recommendations

[1092] The server monitors the length of time an appliance has been in use based on its start date, operational data, and failure history. It analyzes usage and error logs to detect signs of performance degradation. Based on the analyzed data, the server applies an algorithm to predict when it is time to replace the appliance. For example, it references statistical data such as "Model X washing machines typically begin to deteriorate after about seven years." When the optimal time to replace the appliance approaches, the server generates a notification and sends it to the device. The notification provides the user with a list of recommended products and retailer information. The product list includes price comparison data and performance evaluations. This allows users to replace their appliances at the appropriate time and make economical choices.

[1093] As described above, the present invention provides a system that efficiently manages warranties and long-term warranty documents for home appliances, supports maintenance and troubleshooting, and suggests the optimal time to replace them. This system allows users to use home appliances with peace of mind, greatly improving the convenience of their lives.

[1094] The processing flow will be explained below.

[1095] Management of warranties and long-term compensation documents

[1096] Step 1:

[1097] Users take a photo of the home appliance warranty or long-term warranty document with their smartphone or tablet and upload it to the app.

[1098] Step 2:

[1099] The device receives the uploaded image and begins analyzing it using OCR (Optical Character Recognition) technology, recognizing text information using an image processing library.

[1100] Step 3:

[1101] The device analyzes the text information extracted from the image (warranty period, serial number, purchase store information, etc.) and converts it into structured data.

[1102] Step 4:

[1103] The server receives the structured data sent from the terminal and registers it in a database for each user.

[1104] Step 5:

[1105] Whenever necessary, the user opens the app and checks the registered warranty information.

[1106] Maintenance Notifications and Suggestions

[1107] Step 1:

[1108] The device periodically (e.g., daily or weekly) collects usage data from the home appliance. In the case of smart home appliances, the device automatically acquires data through communication with the appliance.

[1109] Step 2:

[1110] The device sends collected usage data to a server, including operating time, number of uses, error logs, etc.

[1111] Step 3:

[1112] The server analyzes the data it receives and predicts the next maintenance date based on the maintenance cycle for each appliance.

[1113] Step 4:

[1114] When the server approaches the predicted maintenance time, it generates a maintenance notice and delivers it to the terminal.

[1115] Step 5:

[1116] The device displays maintenance notifications to the user, providing specific work instructions (e.g., cleaning the filter) and the necessary tools.

[1117] Support for inquiries in case of breakdowns

[1118] Step 1:

[1119] When the user recognizes that their home appliance is malfunctioning, they open the support function within the app and enter details of the malfunction into the "Report a Malfunction" form.

[1120] Step 2:

[1121] The device presents the user with questions related to the malfunction and collects specific malfunction information, such as "Is an error message displayed?"

[1122] Step 3:

[1123] The terminal transmits the collected failure data to the server.

[1124] Step 4:

[1125] The server analyzes the failure data and suggests a self-solution to the user (e.g., "It may be due to dust clogging the drain").

[1126] Step 5:

[1127] If the server's self-help efforts are not effective, it will provide contact information for the appropriate point of contact (such as the manufacturer's support center or the support desk at the store where the product was purchased).

[1128] Replacement timing prediction and recommendations

[1129] Step 1:

[1130] The server monitors the period of use of the appliance based on the date of use, operation data, and failure history.

[1131] Step 2:

[1132] The server analyzes the monitoring data to detect signs of performance degradation and also refers to statistical data to predict performance degradation.

[1133] Step 3:

[1134] When the time for replacement approaches, the server generates a notification for the user and sends it to the terminal.

[1135] Step 4:

[1136] The terminal displays a replacement notice to the user and provides a list of recommended products and information on the cheapest retailer.

[1137] Step 5:

[1138] Based on the information presented, the user selects the most suitable home appliance and considers purchasing it.

[1139] The above is a detailed flow of the program processing of this system.

[1140] Example 1

[1141] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1142] Modern homes use many electrical appliances, and it is necessary to properly store their warranties and long-term warranty documents, and efficiently manage their usage, maintenance, and troubleshooting. However, doing this on your own is cumbersome, and missing appropriate maintenance intervals increases the risk of breakdowns, making it difficult to determine the optimal time to replace appliances. For this reason, a system is needed that increases user convenience and streamlines the lifecycle management of electrical appliances.

[1143] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1144] In this invention, the server includes: means for users to upload warranty or extended warranty documents for electrical appliances; means for analyzing the uploaded documents using optical character recognition technology and extracting warranty-related information; means for managing the extracted information for each user and providing it when needed; means for monitoring the usage status and living environment of electrical appliances and suggesting appropriate maintenance times and cleaning methods; means for receiving and analyzing malfunction reports from users in the event of a malfunction and suggesting self-resolved solutions and contact information using a machine learning model; and means for predicting the optimal replacement time based on the duration of use and performance degradation data of the electrical appliances and suggesting recommended products and optimal retailers. This makes it easier for users to manage their warranty and warranty documents, enabling them to perform maintenance and replacement at the appropriate time.

[1145] "Upload" refers to the operation of a user sending a digital image of an electrical product warranty or extended warranty document to the system through the application.

[1146] "Optical character recognition" is a technology for extracting text information from images, specifically OCR (Optical Character Recognition).

[1147] "Warranty information" refers to important data such as the warranty period, serial number, and purchase store information listed on the warranty card or compensation document for an electrical product.

[1148] "Structured data" refers to a data format that is organized so that extracted information can be stored in a table or database.

[1149] "Monitoring" refers to the act of continuously observing the usage of electrical appliances and living environments and collecting necessary data.

[1150] "Maintenance period" refers to the time when an electrical appliance should be inspected, cleaned, or repaired to keep it functioning properly.

[1151] "Fault reporting" refers to the act of a user notifying the system of detailed information about abnormal operation or malfunction of an electrical appliance.

[1152] A "machine learning model" refers to an algorithm that analyzes collected data, finds patterns, and predicts future trends.

[1153] "Performance degradation data" refers to information regarding malfunctions or performance degradation that occur during the use of an electrical product.

[1154] The "optimal replacement time" refers to the point at which the performance of an electrical appliance deteriorates and it becomes economically advantageous to replace it with a new product rather than continuing to repair and maintain it.

[1155] "Recommended products" refer to products that the system recommends as having the same or better performance as the user's current electrical products.

[1156] "Best seller" refers to the store or online shop where you can purchase the recommended products at the best conditions.

[1157] As an embodiment of the present invention, a system for managing warranty certificates and long-term warranty documents for electrical appliances and improving user convenience will be described. This system allows users to upload warranty certificates and long-term warranty documents for electrical appliances, and through a series of processes that automatically analyze and manage this information, it supports maintenance and responses in the event of a breakdown and suggests the optimal time to replace the appliance.

[1158] Management of warranties and long-term compensation documents

[1159] First, users upload the warranty card or extended warranty document for their electrical appliances to the app using a smartphone or tablet. The device receives the uploaded image and analyzes it using image recognition technology (such as OCR). Specifically, an image processing library is used to extract text information (such as warranty period, serial number, and store information) and generate it as structured data. The extracted data is then registered in a database managed by the server. This allows users to check the warranty information at any time through the app.

[1160] Maintenance Notifications and Suggestions

[1161] The device periodically (e.g., daily or weekly) collects appliance usage data. In the case of smart home appliances, the device communicates with the appliance to automatically obtain the data. The server analyzes this data and predicts when maintenance is required based on the maintenance cycle of each appliance. When the predicted maintenance period approaches, the server generates a notification and sends it to the device. The notification provides the user with specific instructions and tools for the appropriate maintenance work (e.g., cleaning the filter).

[1162] Support for inquiries in case of breakdowns

[1163] When an electrical appliance breaks down, the user opens the support function within the app and enters the details of the malfunction into the "Report a malfunction" form. The device then presents questions to the user and collects details of the malfunction, such as "Is an error message displayed?" The server then analyzes the collected malfunction data and suggests possible self-solutions to the user (e.g., "The drain may be clogged with dust"). If the self-solutions are ineffective, the server also provides contact information for the most appropriate point of contact (the manufacturer's support center or the support desk of the store where the appliance was purchased). This allows the user to quickly and efficiently resolve the malfunction.

[1164] Replacement timing prediction and recommendations

[1165] The server monitors the length of time an appliance has been in use based on its start date, operational data, and fault history. It analyzes usage and error logs to detect signs of performance degradation. Based on the analyzed data, the server applies an algorithm to predict when it's time to replace it. For example, it references statistical data such as "Model X refrigerators typically begin to deteriorate after about seven years." When the optimal time to replace an appliance approaches, the server generates a notification and sends it to the device. The notification provides the user with a list of recommended products and retailer information. The product list includes price comparison data and performance ratings. This allows users to replace appliances at the appropriate time and make economical choices.

[1166] Specific examples

[1167] A user uploads a refrigerator warranty, and the device extracts that the warranty period is "May 1, 2023 to May 1, 2025." The server predicts when the washing machine filter needs cleaning and notifies the user, "Please clean the filter." When a user reports error code "E05," the server suggests a self-solution, saying, "Check the drain hose." When seven years have passed since the refrigerator was first used, the server notifies the user, "It's time to replace it," and displays a list of recommended new refrigerators.

[1168] Prompt Sentence Examples

[1169] "Just take a picture of your appliance's warranty card with your camera and upload it to the app. The warranty period and serial number will be automatically extracted."

[1170] "It collects data every week and notifies you when it's time to maintain your appliances. It lets you know when it's time to clean your filters."

[1171] "If your appliance breaks down, please report the problem through the app and we will suggest a solution for you."

[1172] "It analyzes the usage period of home appliances and notifies you of the optimal time to replace them. It also provides a list of recommended products."

[1173] As described above, the present invention provides a system that efficiently manages warranties and long-term warranty documents for electrical appliances, supports maintenance and troubleshooting, and suggests the optimal time to replace them. This allows users to use electrical appliances with peace of mind, greatly improving the convenience of their lives.

[1174] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1175] Step 1:

[1176] User uploads warranty card

[1177] A user uploads an image of the warranty or extended warranty document for an electrical appliance using an app on their smartphone or tablet. Specifically, the user taps the camera button in the app to launch the camera and take a photo of the warranty or select an existing image.

[1178] Input: Warranty card image taken or selected by the user

[1179] Output: Uploaded warranty image

[1180] Step 2:

[1181] The device receives the image

[1182] The device receives the image uploaded by the user and prepares it for image recognition processing.

[1183] Input: Uploaded warranty image

[1184] Output: warranty card image ready for analysis

[1185] Step 3:

[1186] The device analyzes the image

[1187] The device performs preprocessing using an image processing library (e.g., OpenCV). This includes adjusting the resolution, removing noise, and identifying text regions. It then uses an OCR engine (e.g., Tesseract) to extract text from the image. This allows information such as the warranty period, serial number, and store information to be obtained.

[1188] Input: warranty card image ready for analysis

[1189] Data processing: image resolution adjustment, noise removal, text area identification

[1190] Data Calculation: Text Extraction Using an OCR Engine

[1191] Output: Extracted text data

[1192] Step 4:

[1193] The device structures the data

[1194] The text data extracted by the device is converted into structured data such as JSON format. Specifically, the warranty period, serial number, and purchase store information are stored in the appropriate fields.

[1195] Input: Extracted text data

[1196] Data processing: structuring text data into JSON format

[1197] Output: Structured data in JSON format

[1198] Step 5:

[1199] The device sends data to the server

[1200] The device sends the structured data to the server using HTTPS as the communication protocol.

[1201] Input: Structured data in JSON format

[1202] Output: Structured data sent to the server

[1203] Step 6:

[1204] The server stores the data

[1205] The server stores the received structured data in a database management system (e.g. MySQL). The server first checks the data for errors, and if there are no errors, stores it in the database in the appropriate format.

[1206] Input: Structured data sent to the server

[1207] Data processing: Checking data for errors and converting it to the appropriate format

[1208] Output: Data stored in the database

[1209] Step 7:

[1210] Device collects usage data

[1211] The device periodically collects usage data from the appliance. In the case of smart appliances, the device communicates with the appliance to automatically obtain data, including operating time and error logs.

[1212] Input: Usage data collected from electronic devices

[1213] Output: Usage data stored on the device

[1214] Step 8:

[1215] Server maintenance forecast

[1216] The server analyzes the usage data sent from the device, calculates the maintenance cycle using an analysis tool (e.g., Pandas), and calculates the predicted maintenance time.

[1217] Input: Usage data stored on the device

[1218] Data calculation: Usage data analysis, maintenance cycle calculation

[1219] Output: Estimated maintenance time

[1220] Step 9:

[1221] Server generates maintenance notice

[1222] The server generates a notification based on the predicted maintenance time, specifically a message containing the maintenance procedure and required tools.

[1223] Input: Estimated maintenance time

[1224] Data processing: Notification message generation

[1225] Output: The generated notification message

[1226] Step 10:

[1227] The server sends a notification

[1228] The server generates a notification message and sends it to the device, which receives it and displays it to the user as a push notification.

[1229] Input: The generated notification message

[1230] Output: Notification message sent to the terminal

[1231] Step 11:

[1232] User reports a malfunction

[1233] The user accesses the support function within the app and enters detailed information about the problem into the "Report a Problem" form.

[1234] Input: The details of the malfunction entered in the "Malfunction Report" form

[1235] Output: The entered fault report data

[1236] Step 12:

[1237] The device collects failure data

[1238] The device asks the user additional questions to gather more detailed information about the failure, such as "Is an error message displayed?"

[1239] Input: Fault report data and responses to additional questions

[1240] Output: Collected detailed failure data

[1241] Step 13:

[1242] The server proposes a self-solution

[1243] The device sends the collected failure data to a server, which then analyzes the data using a machine learning model and proposes a self-solution to the user.

[1244] Input: Collected detailed failure data

[1245] Data calculation: failure data analysis, self-resolved solution generation

[1246] Output: Self-solution proposal

[1247] Step 14:

[1248] The server provides contact information

[1249] If self-help measures are not effective, the server provides contact information for the best place to contact (the manufacturer's support center or the support desk at the store where you purchased the product).

[1250] Input: Failure data and self-solving results

[1251] Output: Best contact information

[1252] Step 15:

[1253] The server monitors the usage period

[1254] The server monitors the electrical appliances based on their start date of use, operation data, and failure history.

[1255] Input: Start date of use, operation data, failure history

[1256] Output: Monitoring result data

[1257] Step 16:

[1258] Server replacement time prediction

[1259] The server analyzes the monitoring results data and uses data analysis tools and algorithms to predict when it is time to replace the device.

[1260] Input: Monitoring result data

[1261] Data calculation: Performance degradation analysis, replacement timing prediction

[1262] Output: Predicted replacement time

[1263] Step 17:

[1264] The server generates a recommendation notification

[1265] When the time for replacement approaches, the server generates a notification message including a list of recommended products and information on the best retailer.

[1266] Input: Predicted replacement time

[1267] Data processing: Generation of recommended product list and retailer information

[1268] Output: Recommendation notification message

[1269] Step 18:

[1270] The server sends a notification

[1271] The server generates a recommendation notification and sends it to the device, which receives it and displays it to the user as a push notification.

[1272] Input: Generated recommendation notification message

[1273] Output: Recommendation notification message sent to the device

[1274] (Application example 1)

[1275] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1276] Users often have difficulty managing or lose their home appliance warranties and long-term warranty documents, making it difficult to quickly check the warranty details when needed. Furthermore, notifications regarding home appliance maintenance schedules and troubleshooting are not provided in a timely manner, reducing user convenience. Furthermore, it is difficult for users to properly determine when to replace their home appliances, which can lead to financial losses if they miss the optimal timing. These issues need to be resolved.

[1277] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1278] In this invention, the server includes: means for users to upload warranty or extended warranty documents for electrical appliances; means for analyzing the uploaded documents using image recognition technology and extracting warranty-related information; means for managing the extracted information for each user and providing it as needed; means for monitoring the usage status and installation environment of the electrical appliance and suggesting appropriate maintenance times and cleaning methods; means for receiving and analyzing malfunction reports from users and suggesting self-resolved solutions and contact information; means for predicting the optimal replacement time based on the electrical appliance's usage period and performance degradation data and suggesting recommended products and the lowest-priced retailers; means for extracting warranty information from warranty images using a smartphone and storing it in a database; means for providing maintenance notifications and support in the event of a malfunction; and means for predicting the replacement time based on the start date of use and usage data and providing a list of recommended products. This allows users to efficiently manage warranty and warranty documents, receive timely notifications for maintenance and malfunction response, and use electrical appliances economically and efficiently by knowing the optimal replacement time.

[1279] "Electrical appliances" is a general term for devices and equipment that require electrical power for use.

[1280] A "warranty" is a document stating that the manufacturer or seller guarantees the quality and performance of a product for a certain period of time.

[1281] An "extended warranty document" is a contract that guarantees the repair or replacement of a product for a period beyond the normal warranty period.

[1282] "User" means any person or entity that uses a system or device.

[1283] "Image recognition technology" is a general term for technology that analyzes and extracts specific information from images, and generally includes OCR (optical character recognition).

[1284] "Uploading" is the act of sending data from a local device to a network or server.

[1285] "Warranty information" is data about warranty details such as the warranty period, serial number, and purchase store information, which are written on the warranty card.

[1286] "Extraction" refers to the process of extracting necessary information from images or text.

[1287] "Management" is the act of controlling the organization, storage, access, and updating of information.

[1288] "Providing" refers to supplying information or services required by users.

[1289] "Monitoring" is the process of continuously observing and recording the state of a system or device.

[1290] "Maintenance period" refers to the period when a device or equipment requires regular maintenance, inspection, or repair.

[1291] "Cleaning method" refers to the selection of procedures and materials for cleaning equipment or devices.

[1292] "Fault reporting" is the act of a user notifying a system operator of a fault or problem in a device or system.

[1293] "Analysis" is the process of examining and evaluating data to understand its structure and meaning.

[1294] "Self-help" is a method by which a user can solve a problem on their own without professional support.

[1295] "Contact Information" refers to the location or person that a user should contact when seeking problem resolution or support.

[1296] "Period of use" refers to the period during which a device or system is actually used.

[1297] "Performance degradation data" is data that records how the performance of a device or system deteriorates over time.

[1298] "Replacement time" is the best time to replace an existing device or system with a new one.

[1299] A "recommended product" is a product that the system determines to be most appropriate in a particular situation or under a particular condition, based on specific user needs.

[1300] "Best Dealer" refers to the dealer that offers the lowest price for a particular product.

[1301] A "smartphone" is a type of mobile phone, a handheld device that combines advanced computing power and communication capabilities.

[1302] A database is a system that organizes and stores information and allows it to be retrieved in any format required.

[1303] "Date of Use" means the date on which a device or system is first used.

[1304] As an embodiment of this invention, we will explain the "Home Appliance Security Support" system. This system allows users to manage warranties and long-term warranty documents for electrical appliances, and efficiently recommends maintenance, troubleshooting, and optimal replacement timing. The main hardware and software that make up this system are described below.

[1305] Hardware

[1306] Smartphone

[1307] server

[1308] software

[1309] Python

[1310] OpenCV

[1311] Tesseract OCR

[1312] SQLite

[1313] Requests

[1314] Processing flow

[1315] 1. Upload and analyze the warranty card

[1316] Users use their smartphones to take photos of electrical appliance warranties and long-term warranty documents and upload them to the app. The device receives the uploaded images and analyzes them using image recognition technology (OCR).

[1317] Specifically, OpenCV and Tesseract OCR are used to extract text information (warranty period, serial number, purchase store information, etc.) from uploaded images and generate it as structured data.

[1318] The extracted data is registered in a database (SQLite) managed by the server, allowing users to check warranty information at any time via their smartphone.

[1319] 2. Maintenance Notifications and Suggestions

[1320] The server continuously monitors the usage status and installation environment of electrical appliances to predict maintenance times and cleaning methods.

[1321] When maintenance is due, the server generates a notification and sends it to the smartphone, including specific instructions for the appropriate maintenance task (e.g., cleaning the filter) and the tools required.

[1322] 3. Support in case of breakdown

[1323] When an electrical appliance breaks down, users can use the support function in the smartphone app to "report the malfunction."

[1324] The device asks the user questions to gather details about the failure, such as "Are you seeing any error messages?"

[1325] The server analyzes the collected failure data and suggests possible solutions to the user (for example, "The drain may be clogged with dust.") If the solutions are ineffective, the server provides the user with the best contact point (the manufacturer's support center).

[1326] 4. Replacement timing prediction and recommendations

[1327] The server monitors the period of use of the appliance based on the date of use, operation data, and failure history.

[1328] The system analyzes usage and error logs to detect signs of performance degradation and applies an algorithm to predict when it's time to replace the device. Users are notified of the optimal time to replace the device via email or in-app notifications.

[1329] The notification will also include a list of recommended products, including performance evaluations and price comparison data, as well as retailer information.

[1330] Examples and prompts

[1331] Specific examples

[1332] Users simply take a photo of their washing machine's warranty card with their smartphone and upload it. The app analyzes the warranty information and registers it in a database. When seven years have passed since the purchase date, the app displays a replacement notification and provides a list of recommended products.

[1333] Prompt Sentence Examples

[1334] "Please implement a program to extract warranty information from uploaded images and register the warranty period and serial number in a database. After a certain period of time, send a maintenance notification or suggest a replacement."

[1335] In this way, the "Home Appliance Security Support" system aims to enable users to efficiently manage warranties and compensation documents, receive timely notifications about maintenance and malfunctions, and know the optimal time to replace their electrical appliances, thereby enabling them to use their appliances economically and efficiently.

[1336] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1337] Step 1:

[1338] Warranty upload and analysis

[1339] A user uses a smartphone to take a photo of an electronic product's warranty or extended warranty document and upload it to the app. The image taken by the user is provided to the app as input. The device receives this and uses OpenCV and Tesseract OCR to extract text information from the image (warranty period, serial number, purchase store information, etc.). The output is structured data extracted from the warranty, which is sent to the server.

[1340] Step 2:

[1341] Saving to a database

[1342] The server stores the structured data received from step 1 (warranty period, serial number, purchase store information, etc.) in an SQLite database. As input, the structured data is given to the server, and an insert operation is performed into the database. The output is the warranty information registered in the database.

[1343] Step 3:

[1344] Maintenance forecast and notification

[1345] The server predicts when maintenance is required based on the appliance's usage and installation environment. Warranty information and usage data stored in the database are used as input. Based on this, the server predicts when the next maintenance task (e.g., filter cleaning) will be performed. When the maintenance deadline approaches, the server generates a notification and distributes it to the device. The output is a maintenance notification that is displayed on the device.

[1346] Step 4:

[1347] Fault reporting and support suggestions

[1348] When a user reports a malfunction of an electrical appliance, they use the support function within the smartphone app to make a "fault report." The user's report is provided to the app as input. The device then presents questions to the user and collects detailed information about the malfunction. Based on the information entered, the server performs an analysis and suggests a self-solution (e.g., a possible dust clog in the drain). If the self-solution is insufficient, it also provides information on the most appropriate contact point (the manufacturer's support center). The output is the self-solution presented to the user and the contact information.

[1349] Step 5:

[1350] Replacement timing prediction and recommendations

[1351] The server predicts when to replace a home appliance based on the date it was first used, its operation data, and its failure history. The inputs include warranty information, usage status, and error logs stored in a database. The server analyzes these to detect signs of performance degradation and determine the optimal time to replace the appliance. When the time to replace the appliance approaches, the server generates a notification and sends it to the device along with a list of recommended products. The output is a replacement notification and a list of recommended products to the user.

[1352] This series of processes enables users to efficiently manage warranties and compensation documents, receive timely notifications for maintenance and malfunctions, and know the optimal time to replace their electrical appliances, enabling them to use them economically and efficiently.

[1353] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1354] This invention is a system that manages home appliance warranties and long-term warranty documents to improve user convenience, and also combines an emotion engine to provide support tailored to the user's emotional state. This system allows users to upload their home appliance warranties and long-term warranty documents, and through a series of processes that automatically analyze and manage this information, it supports maintenance and responses in the event of a breakdown, and suggests the optimal time to replace the appliance. Furthermore, by using an emotion engine that recognizes the user's emotions, it provides more appropriate support.

[1355] Management of warranties and long-term compensation documents

[1356] First, users upload their appliance warranty or extended warranty documents to the app using their smartphone or tablet. The device receives the uploaded image and analyzes it using OCR (Optical Character Recognition) technology. Specifically, it uses an image processing library to extract text information (such as warranty period, serial number, and purchase store information) and generates it as structured data. The extracted data is registered in a database managed by the server. This allows users to easily check warranty information at any time through the app.

[1357] Maintenance Notifications and Suggestions

[1358] The device periodically (e.g., daily or weekly) collects usage data from the appliance. In the case of smart appliances, the device communicates with the appliance and automatically acquires the data. The server analyzes this data and predicts when maintenance will be required based on the maintenance cycle of each appliance. When the predicted maintenance period approaches, the server generates a notification and sends it to the device. The notification provides the user with specific instructions and tools for the appropriate maintenance work (e.g., cleaning the filter).

[1359] Support for inquiries in case of breakdowns

[1360] When a home appliance breaks down, the user opens the support function within the app and enters the details of the malfunction into the "Report a Malfunction" form. The device then presents questions to the user and collects details of the malfunction, such as "Is an error message displayed?" The device then sends the collected malfunction data to the server. The server analyzes the malfunction data and suggests possible self-solutions to the user (e.g., "The drain may be clogged with dust"). If the self-solutions are ineffective, the server also provides contact information for the best point of contact (the manufacturer's support center or the support desk of the store where the appliance was purchased). This allows the user to quickly and efficiently handle the malfunction.

[1361] Replacement timing prediction and recommendations

[1362] The server monitors the length of time an appliance has been in use based on its start date, operational data, and failure history. It analyzes usage and error logs to detect signs of performance degradation. Based on the analyzed data, the server applies an algorithm to predict when it is time to replace the appliance. For example, it references statistical data such as "Model X washing machines typically begin to deteriorate after about seven years." When the optimal time to replace the appliance approaches, the server generates a notification and sends it to the device. The notification provides the user with a list of recommended products and retailer information. The product list includes price comparison data and performance evaluations. This allows users to replace their appliances at the appropriate time and make economical choices.

[1363] User support using an emotion engine

[1364] If an emotion engine is built into the system, it can recognize the user's emotional state and provide more effective support. For example, when a user uses the in-app support feature after a home appliance breaks down, the emotion engine can determine the user's level of stress or anxiety from their facial expression, voice, or input.

[1365] Examples:

[1366] One day, the washing machine stops working properly, so the user uses the app to report the malfunction. The device analyzes the user's facial expressions and tone of voice and uses an emotion engine to determine that the user's stress level is high. In this case, the server quickly sets the support priority high and offers the user a self-solution using gentle language or the fastest support option. The emotion engine also takes the user's emotions into consideration when sending maintenance notifications, adjusting them to be sent at the appropriate time. This reduces the user's psychological burden and enables thorough support.

[1367] As described above, this invention is a system that efficiently manages warranties and long-term warranty documents for home appliances, supports maintenance and troubleshooting, and suggests the optimal time to replace the appliance. Furthermore, by utilizing an emotion engine, it provides support according to the user's emotional state. This system allows users to use home appliances with peace of mind, improving their quality of life.

[1368] The processing flow will be explained below.

[1369] Management of warranties and long-term compensation documents

[1370] Step 1:

[1371] Users take a photo of the home appliance warranty or long-term warranty document with their smartphone or tablet and upload it to the app.

[1372] Step 2:

[1373] The device receives the uploaded image and begins analyzing it using OCR (Optical Character Recognition) technology, recognizing text information using an image processing library.

[1374] Step 3:

[1375] The device analyzes the text information extracted from the image (warranty period, serial number, purchase store information, etc.) and converts it into structured data.

[1376] Step 4:

[1377] The server receives the structured data sent from the terminal and registers it in a database for each user.

[1378] Step 5:

[1379] Whenever necessary, the user opens the app and checks the registered warranty information.

[1380] Maintenance Notifications and Suggestions

[1381] Step 1:

[1382] The device periodically (e.g., daily or weekly) collects usage data from the home appliance. In the case of smart home appliances, the device automatically acquires data through communication with the appliance.

[1383] Step 2:

[1384] The device sends collected usage data to a server, including operating time, number of uses, error logs, etc.

[1385] Step 3:

[1386] The server analyzes the data it receives and predicts the next maintenance date based on the maintenance cycle for each appliance.

[1387] Step 4:

[1388] When the server approaches the predicted maintenance time, it generates a maintenance notice and delivers it to the terminal.

[1389] Step 5:

[1390] The device displays maintenance notifications to the user, providing specific work instructions (e.g., cleaning the filter) and the necessary tools.

[1391] Support for inquiries in case of breakdowns

[1392] Step 1:

[1393] When the user recognizes that their home appliance is malfunctioning, they open the support function within the app and enter details of the malfunction into the "Report a Malfunction" form.

[1394] Step 2:

[1395] The device presents the user with questions related to the malfunction and collects specific malfunction information, such as "Is an error message displayed?"

[1396] Step 3:

[1397] The terminal transmits the collected failure data to the server.

[1398] Step 4:

[1399] The server analyzes the failure data and suggests a self-solution to the user (e.g., "It may be due to dust clogging the drain").

[1400] Step 5:

[1401] If the server's self-help efforts are not effective, it will provide contact information for the appropriate point of contact (such as the manufacturer's support center or the support desk at the store where the product was purchased).

[1402] Replacement timing prediction and recommendations

[1403] Step 1:

[1404] The server monitors the period of use of the appliance based on the date of use, operation data, and failure history.

[1405] Step 2:

[1406] The server analyzes the monitoring data to detect signs of performance degradation and also refers to statistical data to predict performance degradation.

[1407] Step 3:

[1408] When the time for replacement approaches, the server generates a notification for the user and sends it to the terminal.

[1409] Step 4:

[1410] The terminal displays a replacement notice to the user and provides a list of recommended products and information on the cheapest retailer.

[1411] Step 5:

[1412] Based on the information presented, the user selects the most suitable home appliance and considers purchasing it.

[1413] User support using an emotion engine

[1414] Step 1:

[1415] When a user reports a malfunction of a home appliance within the app, the device captures the user's facial expressions, voice, or input and sends it to the emotion engine.

[1416] Step 2:

[1417] The emotion engine analyzes the user's emotional data to determine their stress and anxiety levels.

[1418] Step 3:

[1419] The server receives emotional data from the emotion engine and tailors its response based on the user's emotional state: if stress levels are high, it prioritizes providing immediate support.

[1420] Step 4:

[1421] The server suggests ways to respond based on the user's emotional state. For example, if the user is under high stress, it will respond with gentle words and provide a quick solution.

[1422] Step 5:

[1423] The terminal displays the response methods and specific solutions received from the server to the user, and the user solves the problem according to the most appropriate response method.

[1424] Examples:

[1425] Example 1:

[1426] Step 1: The user takes a photo of the washing machine warranty card with their smartphone and uploads it to the app.

[1427] Step 2: The device analyzes the image and extracts the warranty period and serial number.

[1428] Step 3: The server stores the extracted data in the database, and the user can check the warranty information when needed.

[1429] Example 2:

[1430] Step 1: When the washing machine breaks down, the user reports the problem through the app. The device sends the user's tone of voice to the emotion engine.

[1431] Step 2: The emotion engine determines that the user's stress level is high.

[1432] Step 3: The server provides prompt support and uses kind words to ease stress.

[1433] Step 4: The device displays the self-help solution and contact information for the support center to the user.

[1434] Example 2

[1435] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1436] In recent years, many household electrical appliances have been used, but managing the warranties and long-term warranty documents for these appliances has become cumbersome, making it difficult to know when to respond to breakdowns and when to perform maintenance. As a result, users are often unable to take appropriate action within the warranty period, resulting in unnecessary expenses and stress. Furthermore, not knowing when to replace an appliance can lead to problems such as a decline in efficiency due to the extended use of degraded appliances. Furthermore, there is a need to reduce the stress users feel when an appliance breaks down and provide effective support, but no system exists to achieve this.

[1437] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1438] In this invention, the server includes: a means for users to upload warranty or extended warranty documents for household electrical appliances; a means for analyzing the uploaded documents using optical character recognition technology and extracting warranty-related information; a means for managing the extracted information for each user and providing it as needed; a means for monitoring the usage status of the household electrical appliances and suggesting appropriate maintenance and cleaning methods; a means for receiving and analyzing malfunction reports from users and suggesting self-resolved solutions and contact information; a means for predicting the optimal replacement timing based on the usage period and performance degradation data of the household electrical appliances and suggesting recommended products and retailers; and a means for recognizing the user's emotional state using emotion analysis technology, adjusting support priorities, and providing support tailored to the user. This allows users to efficiently manage their warranty and warranty documents, ensure appropriate maintenance and malfunction response, and know the optimal replacement timing. Furthermore, providing support tailored to the user's emotions can reduce stress and help them live a more comfortable life.

[1439] "User" means a person or entity that owns a household electrical appliance and provides information to manage warranty and extended warranty documents within the system.

[1440] "Household electrical appliances" refers to electrical appliances used in the home, such as washing machines, refrigerators, vacuum cleaners, etc.

[1441] A "warranty" is a document issued when purchasing household electrical appliances, which lists the product's warranty period and scope.

[1442] "Extended warranty documents" refer to documents that provide additional warranty services or extended warranties in addition to the warranty certificate.

[1443] "Means for uploading" refers to a function that allows a user to use a terminal to send image data of a warranty or long-term compensation document to the system.

[1444] "Optical character recognition technology (OCR)" refers to the technology that analyzes character information in an image and converts it into text data.

[1445] "Means of extraction" refers to the process of extracting necessary information (e.g., warranty period, serial number) from the analyzed data and registering it in the database.

[1446] "Monitoring means" refers to the function of regularly collecting and analyzing usage and performance data of household electrical appliances.

[1447] "Means to suggest maintenance timing" refers to a function that predicts when maintenance of household electrical appliances is necessary based on collected data and notifies the user.

[1448] The "means for receiving a fault report" refers to an input interface that allows a user to report a fault in a household electrical appliance to the system.

[1449] "Means for proposing self-resolving solutions" refers to the process by which the system analyzes failure data and provides the user with a method for self-resolving the problem.

[1450] "A means of predicting the time to replace an appliance based on the period of use and performance degradation data" refers to a function that analyzes the usage history and performance data of household electrical appliances and calculates the optimal time to replace them.

[1451] "Means for suggesting recommended products and retailers" refers to a function that provides the user with appropriate purchasing options and retailer information when it is time to replace their product.

[1452] "Emotion analysis technology" refers to technology that recognizes a user's emotional state from their facial expression, voice, or input content.

[1453] "Means for adjusting support priority" refers to a process for adjusting the urgency and method of support response based on the user's emotional state.

[1454] This system efficiently manages warranties and long-term warranty documents for users' home electrical appliances, and suggests appropriate maintenance, troubleshooting, and replacement timing. Furthermore, by using emotion analysis technology, it can provide customized support according to the user's emotional state.

[1455] Warranty and long-term coverage document management

[1456] Users use their smartphones or tablets to upload warranty and extended warranty documents for home appliances to the application. The device receives the uploaded image and analyzes the text information in the image using OCR (optical character recognition) technology (e.g., Google Cloud Vision API). At this time, an image processing library (e.g., OpenCV) is used to preprocess the image. The analyzed data (e.g., warranty period, serial number, and purchase store information) is structured and sent from the device to the server. The server connects to the database and inserts the received data into the appropriate tables. This allows users to check the warranty information at any time through the app.

[1457] Maintenance Notification

[1458] The device periodically collects usage data for home appliances. For example, in the case of smart home appliances, the device automatically collects data using Wi-Fi or Bluetooth and sends it to a server. The server analyzes the collected data and predicts when maintenance is required based on the appliance's maintenance cycle. When the predicted maintenance period approaches, the server generates a notification and sends it to the device. The notification includes specific maintenance procedures (e.g., filter cleaning) and information on tools.

[1459] Support for inquiries in case of breakdowns

[1460] When a user reports a malfunction of a home electrical appliance, they use the support function within the app to enter the details of the malfunction into the "Malfunction Report" form. The device asks the user questions (e.g., "Is an error message displayed?") and collects detailed information about the malfunction. The collected information is sent from the device to the server. The server analyzes the malfunction data and suggests a self-solution to the user (e.g., "The drain may be clogged with dust"). If the self-solution is ineffective, the server provides contact information for the best point of contact (e.g., the manufacturer's support center or the support desk of the store where the device was purchased).

[1461] Replacement timing prediction and recommendations

[1462] The server monitors the length of use of home electrical appliances based on their start date of use, operation data, and failure history. It analyzes usage status and error logs to detect signs of performance degradation and applies an algorithm (e.g., regression analysis) to predict when to replace the appliance. When the optimal time to replace the appliance approaches, the server generates a notification and sends it to the device. The notification provides a list of recommended products and retailer information (e.g., price comparison data and performance evaluation).

[1463] User support using emotion analysis technology

[1464] If emotion analysis technology (e.g., Affectiva) is built into the device, the device analyzes the user's facial expressions and tone of voice to recognize the user's emotional state. Specifically, the device uses a camera and microphone to record the user's facial expressions and voice and transmits them to the emotion analysis engine. Based on this information, the server adjusts support priorities and provides friendly self-help solutions or the fastest support options.

[1465] Examples of concrete examples and prompts

[1466] Examples:

[1467] One day, the washing machine stops working properly, so the user reports the malfunction using the app. The device analyzes the user's facial expressions and tone of voice, and uses emotion analysis technology to determine that the user's stress level is high. In this case, the server quickly sets the support priority high and offers the user a self-help solution using kind words or the fastest support option. Emotion analysis technology also adjusts maintenance notifications to be sent at the appropriate time, taking the user's emotions into account.

[1468] Example prompt sentence:

[1469] "When your washing machine isn't working, check the error message first."

[1470] "The filter needs to be cleaned. Please refer to the following steps."

[1471] "Your washing machine drain may be clogged. Try a self-fix solution."

[1472] In this way, the present invention efficiently manages warranties and long-term warranty documents for home electrical appliances, enabling appropriate maintenance and troubleshooting. It also helps users determine the optimal timing for replacement and provides support tailored to their emotions, helping to reduce stress and support a more comfortable life.

[1473] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1474] Step 1:

[1475] Upload warranty and extended warranty documents

[1476] Users use their smartphones or tablets to upload warranty and long-term warranty documents for home electrical appliances to the application.

[1477] Input: Image data of warranty or extended warranty documents taken by the user.

[1478] Output: Image file saved on your device.

[1479] Specific operation: The user taps the "Upload warranty card" button in the app, takes a photo of the warranty card using the camera function, and uploads it.

[1480] Step 2:

[1481] OCR text analysis

[1482] The device receives the uploaded image and analyzes the text information in the image using OCR technology (e.g., Google Cloud Vision API).

[1483] Input: An image file saved on your device.

[1484] Output: Parsed text data (e.g. warranty period, serial number, store information).

[1485] What happens: The device uses an image processing library (e.g., OpenCV) to preprocess the image and perform character recognition.

[1486] Step 3:

[1487] Structuring and transferring text data

[1488] The analyzed text data is structured and sent from the terminal to the server.

[1489] Input: Text data generated by OCR analysis.

[1490] Output: The structured data is sent to the server and stored in a database.

[1491] What it does: The device converts text data into a structured format, such as JSON, and sends it to the server, which then connects to a database and inserts the received data into the appropriate table.

[1492] Step 4:

[1493] Maintenance data collection and analysis

[1494] The terminal periodically collects usage data of the home electrical appliances and sends it to the server.

[1495] Input: Usage data obtained from home appliances.

[1496] Output: Analyzed maintenance timing forecast data.

[1497] Specific operation: The device automatically collects operational data from home appliances using Wi-Fi or Bluetooth and sends it to a server. The server analyzes the collected data and predicts when maintenance will be required based on the maintenance cycle of the home appliance.

[1498] Step 5:

[1499] Maintenance notification generation and distribution

[1500] When the server approaches the predicted maintenance period, it generates a notification and delivers it to the terminal.

[1501] Input: Maintenance forecast information based on analytical data.

[1502] Output: Maintenance notification displayed on the terminal.

[1503] Specific operation: The server creates a notification message and sends it to the device as a push notification. The notification contains information about the filter cleaning procedure and the necessary tools.

[1504] Step 6:

[1505] Receiving and analyzing malfunction reports

[1506] When a user reports a malfunction within the app, the device collects detailed information and sends it to the server.

[1507] Input: User-entered fault description and answers to follow-up questions.

[1508] Output: Self-help solutions and contact information as a result of analyzing the failure information.

[1509] Specific operation: The device acquires additional information from the user in the form of interactive questions and sends it to the server. The server then refers to a database, generates a self-solution based on past troubleshooting data, and proposes it to the user.

[1510] Step 7:

[1511] Replacement timing prediction and recommendations

[1512] The server predicts when it is time to replace household electrical appliances based on usage data.

[1513] Input: Date the appliance was first used, operation data, and failure history.

[1514] Output: Notification of replacement time and recommended product list.

[1515] Specific operation: The server uses statistical models (e.g., regression analysis) to predict signs of deterioration and the recommended replacement time for home appliances. When the optimal replacement time approaches, it creates a notification and sends it to the device, including a list of recommended products and retailer information.

[1516] Step 8:

[1517] Optimizing support through sentiment analysis

[1518] Emotion analysis technology recognizes the user's emotional state and adjusts support priorities.

[1519] Input: The user's facial expression and tone of voice.

[1520] Output: Adjusted support priority and customized support message.

[1521] How it works: The device uses a camera and microphone to record the user's facial expressions and voice, and sends them to an emotion analysis engine. The server analyzes the user's emotional data and generates a customized message based on the user's stress level, which is then provided to the user.

[1522] By explaining the specific operations and processing flow in this step-by-step manner, this system can efficiently manage home appliance warranties, perform maintenance, handle malfunctions, and suggest replacements, thereby improving user convenience.

[1523] (Application example 2)

[1524] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1525] Conventional management systems for home appliance warranties and long-term warranty documents are inconvenient for many users and lack the functionality to provide fast and effective support in the event of breakdowns or maintenance. Furthermore, support does not take into account the psychological state of the user, making it difficult to provide appropriate support, especially during busy periods or emergencies. Furthermore, when it comes to security devices, warranty management and fast response in the event of breakdowns are required.

[1526] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1527] In this invention, the server includes: means for users to upload their home appliance warranties or extended warranty documents; means for analyzing the uploaded documents using recognition technology and extracting warranty-related information; means for managing the extracted information for each user and providing it as needed; means for monitoring the usage and environment of the home appliance and suggesting appropriate maintenance times and work methods; means for receiving and analyzing reports from users when a malfunction occurs and suggesting self-resolved solutions and contact information; means for predicting the optimal replacement timing based on the duration of use and performance degradation data of the home appliance and suggesting recommended products and optimal retailers; means for managing the device warranties, recognizing the user's emotional state, and providing appropriate support; means for setting priorities according to the user's emotional state in the event of a security device malfunction and providing a prompt response; and means for providing security services using an emotion engine. This allows users to efficiently manage their warranties and extended warranty documents and receive prompt and appropriate support in the event of a malfunction or maintenance, particularly for security devices, enabling warranty management and prompt response in the event of a malfunction.

[1528] "User" refers to an individual or organization that uses the system.

[1529] "Home appliances" refers to electrical appliances used in ordinary households, such as refrigerators, washing machines, air conditioners, and televisions.

[1530] A "warranty" is a document that guarantees the quality and performance of a product and states that if a product breaks or is damaged within a certain period of time, it will be repaired or replaced free of charge.

[1531] "Extended warranty documents" refer to contracts or certificates that extend the product warranty for a certain period of time.

[1532] "Means of uploading" refers to the function of sending images and data to a cloud server, etc. using a smartphone or tablet.

[1533] "Recognition technology" refers to technology for analyzing images and text data, such as OCR (optical character recognition) and image analysis technology.

[1534] "Means for extracting" refers to the function of extracting necessary information from an image using recognition technology.

[1535] "Means for managing information for each user" refers to the function of linking extracted information to each user and storing and managing it in a database.

[1536] "Means for providing" refers to the function of immediately displaying or providing stored information when a user needs the information.

[1537] "Usage status" refers to data on the actual usage of home appliances, such as operating time and frequency of use.

[1538] "Environment" refers to data on external factors, including the location where the home appliance is placed and the conditions of use.

[1539] "Means to suggest maintenance timing" refers to a function that predicts and notifies appropriate maintenance timing based on usage and environmental data.

[1540] "Means for suggesting work methods" refers to a function that provides users with specific procedures and tools for maintenance.

[1541] "Means for accepting reports" refers to the functionality that allows users to input and transmit faults and problems into the system.

[1542] "Means of analysis" refers to the function of analyzing the reported information and identifying the cause of the failure and how to deal with it.

[1543] "Means for suggesting self-solutions" refers to a function that suggests ways for users to solve malfunctions or problems on their own.

[1544] "Means to suggest contact points" refers to a function that provides contact information for the most appropriate support desk when you are unable to resolve the issue yourself.

[1545] "Usage period" refers to data on the period of time that a home appliance is actually used after it is purchased.

[1546] "Performance degradation data" refers to data that indicates a decline in performance, such as the usage time or malfunction history of a home appliance.

[1547] "Means for predicting replacement timing" refers to a function that predicts the optimal time to replace a product based on the length of use and performance degradation data.

[1548] "Means for suggesting recommended products" refers to the function of listing and providing products that meet the user's needs when replacement is required.

[1549] "Means to suggest the best retailer" refers to the function of providing information on stores that sell recommended products at the most reasonable prices.

[1550] "Emotional state" refers to the psychological state of the user as judged from facial expressions, tone of voice, input content, etc.

[1551] "Emotion engine" refers to technology that recognizes the user's emotional state and adjusts support priorities and response methods.

[1552] "Security devices" refers to devices used to secure a home or business, such as security cameras and door locks.

[1553] An embodiment of the present invention will be described.

[1554] This invention is a system that allows users to efficiently manage warranties or long-term warranty documents for home appliances and perform maintenance or troubleshooting in a timely manner. It also has the feature of recognizing the user's emotional state and providing prompt and appropriate support. The system of the present invention includes the following means.

[1555] 1. Upload and analyze the warranty card

[1556] First, users upload their appliance warranty or extended warranty documents using their smartphone or tablet. The uploaded documents are then analyzed using OCR technology (e.g., Tesseract OCR), and warranty-related information, such as the warranty period, serial number, and purchase store information, is extracted as text. The extracted information is managed for each user and securely stored in a cloud database. This allows users to instantly check warranty information whenever needed.

[1557] 2. Maintenance Notifications and Suggestions

[1558] The server monitors the usage status and environmental data of the home appliances to detect signs of deterioration or failure. This involves analyzing the log data and usage frequency emitted by the appliances. For example, it monitors the operating time of a refrigerator compressor and the number of times a washing machine is used. The server analyzes the collected data and predicts the appropriate time for maintenance. When the predicted maintenance time approaches, it notifies the user using a notification service such as Firebase and suggests specific work methods.

[1559] 3. Support for inquiries in the event of a breakdown

[1560] When a malfunction occurs, the user uses the malfunction reporting function within the app and enters details of the problem. The server receives this information, analyzes it, and proposes a self-solution as quickly as possible. If self-solution is difficult, the server suggests the most appropriate point of contact (for example, contact information for the manufacturer's support center or the store where the device was purchased). Furthermore, an emotion engine (for example, Google Cloud AutoML) that recognizes the user's emotional state is used to determine the user's level of stress or anxiety. Depending on the user's emotional state, the server quickly raises the priority of support and takes appropriate action.

[1561] 4. Replacement timing prediction and recommendations

[1562] The server predicts the optimal time to replace the appliance based on its usage period and performance degradation data. It analyzes statistical data and the usage of the appliance to detect signs of deterioration. Based on this information, when the optimal time to replace the appliance approaches, it provides the user with a list of recommended products and information on the best retailers. The list includes product performance evaluations and price comparison data, allowing the user to make the most economical choice.

[1563] 5. Security Devices and Emotion Engines

[1564] The present invention is also effective in managing warranty certificates for security devices. When a user reports a malfunction of a security device, the emotion engine analyzes the user's facial expression and tone of voice to determine their emotional state. If the engine determines that the user's stress level is high, it increases the priority of support and provides a prompt response.

[1565] As a concrete example, consider a situation where a washing machine breaks down. The user opens the app and fills in the breakdown report form with details. The server analyzes the user's facial expressions and tone of voice, and uses an emotion engine to determine that the user's stress level is high. In this case, the server sets the support priority high and offers a kind self-help solution or the fastest support option.

[1566] An example of a prompt sentence is, "Analyze the user's current emotional state based on facial expressions and voice data, and if the stress level is high, set a high priority for support, and provide the user with troubleshooting procedures and the fastest support options in user-friendly language."

[1567] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1568] Step 1:

[1569] Users scan the warranty or extended warranty document of their home appliance with a smartphone or tablet and upload it to the system. The input data is an image file of the warranty or extended warranty document. The output is an image file that can be analyzed.

[1570] Step 2:

[1571] The server receives the uploaded image and analyzes the text information in the image using OCR (Optical Character Recognition) technology. The input data is the image file. The output is text data such as the extracted warranty period, serial number, and purchase store information. Specifically, the server uses Tesseract OCR to extract the text.

[1572] Step 3:

[1573] The server structures the extracted text data and stores it in a database. The input data is the text information extracted by OCR. The output is the structured information registered in the database. Specifically, the text information is inserted into a MySQL database.

[1574] Step 4:

[1575] The server periodically monitors the usage status and environmental data of the home appliances. The input data is usage data and sensor data obtained from the home appliances. The output is information on predicted maintenance times. Specifically, the server communicates with the home appliances, collects log data, and analyzes it.

[1576] Step 5:

[1577] Based on the analyzed data, the server predicts the appropriate maintenance time and notifies the user. The input data is the analysis results of usage status and environmental data. The output is a maintenance notification sent to the user. Specifically, the notification is sent to the user's smartphone using a notification service such as Firebase.

[1578] Step 6:

[1579] When a malfunction occurs, the user enters details into a malfunction report form within the app. The input data is text information about the malfunction. The output is the malfunction data that is sent to the server. Specifically, the user opens the app and submits the malfunction report.

[1580] Step 7:

[1581] The server receives the failure data and uses an emotion engine to analyze the user's emotional state. The input data is the failure report and the user's emotional data (facial expressions and tone of voice). The output is the result of the user's emotional state assessment. Specifically, emotion analysis is performed using Google Cloud AutoML.

[1582] Step 8:

[1583] The server sets support priorities according to the user's emotional state and responds promptly and appropriately. The input data is the user's emotional state assessment result and fault data. The output is the support priority setting and specific countermeasures. Specific operations include suggesting a self-solution to resolve the fault, and if that does not resolve the issue, directing the user to the most appropriate support desk.

[1584] Step 9:

[1585] The server predicts the optimal time to replace appliances based on their usage period and performance degradation data. The input data is usage period data and performance degradation data. The output is the replacement period and a list of recommended products. Specifically, it references statistical data and applies a prediction algorithm.

[1586] Step 10:

[1587] The server provides users with a list of recommended products and information on the best retailers. The input data is the predicted replacement timing and market data. The output is a list of recommended products and retailer information sent to the user. Specifically, it presents product performance evaluations and price comparison data.

[1588] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1589] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1590] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1591] [Fourth embodiment]

[1592] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1593] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1594] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1595] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1596] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1597] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1598] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1599] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1600] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1601] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1602] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1603] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1604] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1605] As an embodiment of the present invention, a system for managing warranties and long-term warranty documents for home appliances and improving user convenience will be described. This system allows users to upload their warranties and long-term warranty documents for home appliances, and through a series of processes that automatically analyze and manage this information, it supports maintenance and responses in the event of a breakdown and suggests the optimal time to replace the appliance.

[1606] Management of warranties and long-term compensation documents

[1607] First, users upload their appliance warranty or extended warranty documents to the app using their smartphone or tablet. The device receives the uploaded image and analyzes it using image recognition technology (such as OCR). Specifically, it uses an image processing library to extract text information (such as warranty period, serial number, and store information) and generates it as structured data. The extracted data is registered in a database managed by the server. This allows users to check the warranty information at any time through the app.

[1608] Maintenance Notifications and Suggestions

[1609] The device periodically (e.g., daily or weekly) collects usage data from the appliance. In the case of smart appliances, the device communicates with the appliance and automatically acquires the data. The server analyzes this data and predicts when maintenance will be required based on the maintenance cycle of each appliance. When the predicted maintenance period approaches, the server generates a notification and sends it to the device. The notification provides the user with specific instructions and tools for the appropriate maintenance work (e.g., cleaning the filter).

[1610] Support for inquiries in case of breakdowns

[1611] When a home appliance breaks down, the user opens the support function within the app and enters the details of the malfunction into the "Report a Malfunction" form. The device then presents questions to the user and collects details of the malfunction, such as "Is an error message displayed?" The server then analyzes the collected malfunction data and suggests possible self-solutions to the user (e.g., "The drain may be clogged with dust"). If the self-solutions are ineffective, the server also provides contact information for the most appropriate point of contact (the manufacturer's support center or the support desk of the store where the appliance was purchased). This allows the user to quickly and efficiently deal with the malfunction.

[1612] Replacement timing prediction and recommendations

[1613] The server monitors the length of time an appliance has been in use based on its start date, operational data, and failure history. It analyzes usage and error logs to detect signs of performance degradation. Based on the analyzed data, the server applies an algorithm to predict when it is time to replace the appliance. For example, it references statistical data such as "Model X washing machines typically begin to deteriorate after about seven years." When the optimal time to replace the appliance approaches, the server generates a notification and sends it to the device. The notification provides the user with a list of recommended products and retailer information. The product list includes price comparison data and performance evaluations. This allows users to replace their appliances at the appropriate time and make economical choices.

[1614] As described above, the present invention provides a system that efficiently manages warranties and long-term warranty documents for home appliances, supports maintenance and troubleshooting, and suggests the optimal time to replace them. This system allows users to use home appliances with peace of mind, greatly improving the convenience of their lives.

[1615] The processing flow will be explained below.

[1616] Management of warranties and long-term compensation documents

[1617] Step 1:

[1618] Users take a photo of the home appliance warranty or long-term warranty document with their smartphone or tablet and upload it to the app.

[1619] Step 2:

[1620] The device receives the uploaded image and begins analyzing it using OCR (Optical Character Recognition) technology, recognizing text information using an image processing library.

[1621] Step 3:

[1622] The device analyzes the text information extracted from the image (warranty period, serial number, purchase store information, etc.) and converts it into structured data.

[1623] Step 4:

[1624] The server receives the structured data sent from the terminal and registers it in a database for each user.

[1625] Step 5:

[1626] Whenever necessary, the user opens the app and checks the registered warranty information.

[1627] Maintenance Notifications and Suggestions

[1628] Step 1:

[1629] The device periodically (e.g., daily or weekly) collects usage data from the home appliance. In the case of smart home appliances, the device automatically acquires data through communication with the appliance.

[1630] Step 2:

[1631] The device sends collected usage data to a server, including operating time, number of uses, error logs, etc.

[1632] Step 3:

[1633] The server analyzes the data it receives and predicts the next maintenance date based on the maintenance cycle for each appliance.

[1634] Step 4:

[1635] When the server approaches the predicted maintenance time, it generates a maintenance notice and delivers it to the terminal.

[1636] Step 5:

[1637] The device displays maintenance notifications to the user, providing specific work instructions (e.g., cleaning the filter) and the necessary tools.

[1638] Support for inquiries in case of breakdowns

[1639] Step 1:

[1640] When the user recognizes that their home appliance is malfunctioning, they open the support function within the app and enter details of the malfunction into the "Report a Malfunction" form.

[1641] Step 2:

[1642] The device presents the user with questions related to the malfunction and collects specific malfunction information, such as "Is an error message displayed?"

[1643] Step 3:

[1644] The terminal transmits the collected failure data to the server.

[1645] Step 4:

[1646] The server analyzes the failure data and suggests a self-solution to the user (e.g., "It may be due to dust clogging the drain").

[1647] Step 5:

[1648] If the server's self-help efforts are not effective, it will provide contact information for the appropriate point of contact (such as the manufacturer's support center or the support desk at the store where the product was purchased).

[1649] Replacement timing prediction and recommendations

[1650] Step 1:

[1651] The server monitors the period of use of the appliance based on the date of use, operation data, and failure history.

[1652] Step 2:

[1653] The server analyzes the monitoring data to detect signs of performance degradation and also refers to statistical data to predict performance degradation.

[1654] Step 3:

[1655] When the time for replacement approaches, the server generates a notification for the user and sends it to the terminal.

[1656] Step 4:

[1657] The terminal displays a replacement notice to the user and provides a list of recommended products and information on the cheapest retailer.

[1658] Step 5:

[1659] Based on the information presented, the user selects the most suitable home appliance and considers purchasing it.

[1660] The above is a detailed flow of the program processing of this system.

[1661] Example 1

[1662] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1663] Modern homes use many electrical appliances, and it is necessary to properly store their warranties and long-term warranty documents, and efficiently manage their usage, maintenance, and troubleshooting. However, doing this on your own is cumbersome, and missing appropriate maintenance intervals increases the risk of breakdowns, making it difficult to determine the optimal time to replace appliances. For this reason, a system is needed that increases user convenience and streamlines the lifecycle management of electrical appliances.

[1664] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1665] In this invention, the server includes: means for users to upload warranty or extended warranty documents for electrical appliances; means for analyzing the uploaded documents using optical character recognition technology and extracting warranty-related information; means for managing the extracted information for each user and providing it when needed; means for monitoring the usage status and living environment of electrical appliances and suggesting appropriate maintenance times and cleaning methods; means for receiving and analyzing malfunction reports from users in the event of a malfunction and suggesting self-resolved solutions and contact information using a machine learning model; and means for predicting the optimal replacement time based on the duration of use and performance degradation data of the electrical appliances and suggesting recommended products and optimal retailers. This makes it easier for users to manage their warranty and warranty documents, enabling them to perform maintenance and replacement at the appropriate time.

[1666] "Upload" refers to the operation of a user sending a digital image of an electrical product warranty or extended warranty document to the system through the application.

[1667] "Optical character recognition" is a technology for extracting text information from images, specifically OCR (Optical Character Recognition).

[1668] "Warranty information" refers to important data such as the warranty period, serial number, and purchase store information listed on the warranty card or compensation document for an electrical product.

[1669] "Structured data" refers to a data format that is organized so that extracted information can be stored in a table or database.

[1670] "Monitoring" refers to the act of continuously observing the usage of electrical appliances and living environments and collecting necessary data.

[1671] "Maintenance period" refers to the time when an electrical appliance should be inspected, cleaned, or repaired to keep it functioning properly.

[1672] "Fault reporting" refers to the act of a user notifying the system of detailed information about abnormal operation or malfunction of an electrical appliance.

[1673] A "machine learning model" refers to an algorithm that analyzes collected data, finds patterns, and predicts future trends.

[1674] "Performance degradation data" refers to information regarding malfunctions or performance degradation that occur during the use of an electrical product.

[1675] The "optimal replacement time" refers to the point at which the performance of an electrical appliance deteriorates and it becomes economically advantageous to replace it with a new product rather than continuing to repair and maintain it.

[1676] "Recommended products" refer to products that the system recommends as having the same or better performance as the user's current electrical products.

[1677] "Best seller" refers to the store or online shop where you can purchase the recommended products at the best conditions.

[1678] As an embodiment of the present invention, a system for managing warranty certificates and long-term warranty documents for electrical appliances and improving user convenience will be described. This system allows users to upload warranty certificates and long-term warranty documents for electrical appliances, and through a series of processes that automatically analyze and manage this information, it supports maintenance and responses in the event of a breakdown and suggests the optimal time to replace the appliance.

[1679] Management of warranties and long-term compensation documents

[1680] First, users upload the warranty card or extended warranty document for their electrical appliances to the app using a smartphone or tablet. The device receives the uploaded image and analyzes it using image recognition technology (such as OCR). Specifically, an image processing library is used to extract text information (such as warranty period, serial number, and store information) and generate it as structured data. The extracted data is then registered in a database managed by the server. This allows users to check the warranty information at any time through the app.

[1681] Maintenance Notifications and Suggestions

[1682] The device periodically (e.g., daily or weekly) collects appliance usage data. In the case of smart home appliances, the device communicates with the appliance to automatically obtain the data. The server analyzes this data and predicts when maintenance is required based on the maintenance cycle of each appliance. When the predicted maintenance period approaches, the server generates a notification and sends it to the device. The notification provides the user with specific instructions and tools for the appropriate maintenance work (e.g., cleaning the filter).

[1683] Support for inquiries in case of breakdowns

[1684] When an electrical appliance breaks down, the user opens the support function within the app and enters the details of the malfunction into the "Report a malfunction" form. The device then presents questions to the user and collects details of the malfunction, such as "Is an error message displayed?" The server then analyzes the collected malfunction data and suggests possible self-solutions to the user (e.g., "The drain may be clogged with dust"). If the self-solutions are ineffective, the server also provides contact information for the most appropriate point of contact (the manufacturer's support center or the support desk of the store where the appliance was purchased). This allows the user to quickly and efficiently resolve the malfunction.

[1685] Replacement timing prediction and recommendations

[1686] The server monitors the length of time an appliance has been in use based on its start date, operational data, and fault history. It analyzes usage and error logs to detect signs of performance degradation. Based on the analyzed data, the server applies an algorithm to predict when it's time to replace it. For example, it references statistical data such as "Model X refrigerators typically begin to deteriorate after about seven years." When the optimal time to replace an appliance approaches, the server generates a notification and sends it to the device. The notification provides the user with a list of recommended products and retailer information. The product list includes price comparison data and performance ratings. This allows users to replace appliances at the appropriate time and make economical choices.

[1687] Specific examples

[1688] A user uploads a refrigerator warranty, and the device extracts that the warranty period is "May 1, 2023 to May 1, 2025." The server predicts when the washing machine filter needs cleaning and notifies the user, "Please clean the filter." When a user reports error code "E05," the server suggests a self-solution, saying, "Check the drain hose." When seven years have passed since the refrigerator was first used, the server notifies the user, "It's time to replace it," and displays a list of recommended new refrigerators.

[1689] Prompt Sentence Examples

[1690] "Just take a picture of your appliance's warranty card with your camera and upload it to the app. The warranty period and serial number will be automatically extracted."

[1691] "It collects data every week and notifies you when it's time to maintain your appliances. It lets you know when it's time to clean your filters."

[1692] "If your appliance breaks down, please report the problem through the app and we will suggest a solution for you."

[1693] "It analyzes the usage period of home appliances and notifies you of the optimal time to replace them. It also provides a list of recommended products."

[1694] As described above, the present invention provides a system that efficiently manages warranties and long-term warranty documents for electrical appliances, supports maintenance and troubleshooting, and suggests the optimal time to replace them. This allows users to use electrical appliances with peace of mind, greatly improving the convenience of their lives.

[1695] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1696] Step 1:

[1697] User uploads warranty card

[1698] A user uploads an image of the warranty or extended warranty document for an electrical appliance using an app on their smartphone or tablet. Specifically, the user taps the camera button in the app to launch the camera and take a photo of the warranty or select an existing image.

[1699] Input: Warranty card image taken or selected by the user

[1700] Output: Uploaded warranty image

[1701] Step 2:

[1702] The device receives the image

[1703] The device receives the image uploaded by the user and prepares it for image recognition processing.

[1704] Input: Uploaded warranty image

[1705] Output: warranty card image ready for analysis

[1706] Step 3:

[1707] The device analyzes the image

[1708] The device performs preprocessing using an image processing library (e.g., OpenCV). This includes adjusting the resolution, removing noise, and identifying text regions. It then uses an OCR engine (e.g., Tesseract) to extract text from the image. This allows information such as the warranty period, serial number, and store information to be obtained.

[1709] Input: warranty card image ready for analysis

[1710] Data processing: image resolution adjustment, noise removal, text area identification

[1711] Data Calculation: Text Extraction Using an OCR Engine

[1712] Output: Extracted text data

[1713] Step 4:

[1714] The device structures the data

[1715] The text data extracted by the device is converted into structured data such as JSON format. Specifically, the warranty period, serial number, and purchase store information are stored in the appropriate fields.

[1716] Input: Extracted text data

[1717] Data processing: structuring text data into JSON format

[1718] Output: Structured data in JSON format

[1719] Step 5:

[1720] The device sends data to the server

[1721] The device sends the structured data to the server using HTTPS as the communication protocol.

[1722] Input: Structured data in JSON format

[1723] Output: Structured data sent to the server

[1724] Step 6:

[1725] The server stores the data

[1726] The server stores the received structured data in a database management system (e.g. MySQL). The server first checks the data for errors, and if there are no errors, stores it in the database in the appropriate format.

[1727] Input: Structured data sent to the server

[1728] Data processing: Checking data for errors and converting it to the appropriate format

[1729] Output: Data stored in the database

[1730] Step 7:

[1731] Device collects usage data

[1732] The device periodically collects usage data from the appliance. In the case of smart appliances, the device communicates with the appliance to automatically obtain data, including operating time and error logs.

[1733] Input: Usage data collected from electronic devices

[1734] Output: Usage data stored on the device

[1735] Step 8:

[1736] Server maintenance forecast

[1737] The server analyzes the usage data sent from the device, calculates the maintenance cycle using an analysis tool (e.g., Pandas), and calculates the predicted maintenance time.

[1738] Input: Usage data stored on the device

[1739] Data calculation: Usage data analysis, maintenance cycle calculation

[1740] Output: Estimated maintenance time

[1741] Step 9:

[1742] Server generates maintenance notice

[1743] The server generates a notification based on the predicted maintenance time, specifically a message containing the maintenance procedure and required tools.

[1744] Input: Estimated maintenance time

[1745] Data processing: Notification message generation

[1746] Output: The generated notification message

[1747] Step 10:

[1748] The server sends a notification

[1749] The server generates a notification message and sends it to the device, which receives it and displays it to the user as a push notification.

[1750] Input: The generated notification message

[1751] Output: Notification message sent to the terminal

[1752] Step 11:

[1753] User reports a malfunction

[1754] The user accesses the support function within the app and enters detailed information about the problem into the "Report a Problem" form.

[1755] Input: The details of the malfunction entered in the "Malfunction Report" form

[1756] Output: The entered fault report data

[1757] Step 12:

[1758] The device collects failure data

[1759] The device asks the user additional questions to gather more detailed information about the failure, such as "Is an error message displayed?"

[1760] Input: Fault report data and responses to additional questions

[1761] Output: Collected detailed failure data

[1762] Step 13:

[1763] The server proposes a self-solution

[1764] The device sends the collected failure data to a server, which then analyzes the data using a machine learning model and proposes a self-solution to the user.

[1765] Input: Collected detailed failure data

[1766] Data calculation: failure data analysis, self-resolved solution generation

[1767] Output: Self-solution proposal

[1768] Step 14:

[1769] The server provides contact information

[1770] If self-help measures are not effective, the server provides contact information for the best place to contact (the manufacturer's support center or the support desk at the store where you purchased the product).

[1771] Input: Failure data and self-solving results

[1772] Output: Best contact information

[1773] Step 15:

[1774] The server monitors the usage period

[1775] The server monitors the electrical appliances based on their start date of use, operation data, and failure history.

[1776] Input: Start date of use, operation data, failure history

[1777] Output: Monitoring result data

[1778] Step 16:

[1779] Server replacement time prediction

[1780] The server analyzes the monitoring results data and uses data analysis tools and algorithms to predict when it is time to replace the device.

[1781] Input: Monitoring result data

[1782] Data calculation: Performance degradation analysis, replacement timing prediction

[1783] Output: Predicted replacement time

[1784] Step 17:

[1785] The server generates a recommendation notification

[1786] When the time for replacement approaches, the server generates a notification message including a list of recommended products and information on the best retailer.

[1787] Input: Predicted replacement time

[1788] Data processing: Generation of recommended product list and retailer information

[1789] Output: Recommendation notification message

[1790] Step 18:

[1791] The server sends a notification

[1792] The server generates a recommendation notification and sends it to the device, which receives it and displays it to the user as a push notification.

[1793] Input: Generated recommendation notification message

[1794] Output: Recommendation notification message sent to the device

[1795] (Application example 1)

[1796] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1797] Users often have difficulty managing or lose their home appliance warranties and long-term warranty documents, making it difficult to quickly check the warranty details when needed. Furthermore, notifications regarding home appliance maintenance schedules and troubleshooting are not provided in a timely manner, reducing user convenience. Furthermore, it is difficult for users to properly determine when to replace their home appliances, which can lead to financial losses if they miss the optimal timing. These issues need to be resolved.

[1798] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1799] In this invention, the server includes: means for users to upload warranty or extended warranty documents for electrical appliances; means for analyzing the uploaded documents using image recognition technology and extracting warranty-related information; means for managing the extracted information for each user and providing it as needed; means for monitoring the usage status and installation environment of the electrical appliance and suggesting appropriate maintenance times and cleaning methods; means for receiving and analyzing malfunction reports from users and suggesting self-resolved solutions and contact information; means for predicting the optimal replacement time based on the electrical appliance's usage period and performance degradation data and suggesting recommended products and the lowest-priced retailers; means for extracting warranty information from warranty images using a smartphone and storing it in a database; means for providing maintenance notifications and support in the event of a malfunction; and means for predicting the replacement time based on the start date of use and usage data and providing a list of recommended products. This allows users to efficiently manage warranty and warranty documents, receive timely notifications for maintenance and malfunction response, and use electrical appliances economically and efficiently by knowing the optimal replacement time.

[1800] "Electrical appliances" is a general term for devices and equipment that require electrical power for use.

[1801] A "warranty" is a document stating that the manufacturer or seller guarantees the quality and performance of a product for a certain period of time.

[1802] An "extended warranty document" is a contract that guarantees the repair or replacement of a product for a period beyond the normal warranty period.

[1803] "User" means any person or entity that uses a system or device.

[1804] "Image recognition technology" is a general term for technology that analyzes and extracts specific information from images, and generally includes OCR (optical character recognition).

[1805] "Uploading" is the act of sending data from a local device to a network or server.

[1806] "Warranty information" is data about warranty details such as the warranty period, serial number, and purchase store information, which are written on the warranty card.

[1807] "Extraction" refers to the process of extracting necessary information from images or text.

[1808] "Management" is the act of controlling the organization, storage, access, and updating of information.

[1809] "Providing" refers to supplying information or services required by users.

[1810] "Monitoring" is the process of continuously observing and recording the state of a system or device.

[1811] "Maintenance period" refers to the period when a device or equipment requires regular maintenance, inspection, or repair.

[1812] "Cleaning method" refers to the selection of procedures and materials for cleaning equipment or devices.

[1813] "Fault reporting" is the act of a user notifying a system operator of a fault or problem in a device or system.

[1814] "Analysis" is the process of examining and evaluating data to understand its structure and meaning.

[1815] "Self-help" is a method by which a user can solve a problem on their own without professional support.

[1816] "Contact Information" refers to ...

Claims

1. a means for a user to upload a warranty or extended warranty document for the appliance; A means for analyzing the uploaded documents using image recognition technology to extract information related to the warranty; A means for managing the extracted information for each user and providing it when needed; A means to monitor the usage of home appliances and the living environment, and to suggest appropriate maintenance times and cleaning methods; A means for receiving and analyzing failure reports from users when a failure occurs, and proposing self-solutions and contact information; Based on the usage period and performance degradation data of home appliances, the system predicts the optimal time to replace them, and suggests recommended products and the cheapest retailers. A system including:

2. The system according to claim 1 , further comprising means for extracting, as structured data, the warranty period, serial number, and purchase store information written on the uploaded warranty card.

3. The system according to claim 1 , further comprising means for predicting a maintenance timing based on a maintenance cycle for each home appliance.

Images