System

Abstract

A system is provided.SOLUTION: The system includes a means for inputting equipment information by a user, a means for storing the input equipment information in a database, a means for analyzing the stored equipment information and providing advice for solving problems, and a means for periodically checking the equipment information and notifying a maintenance time limit and a recommended replacement time.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, the number of electronic devices used in homes has increased, making the setup, troubleshooting, and maintenance of each device complex. Problems related to the wiring and malfunctions of each device are particularly difficult for users, often requiring specialized support. However, traditional support methods require complex and time-consuming information searches, and in some cases, an appropriate solution cannot be found. Furthermore, it is difficult to grasp maintenance deadlines and recommended replacement times, making it difficult to update devices at the appropriate time. To solve these issues, a system that unifies device management, problem-solving support, and maintenance management is needed. [Means for solving the problem]

[0005] This invention provides a means for users to input device information and store that device information in a database. It then builds a means for analyzing the stored device information and providing users with advice for resolving problems. It also provides a means for periodically checking device information and notifying them of maintenance deadlines and recommended replacement times. This allows users to easily understand the status of each device and quickly receive the necessary support. Furthermore, providing solutions based on past case studies enables more accurate and specific advice. Combining these means improves the efficiency of device management and problem solving, reducing the burden on users.

[0006] "Device information" refers to information about various electronic devices used by the user, including the device name, manufacturer, model, serial number, purchase date, warranty period, etc.

[0007] A "database" is a collection of information that systematically stores data such as equipment information and past case studies, allowing for quick search and retrieval.

[0008] "Analysis" is the process of processing collected data within a system to extract and evaluate information for specific purposes.

[0009] "Advice for solving problems" is information that specifically provides appropriate solutions and procedures for problems or questions that users are facing.

[0010] The "maintenance expiration date" is the end date of the support or warranty period provided by the manufacturer or distributor of the equipment.

[0011] "Recommended replacement timing" refers to the timing at which it is appropriate to replace existing equipment with a newer model, based on performance degradation or technological innovation.

[0012] A "notification" is a message or alert that the system uses to communicate specific information or important matters to the user.

[0013] A "case study" is data that records actual examples of problems that occurred in the past and their solutions, and is used as a reference for solving future problems. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The following describes in detail the mode for carrying out the present invention: This system allows a user to manage information about electronic devices in the home and receive support for solving problems.

[0036] Building the user interface

[0037] 1. Access and Dashboard View

[0038] A user accesses the system using a web browser or a mobile app.

[0039] The device displays a list of the user's registered devices on a dashboard, along with visual status information for each device, such as which devices are currently in operation, which devices are nearing the end of their maintenance period, and which devices are nearing the recommended replacement date.

[0040] Input and analysis of equipment information

[0041] 1. Registering device information

[0042] Users register information about newly purchased or existing devices using an input form within the system, including details such as device name, manufacturer, model, serial number, purchase date, and warranty period.

[0043] 2. Saving to the database

[0044] The terminal transmits the input device information to the server.

[0045] The server analyzes the received information and stores it in a database in an appropriate format.

[0046] Device information analysis and problem solving

[0047] 1. Reporting and analyzing problems

[0048] Users report problems or questions about specific devices by filling out a form within the system.

[0049] The server scans the reports it receives and analyzes them according to the type of problem, using natural language processing (NLP) techniques to understand the details of the problem.

[0050] 2. Providing advice

[0051] The server searches for similar problems from past case studies in a database and extracts the best solutions.

[0052] The device will display solutions to the user, such as "If the network connection is unstable, please restart the router."

[0053] Bug debugging support

[0054] 1. Reporting a Bug

[0055] Users report in detail any issues they are experiencing with a particular feature within the system.

[0056] 2. Initial diagnosis

[0057] The server provides a basic diagnostic flow to the user, for example, by providing instructions such as "Make sure the power cord is properly connected."

[0058] 3. Perform detailed diagnostics

[0059] If the user is unable to resolve the problem through initial diagnosis, the terminal collects additional details from the user and sends them to the server.

[0060] The server then analyzes the details received and suggests more advanced diagnostic steps, such as "The internal filter may be clogged. Please clean the filter."

[0061] Expired maintenance / replacement support

[0062] 1. Check the maintenance period

[0063] The server periodically scans the user's device list and checks the maintenance deadline for each device.

[0064] 2. Generate and send notifications

[0065] The server detects devices whose maintenance period is approaching and generates a notification to the user. For example, it generates a notification saying, "The maintenance period for your washing machine will expire at the end of next month," and displays it on the device.

[0066] 3. Providing recommendations

[0067] The server searches for new device candidates suitable for the user based on market information.

[0068] The device displays a recommendation that reads, "Your washing machine's maintenance deadline is approaching, so consider purchasing a new washing machine (manufacturer name) (model name)."

[0069] Specific examples

[0070] Example 1: When a wiring problem occurs

[0071] 1. Report a problem

[0072] User: Reports "TV won't connect to the Internet."

[0073] 2. Analysis and Advice

[0074] The device will ask the user the question, "Are the lights on your router working properly?"

[0075] User: "The green light is not on."

[0076] The server will analyze the problem and provide specific advice such as "Turn the router off and on again and connect the LAN cable correctly."

[0077] Example 2: Notification of expiration of maintenance period and recommendation to purchase a new product

[0078] 1. Notification of Maintenance Expiration

[0079] The server periodically checks to detect when a particular router is nearing the end of its maintenance period.

[0080] 2. Providing notifications and recommendations

[0081] The server generates a notification that says, "Your router's maintenance will expire at the end of this month," and the device displays it to the user.

[0082] Based on market information, the server suggests new router candidates to the user, such as "(Manufacturer name) (Model name) is recommended."

[0083] The present invention can be implemented in the above manner, and can perform device management, problem solving, and maintenance management for users in a unified manner. This allows users to use their electronic devices with peace of mind and receive prompt and accurate support when a problem occurs.

[0084] The processing flow will be explained below.

[0085] Building the user interface

[0086] Step 1:

[0087] A user opens a web browser or mobile app and accesses the system.

[0088] Step 2:

[0089] The terminal sends the user's login information to the server for authentication.

[0090] Step 3:

[0091] The server verifies the user's credentials and starts a session to allow access to the dashboard.

[0092] Step 4:

[0093] The device displays a dashboard that provides a visual list of the devices the user has registered and their respective status information.

[0094] Input and analysis of equipment information

[0095] Step 1:

[0096] When a user purchases or installs new equipment, they enter the equipment information into an input form within the system.

[0097] Step 2:

[0098] The device sends the entered device information (device name, manufacturer, model, serial number, purchase date, warranty period, etc.) to the server.

[0099] Step 3:

[0100] The server analyzes the received device information and stores it in a database in an appropriate format.

[0101] Step 4:

[0102] The server checks the saved device information and notifies the user that it has been correctly stored in the database.

[0103] Device information analysis and problem solving

[0104] Step 1:

[0105] Users submit problems or questions about specific devices by filling out a report form within the system.

[0106] Step 2:

[0107] The terminal transmits the user's report to the server.

[0108] Step 3:

[0109] The server analyzes the report received using natural language processing (NLP) technology to identify the type of problem.

[0110] Step 4:

[0111] The server searches a database of past case studies for similar problems and their solutions.

[0112] Step 5:

[0113] The server will extract the optimal solution and provide advice to the user, for example, "If your Internet connection is unstable, please reboot your router."

[0114] Bug debugging support

[0115] Step 1:

[0116] The user utilizes the debug reporting feature within the system to enter details of the problem they are experiencing and report it.

[0117] Step 2:

[0118] The terminal transmits the user's report to the server.

[0119] Step 3:

[0120] The server generates a basic diagnostic flow and presents it to the user, for example, showing a procedure such as "Check if the power cord is properly connected."

[0121] Step 4:

[0122] If the user performs an initial diagnosis and the problem persists, they enter additional details into a form within the system.

[0123] Step 5:

[0124] The terminal sends the detailed information back to the server.

[0125] Step 6:

[0126] The server then performs deep analysis of the details received and generates more advanced diagnostic procedures, such as displaying advice such as "The internal filter may be clogged. Please clean the filter."

[0127] Expired maintenance / replacement support

[0128] Step 1:

[0129] The server periodically scans the user's device list and checks the maintenance expiration date for each device.

[0130] Step 2:

[0131] The server detects devices that are close to expiring for maintenance and generates a notification message.

[0132] Step 3:

[0133] The device displays a notification message to the user, for example, "Your washing machine's maintenance period expires at the end of next month."

[0134] Step 4:

[0135] The server searches for new device candidates suitable for the user based on market information.

[0136] Step 5:

[0137] The device displays a recommendation to the user saying, "Your washing machine's maintenance deadline is approaching, so consider purchasing a new washing machine (manufacturer name) (model name)."

[0138] Example 1

[0139] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0140] Management of electronic devices in the home is generally done on a per-device basis, and when a problem occurs, the response varies from device to device, which can be time-consuming and difficult for users to understand. Furthermore, managing maintenance deadlines and replacement times manually can easily lead to errors, and there is a risk that the device will not last the full lifespan of the device. A system is needed to centrally manage these issues, reduce the burden on users, and enable rapid problem resolution.

[0141] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0142] In this invention, the server includes means for a user to input device information, means for saving the input device information in a database, means for analyzing the saved device information and providing advice for solving problems, means for periodically checking device information and notifying the user of maintenance deadlines and recommended replacement times, means for a user to report problems, means for analyzing reported problems and providing appropriate solutions, means for a user to report malfunctions, means for providing initial and detailed diagnoses, and means for displaying detailed advice. This allows for centralized management of electronic devices in the home, enabling confirmation of maintenance deadlines, notifications of recommended replacements, and rapid problem resolution.

[0143] A "user" is a person who uses this system to manage information about devices in the home.

[0144] "Device information" is detailed information about electronic devices in the home, including, for example, the device name, manufacturer, model, serial number, purchase date, warranty period, and so on.

[0145] "Input means" refers to the interface through which users input device information and problem details into the system, and includes web browsers and mobile apps.

[0146] A "database" is a collection of data managed by a server, and stores registered device information and data for problem solving.

[0147] "Analysis means" is a function in which the server analyzes device information and reports from users, and performs processing to provide appropriate advice and solutions.

[0148] The "advice providing means" is a function that provides the user with the solution obtained by the analysis means, and displays specific operating methods and countermeasures.

[0149] "Maintenance expiration date" means the end of the equipment warranty or maintenance contract period.

[0150] The "notification means" is a function that notifies the user of devices whose maintenance deadlines are approaching or when replacement is recommended.

[0151] The "recommendation means" is a function that suggests suitable new device candidates to the user, for example, by displaying recommended devices based on the latest market information.

[0152] The "problem reporting means" is a function for a user to report a device problem they are experiencing to the system, and includes an input form and a feedback function.

[0153] The "initial diagnosis means" is a function for performing basic diagnosis of a problem reported by the problem reporting means.

[0154] The "detailed diagnosis means" is a function for collecting more detailed information and conducting in-depth analysis of problems that could not be resolved through initial diagnosis.

[0155] "Advice display means" is a function that displays specific advice or solutions to the user. For example, it may display on the terminal screen, "Please turn the router off and on again, and connect the LAN cable correctly."

[0156] A specific embodiment for carrying out this invention will be described in detail below. This system allows users to manage information about electronic devices in their homes and receive support for solving problems. The system is designed to allow users to register and manage device information, and report and diagnose problems using a web browser or mobile app.

[0157] Building the user interface

[0158] Access and Dashboard Viewing

[0159] A user accesses the system using a web browser or mobile app. The device sends the authentication information entered by the user to the server, which then authenticates the user based on that information. Once authentication is complete, the server obtains the user's device information and displays it on the device in dashboard format. For example, it visually provides information such as devices currently in operation, devices nearing the end of their maintenance period, and devices nearing the recommended replacement date.

[0160] Input and analysis of equipment information

[0161] Registering device information

[0162] Users register information about newly purchased or existing devices through an input form, including details such as device name, manufacturer, model, serial number, purchase date, warranty period, etc. The device sends this information to the server, which stores it in a database in the appropriate format.

[0163] Saving to a database

[0164] The server analyzes the device information received from the terminal and stores it in a database. For example, the server normalizes the device information and stores it in a database to enable efficient searches. This process uses a database management system (DBMS) or similar.

[0165] Device information analysis and problem solving

[0166] Problem reporting and analysis

[0167] Users report problems or questions about a specific device by filling out a form within the system. For example, if a user reports that "my TV cannot connect to the Internet," the server analyzes the report using natural language processing (NLP) techniques, including the Google® NLP API and other generative AI models.

[0168] Providing advice

[0169] The server searches for similar problems from past case studies in a database and extracts the optimal solution. For example, it displays a question on the device, such as "Are the lights on your router lit normally?", and provides specific advice based on the user's answer. The server generates advice such as "If your network connection is unstable, please turn the router off and on again and properly connect the LAN cable," and the advice is displayed on the device.

[0170] Bug debugging support

[0171] Reporting a Bug

[0172] A user reports a problem in detail using a specific function within the system. For example, if a user reports that their washing machine has stopped working, the details of the report are collected.

[0173] Initial diagnosis

[0174] The server analyzes the problem and provides a basic diagnostic flow, such as prompting the user to "Make sure the power cord is properly connected."

[0175] Performing detailed diagnostics

[0176] If the initial diagnosis doesn't solve the problem, the device collects additional details from the user and sends them to the server, which then performs a more detailed diagnosis and displays advanced advice, such as "The filter may be clogged, so please clean it."

[0177] Expired maintenance / replacement support

[0178] Check maintenance expiration date

[0179] The server periodically scans the user's device list and checks the maintenance expiration date for each device. For example, it detects information such as "The maintenance expiration date for the washing machine will expire in one month."

[0180] Generate and send notifications

[0181] The server detects approaching maintenance deadlines and generates notifications such as, "Your washing machine's maintenance deadline will expire at the end of next month." The generated notifications are displayed on the device.

[0182] Providing recommendations

[0183] The server searches for new appliance candidates suitable for the user based on market information. For example, it displays a recommendation message on the device saying, "A new washing machine from (manufacturer name) (model name) is recommended."

[0184] Specific examples

[0185] Below are some examples of prompt sentences.

[0186] Example prompt 1:

[0187] "Please explain the programming process for a home electronics information management system."

[0188] Example prompt 2:

[0189] "Could you please explain in detail what the process is when a user reports an issue?"

[0190] Example prompt 3:

[0191] "Please give specific examples of support for replacement of electronic devices in the home when their maintenance expires."

[0192] As described above, this system realizes the unification of management, problem solving, and maintenance of electronic devices used in the home, thereby greatly improving user convenience.

[0193] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0194] Step 1:

[0195] Access and Authentication

[0196] A user accesses the system using a web browser or mobile app. The device enters the user's authentication information (username and password) and sends it to the server. The server checks the authentication information against a database and sends the authentication result back to the device.

[0197] Input: Username, Password

[0198] Data processing: Authentication verification

[0199] Output: Authentication success / failure message

[0200] Step 2:

[0201] Viewing the Dashboard

[0202] If authentication is successful, the device displays the dashboard. The server retrieves the user's device information from the database and sends it to the device. The device displays the device information received in a list on the dashboard.

[0203] Input: User ID

[0204] Data processing: Acquisition of device information

[0205] Output: Dashboard screen

[0206] Step 3:

[0207] Registering device information

[0208] The user registers information about a newly purchased or existing device using an input form. The device enters the device information (device name, manufacturer, model, serial number, purchase date, warranty period, etc.) and sends it to the server. The server saves the information in a database.

[0209] Input: Device information (device name, manufacturer, model, serial number, purchase date, warranty period, etc.)

[0210] Data processing: Saving device information

[0211] Output: Registration complete message

[0212] Step 4:

[0213] Reporting an issue

[0214] Users fill out a form about a specific device issue and send it from their device to a server, which then analyzes the report using natural language processing (NLP) techniques.

[0215] Input: Problem report

[0216] Data processing: Problem analysis using NLP

[0217] Output: Problem analysis results

[0218] Step 5:

[0219] Providing advice

[0220] The server searches for similar problems from past case studies in a database and extracts appropriate solutions. The device displays the received solutions to the user. For example, for the problem "My TV can't connect to the Internet," the device displays the advice "Power cycle my router."

[0221] Input: Problem analysis results

[0222] Data processing: Finding case studies and extracting solutions

[0223] Output: Show solution

[0224] Step 6:

[0225] Reporting a Bug

[0226] A user reports a problem using a specific function within the system, and the report is sent from the terminal to the server.

[0227] Input: Bug report

[0228] Data processing: Saving report contents

[0229] Output: Report confirmation message

[0230] Step 7:

[0231] Providing an initial diagnosis

[0232] The server provides the user with a basic diagnostic flow. The terminal displays the diagnostic procedure received from the server to the user. For example, it displays an instruction such as "Please check if the power cord is connected properly."

[0233] Input: Bug report

[0234] Data processing: Basic diagnostic flow generation

[0235] Output: Display of diagnostic steps

[0236] Step 8:

[0237] Performing detailed diagnostics

[0238] If the user is unable to resolve the issue through initial diagnosis, the device will collect additional details from the user and send them to the server, which will then perform further analysis and provide advanced diagnostic procedures, such as displaying advice such as "Please clean the internal filter."

[0239] Input: More information

[0240] Data processing: Detailed diagnostic analysis

[0241] Output: Display of advanced diagnostic procedures

[0242] Step 9:

[0243] Check maintenance expiration date

[0244] The server periodically scans the database to check the maintenance deadlines of each device, detects devices that are close to the end of their maintenance period, and generates a notification to the user.

[0245] Input: Device information

[0246] Data processing: Check maintenance period

[0247] Output: Maintenance expiration notice

[0248] Step 10:

[0249] Providing recommendations

[0250] The server searches for new appliance candidates suitable for the user based on market information. The device displays a recommendation message to the user. For example, it displays a message saying, "We recommend a new washing machine from (manufacturer name) (model name)."

[0251] Input: Maintenance expiration information

[0252] Data processing: recommendation generation

[0253] Output: Recommendation notification

[0254] Through these steps, this system can centrally manage all electronic devices in the home, and provide quick problem resolution and appropriate maintenance support.

[0255] (Application example 1)

[0256] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0257] Conventional electronic device management systems for homes and brick-and-mortar stores have the problem of taking time and effort to monitor the status of devices and manage maintenance deadlines. Furthermore, it is difficult to quickly address problems when they occur, making rapid response particularly important in brick-and-mortar stores.

[0258] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0259] In this invention, the server includes: a means for a user to input device information; a means for saving the input device information in a database; a means for analyzing the saved device information and providing advice for resolving problems; a means for periodically checking device information and notifying users of maintenance deadlines and recommended replacement times; a means for providing notifications of maintenance deadlines and recommended replacement times using a visual display device; and a means for reporting device malfunctions and providing rapid problem resolution in real time via the visual display device. This makes it possible to monitor the status of electronic devices in real time and notify users of appropriate timing for maintenance or replacement. Furthermore, rapid problem resolution enables significant efficiency improvements in electronic device management, especially in physical stores.

[0260] The "means for users to input device information" is an interface that allows users to register information about newly purchased devices and existing devices in the system.

[0261] The "means for saving input device information in a database" is a system function that saves device information entered by a user in a database in an appropriate format.

[0262] The "means for analyzing stored device information" is a function for analyzing device information in the database and understanding the device status, maintenance deadlines, etc.

[0263] "Means for providing advice for solving problems" is a system function that presents appropriate solutions to users who report equipment malfunctions or problems.

[0264] "Means for periodically checking device information" refers to a function in which the server periodically scans the device information in the database to check things like maintenance deadlines and recommended replacement times.

[0265] The "means for notifying the user of the expiration date of maintenance or the recommended time for replacement" is a function for notifying the user of devices whose maintenance deadline is approaching or whose replacement is recommended.

[0266] "Means for providing notifications of maintenance deadlines and recommended replacement times using a visual display device" refers to a function that uses a visual display device (e.g., smart glasses, head-mounted display) to visually notify the user of maintenance deadlines and recommended replacement times.

[0267] "Means for reporting equipment malfunctions and quickly resolving problems in real time through a visual display device" is a function that analyzes equipment malfunctions reported by users in real time through a visual display device and provides quick solutions.

[0268] A "server" is a central computer that manages and analyzes data for the entire system.

[0269] The following describes in detail an embodiment of the present invention. This system allows users to manage electronic device information in a store and receive support for problem-solving. The users are primarily store clerks who use smart glasses or a head-mounted display to manage the devices.

[0270] Building the user interface

[0271] 1. Access and Dashboard View

[0272] Store associates access the system using smart glasses or a head-mounted display.

[0273] The device displays a list of the user's registered devices on a dashboard, along with visual status information for each device, such as which devices are currently in operation, which devices are nearing the end of their maintenance period, and which devices are nearing the recommended replacement date.

[0274] Input and analysis of equipment information

[0275] 1. Registering device information

[0276] The store clerk registers information about newly purchased or existing devices using an input form in the system, entering details such as the device name, manufacturer, model, serial number, purchase date, and warranty period.

[0277] 2. Saving to the database

[0278] The device sends the entered device information to the server, which analyzes the received information and stores it in a database in an appropriate format.

[0279] Device information analysis and problem solving

[0280] 1. Reporting and analyzing problems

[0281] A store clerk reports a problem or question about a specific device by filling out a form in the system. The server scans the report and analyzes it based on the type of problem. It then uses natural language processing (NLP) technology to understand the details of the problem.

[0282] 2. Providing advice

[0283] The server searches for similar problems from past case studies in a database and extracts the optimal solution. The device displays the solution to the user, providing specific advice such as "If your network connection is unstable, please restart your router."

[0284] Bug debugging support

[0285] 1. Reporting a Bug

[0286] The store associate uses a specific feature within the system to report in detail the problem they are experiencing.

[0287] 2. Initial diagnosis

[0288] The server provides a basic diagnostic flow to the user, for example, by providing instructions such as "Make sure the power cord is properly connected."

[0289] 3. Perform detailed diagnostics

[0290] If the store attendant is unable to resolve the problem through initial diagnosis, the terminal collects additional details from the store attendant and transmits them to the server.

[0291] The server then analyzes the details received and suggests more advanced diagnostic steps, such as "The internal filter may be clogged. Please clean the filter."

[0292] Expired maintenance / replacement support

[0293] 1. Check the maintenance period

[0294] The server periodically scans the user's device list and checks the maintenance deadline for each device.

[0295] 2. Generate and send notifications

[0296] The server detects devices whose maintenance period is approaching and generates a notification to the user. Specifically, it generates a notification saying, "The maintenance period for your washing machine will expire at the end of next month," and displays it on the device.

[0297] 3. Providing recommendations

[0298] The server searches for suitable new appliances for the user based on market information, and the device displays a recommendation such as, "Your current washing machine's maintenance deadline is approaching, so consider (manufacturer name) (model name) as your new washing machine."

[0299] Specific examples

[0300] Example 1: When a wiring problem occurs

[0301] 1. Report a problem

[0302] Store Clerk: Report "TV won't connect to the internet."

[0303] 2. Analysis and Advice

[0304] The device will ask the user the question, "Are the lights on your router working properly?"

[0305] Store Clerk: "The green light is not on."

[0306] The server will analyze the problem and provide specific advice such as "Turn the router off and on again and connect the LAN cable correctly."

[0307] Examples of prompt statements

[0308] Shop Electronics Problem Report: The register display is not working. Please identify the cause and provide a solution.

[0309] This system ensures thorough management of electronic devices in stores, ensuring adherence to maintenance deadlines and rapid troubleshooting. Real-time status monitoring and information provided through visual display devices dramatically improves user convenience.

[0310] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0311] Step 1:

[0312] The user enters the device information

[0313] Input: The sales associate enters details of the newly purchased or existing device (device name, manufacturer, model, serial number, purchase date, warranty period) into an input form.

[0314] How it works: This information is sent to the system via smart glasses or a head-mounted display.

[0315] Output: The entered data is sent to the system.

[0316] Step 2:

[0317] Save the entered device information in the database

[0318] Input: The device details obtained in step 1.

[0319] How it works: The server parses the information it receives and stores it in a database (such as SQLite) in an appropriate format.

[0320] Output: New device information is added to the database.

[0321] Step 3:

[0322] Dashboard View

[0323] Input: Equipment information stored in the database.

[0324] How it works: The device (smart glasses or head-mounted display) displays a list of device information on a dashboard, providing visual information on the status of each device (operating, approaching maintenance deadline, recommended replacement, etc.).

[0325] Output: Store staff can check the status of the equipment.

[0326] Step 4:

[0327] Problem reporting and analysis

[0328] Input: The store clerk fills out a problem report form to describe the equipment malfunction or problem.

[0329] How it works: The server analyzes the report using natural language processing (NLP) to understand the details of the problem. It uses a generative AI model to extract the type and details of the problem.

[0330] Output: Provides advice on next steps based on the extracted problem details.

[0331] Step 5:

[0332] Providing advice

[0333] Input: Detailed information about the problem analyzed in step 4.

[0334] How it works: The server searches a database of past case studies for similar problems and extracts the best solution. The generated advice is then displayed to the salesperson on the terminal.

[0335] Output: The store clerk receives specific advice, for example, "If your network connection is unstable, please power cycle your router."

[0336] Step 6:

[0337] Check maintenance expiration date

[0338] Input: Equipment information stored in the database.

[0339] How it works: The server periodically scans device information and checks the maintenance expiration date and usage period of each device.

[0340] Output: A list of devices due for maintenance soon is generated.

[0341] Step 7:

[0342] Notification of maintenance deadlines and recommended replacement times

[0343] Input: The list of equipment due for maintenance soon, generated in step 6.

[0344] How it works: The server generates notifications for devices that are due for maintenance soon, and the device (smart glasses or head-mounted display) visually displays them to the store clerk.

[0345] Output: The store clerk sees a notification that says, "Your washing machine's maintenance deadline will expire at the end of next month."

[0346] Step 8:

[0347] Providing recommendations

[0348] Input: New device candidates generated by the server based on market information.

[0349] How it works: The server generates a recommendation that reads, "Your current washing machine's maintenance deadline is approaching, so please consider purchasing a new washing machine (manufacturer name) (model name)," and displays it to the store clerk on the terminal.

[0350] Output: The store clerk reviews the new equipment candidates.

[0351] Step 9:

[0352] Detailed diagnosis of the problem

[0353] Input: Detailed information if the store attendant was unable to resolve the issue with initial diagnosis.

[0354] How it works: The device collects additional details from the store attendant and sends them to the server, which analyzes them and suggests more advanced diagnostic steps, such as "The internal filter may be clogged. Please clean the filter."

[0355] Output: The store associate receives detailed diagnostic results and instructions.

[0356] In this way, the entire system can efficiently support a series of processes from registering device information to problem solving, maintenance management, and replacement recommendations.

[0357] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0358] The following describes in detail the mode for carrying out the present invention. This system allows users to manage information about electronic devices in their homes and receive support for solving problems. It also aims to flexibly adjust advice and responses by combining it with an emotion engine that recognizes the user's emotions.

[0359] Building the user interface

[0360] 1. Access and Dashboard View

[0361] A user accesses the system using a web browser or a mobile app.

[0362] The terminal sends the user's login information to the server for authentication.

[0363] The server verifies the user's credentials and starts a session to allow access to the dashboard.

[0364] The device displays a dashboard that visually lists the devices registered by the user and their status information, such as which devices are currently in operation, which devices are nearing the end of their maintenance period, and which devices are nearing the recommended replacement date.

[0365] Input and analysis of equipment information

[0366] 1. Registering device information

[0367] Users register information about newly purchased or existing devices using an input form within the system, including details such as device name, manufacturer, model, serial number, purchase date, and warranty period.

[0368] The terminal transmits the input device information to the server.

[0369] The server analyzes the received device information and stores it in a database in an appropriate format.

[0370] The server checks the saved device information and notifies the user that it has been correctly stored in the database.

[0371] Device information analysis and problem solving

[0372] 1. Reporting and analyzing problems

[0373] Users submit problems or questions about specific devices by filling out a report form within the system.

[0374] The terminal transmits the user's report to the server.

[0375] The server analyzes the report received using natural language processing (NLP) technology to identify the type of problem.

[0376] The server searches a database of past case studies for similar problems and their solutions.

[0377] The server will extract the optimal solution and provide advice to the user, for example, "If your Internet connection is unstable, please reboot your router."

[0378] Bug debugging support

[0379] 1. Reporting a Bug

[0380] The user utilizes the debug reporting feature within the system to enter details of the problem they are experiencing and report it.

[0381] The terminal transmits the user's report to the server.

[0382] The server generates a basic diagnostic flow and presents it to the user, for example, showing a procedure such as "Check if the power cord is properly connected."

[0383] If the user performs an initial diagnosis and the problem persists, they enter additional details into a form within the system.

[0384] The terminal sends the detailed information back to the server.

[0385] The server then performs deep analysis of the details received and generates more advanced diagnostic procedures, such as displaying advice such as "The internal filter may be clogged. Please clean the filter."

[0386] Expired maintenance / replacement support

[0387] 1. Check the maintenance period

[0388] The server periodically scans the user's device list and checks the maintenance expiration date for each device.

[0389] The server detects devices that are close to expiring for maintenance and generates a notification message.

[0390] The device displays a notification message to the user, for example, "Your washing machine's maintenance period expires at the end of next month."

[0391] 2. Providing recommendations

[0392] The server searches for new device candidates suitable for the user based on market information.

[0393] The device displays a recommendation to the user saying, "Your washing machine's maintenance deadline is approaching, so consider purchasing a new washing machine (manufacturer name) (model name)."

[0394] Emotional engine regulation

[0395] 1. Emotional awareness

[0396] The device collects the user's input, facial expressions and voice during operation, and sends them to the server.

[0397] The server uses an emotion engine to analyze the received data and recognize the user's emotional state. For example, the server may recognize that the user is feeling stressed.

[0398] 2. Coordination of advice

[0399] The server adjusts the advice and responses it provides based on the emotions recognized by the emotion engine, for example providing simpler, gentler advice to a user who is feeling stressed.

[0400] Specific examples

[0401] Example 1: When a wiring problem occurs

[0402] 1. Report a problem

[0403] User: Reports "TV won't connect to the Internet."

[0404] Emotion engine: Recognizes irritation from the user's facial expression.

[0405] 2. Analysis and Advice

[0406] The device will ask the user in a soft tone, "Are the router lights up properly?"

[0407] User: "The green light is not on."

[0408] The server will analyze the problem and provide gentle advice such as, "Please turn the router off and on again and connect the LAN cable correctly."

[0409] Example 2: Notification of expiration of maintenance period and recommendation to purchase a new product

[0410] 1. Notification of Maintenance Expiration

[0411] The server periodically checks to detect when a particular router is nearing the end of its maintenance period.

[0412] Emotion engine: Recognizes that the user receiving the notification is tired.

[0413] 2. Providing notifications and recommendations

[0414] The server generates a notification that says, "Your router's maintenance will expire at the end of this month," and the device displays it to the user in a soft tone.

[0415] Based on market information, the server presents new router candidates, such as "(Manufacturer name) (Model name) is recommended," to tired users.

[0416] The present invention, embodied in the above-described manner, allows users to centrally manage their devices, solve problems, and manage maintenance. This allows users to use their electronic devices with peace of mind and receive prompt and accurate support when problems occur. Furthermore, the use of an emotion engine enables flexible responses tailored to the user's emotional state, improving the user experience.

[0417] The processing flow will be explained below.

[0418] Building the user interface

[0419] Step 1:

[0420] A user opens a web browser or mobile app and accesses the system.

[0421] Step 2:

[0422] The terminal sends the user's login information to the server for authentication.

[0423] Step 3:

[0424] The server verifies the user's credentials and starts a session to allow access to the dashboard.

[0425] Step 4:

[0426] The device displays a dashboard that visually provides information about the user's registered devices and their status.

[0427] Input and analysis of equipment information

[0428] Step 1:

[0429] When a user purchases or installs new equipment, they enter the equipment information into an input form within the system.

[0430] Step 2:

[0431] The device sends the entered device information (device name, manufacturer, model, serial number, purchase date, warranty period, etc.) to the server.

[0432] Step 3:

[0433] The server analyzes the received device information and stores it in a database in an appropriate format.

[0434] Step 4:

[0435] The server checks the saved device information and notifies the user that it has been correctly stored in the database.

[0436] Device information analysis and problem solving

[0437] Step 1:

[0438] Users submit problems or questions about specific devices by filling out a report form within the system.

[0439] Step 2:

[0440] The terminal transmits the user's report to the server.

[0441] Step 3:

[0442] The server analyzes the report received using natural language processing (NLP) technology to identify the type of problem.

[0443] Step 4:

[0444] The server searches a database of past case studies for similar problems and their solutions.

[0445] Step 5:

[0446] The server will extract the optimal solution and provide advice to the user, for example, "If your Internet connection is unstable, please reboot your router."

[0447] Bug debugging support

[0448] Step 1:

[0449] The user reports the problem they are experiencing by entering it using the debug reporting function within the system.

[0450] Step 2:

[0451] The terminal transmits the user's report to the server.

[0452] Step 3:

[0453] The server generates a basic diagnostic flow and presents it to the user, for example, showing a procedure such as "Check if the power cord is properly connected."

[0454] Step 4:

[0455] If the user performs an initial diagnosis and the problem persists, they enter additional details into a form within the system.

[0456] Step 5:

[0457] The terminal sends the detailed information back to the server.

[0458] Step 6:

[0459] The server then performs deep analysis of the details received and generates more advanced diagnostic procedures, such as displaying advice such as "The internal filter may be clogged. Please clean the filter."

[0460] Expired maintenance / replacement support

[0461] Step 1:

[0462] The server periodically scans the user's device list and checks the maintenance expiration date for each device.

[0463] Step 2:

[0464] The server detects devices that are close to expiring for maintenance and generates a notification message.

[0465] Step 3:

[0466] The device displays a notification message to the user, for example, "Your washing machine's maintenance period expires at the end of next month."

[0467] Step 4:

[0468] The server searches for new device candidates suitable for the user based on market information.

[0469] Step 5:

[0470] The device displays a recommendation to the user saying, "Your washing machine's maintenance deadline is approaching, so consider purchasing a new washing machine (manufacturer name) (model name)."

[0471] Emotional engine regulation

[0472] Step 1:

[0473] The device uses an emotion engine to collect the user's input, facial expressions, and voice during operation, and transmits these to the server.

[0474] Step 2:

[0475] The server uses an emotion engine to analyze the received data and recognize the user's emotional state, for example, recognizing that the user is feeling stressed.

[0476] Step 3:

[0477] The server adjusts the advice and response it provides based on the emotions recognized by the emotion engine.

[0478] Step 4:

[0479] The device will then display tailored advice and solutions to the user, such as "Please power cycle the router and properly connect the LAN cable" in a soft tone.

[0480] Specific examples

[0481] Example 1: When a wiring problem occurs

[0482] Step 1:

[0483] A user reports that their TV cannot connect to the Internet.

[0484] Step 2:

[0485] The emotion engine recognizes irritation from the user's facial expression.

[0486] Step 3:

[0487] The device will ask the user in a soft tone, "Are the router lights up properly?"

[0488] Step 4:

[0489] The user responds, "The green light is not on."

[0490] Step 5:

[0491] The server will analyze the problem and provide gentle advice such as, "Please turn the router off and on again and connect the LAN cable correctly."

[0492] Example 2: Notification of expiration of maintenance period and recommendation to purchase a new product

[0493] Step 1:

[0494] The server periodically checks to detect when a particular router is nearing the end of its maintenance period.

[0495] Step 2:

[0496] The emotion engine recognizes that the user receiving the notification is tired.

[0497] Step 3:

[0498] The server generates a notification that says, "Your router's maintenance will expire at the end of this month," and the device displays it to the user in a soft tone.

[0499] Step 4:

[0500] Based on market information, the server presents new router candidates, such as "(Manufacturer name) (Model name) is recommended," to tired users.

[0501] The present invention, embodied in the above-described manner, allows users to centrally manage their devices, solve problems, and manage maintenance. This allows users to use their electronic devices with peace of mind and receive prompt and accurate support when problems occur. Furthermore, the use of an emotion engine enables flexible responses tailored to the user's emotional state, improving the user experience.

[0502] Example 2

[0503] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0504] In conventional home electronic device management systems, it is difficult to manually input device information and determine when maintenance or replacement is required. Furthermore, when a problem occurs with an electronic device, it is often difficult for the user to resolve the problem themselves, requiring time and effort. Furthermore, when users experience a problem, they often experience emotional stress, which can remain as stress. The purpose of this invention is to solve these problems and enable users to use electronic devices with peace of mind.

[0505] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for a user to input device information, means for saving the input device information in a database, means for analyzing the saved device information and providing advice for solving problems, means for periodically checking the device information and notifying the user of maintenance deadlines and recommended replacement times, and emotion engine means for recognizing the user's emotional state and adjusting the advice content based on the recognized emotion. This enables the user to efficiently manage electronic devices in their home, receive prompt and accurate support when a problem occurs, and realize flexible responses tailored to the user's emotional state.

[0506] "Means for users to input device information" refers to a function that allows users to input information about new or existing devices through an interface within the system and send it to the server.

[0507] The "means for saving input device information in a database" is a function for receiving device information sent from a terminal and storing it in a database in an appropriate format.

[0508] The "means for analyzing stored device information and providing advice for resolving problems" is a function for analyzing device information stored in a database and providing appropriate advice and solutions for specific problems.

[0509] "Means to periodically check device information and notify users of maintenance deadlines and recommended replacement times" is a function that periodically scans the device information in the database, detects devices that are approaching their maintenance deadlines or devices that are recommended for replacement, and notifies the user.

[0510] "Emotion engine means for recognizing the user's emotional state and adjusting advice content based on the recognized emotion" is a function for recognizing the user's emotional state by analyzing the user's input content and facial expressions and voice during operation, and adjusting the advice and response provided based on the recognition results.

[0511] The present invention is a system for allowing users to manage information about electronic devices in their homes and receive assistance in solving problems, and further combines it with an emotion engine that recognizes the user's emotional state. Detailed embodiments of the present invention are described below.

[0512] System Configuration

[0513] This system consists of a user device (e.g., PC, smartphone, tablet), a central server, and a database. Users access the system using their device to enter device information and report problems. The server analyzes the received information and generates appropriate advice and notifications. The database stores device information and past case studies.

[0514] Building the user interface

[0515] A user accesses the system using a web browser or mobile app. The device sends the user's login information to the server for authentication. The server verifies the user's authentication information and initiates a session to access the dashboard. The device displays the dashboard, providing a visual list of the devices the user has registered and their respective status information.

[0516] For example, a user launches a smartphone app, logs in, and accesses the dashboard, which displays the operating status of their refrigerator or washing machine, maintenance deadlines, and recommended replacement times.

[0517] Input and analysis of equipment information

[0518] Users register information about newly purchased or existing devices using an input form within the system. Detailed information such as the device name, manufacturer, model, serial number, purchase date, and warranty period is entered. The terminal sends the entered device information to the server. The server analyzes the received device information and stores it in the database in the appropriate format. The server verifies the saved device information and notifies the user that it has been correctly stored in the database.

[0519] An example prompt is, "Please enter information about your newly purchased refrigerator. Include details such as the manufacturer, model, purchase date, serial number, and warranty expiration date."

[0520] Device information analysis and problem solving

[0521] The user enters a problem or question about a specific device into a report form within the system and submits it. The device then sends the user's report to the server. The server then analyzes the received report using natural language processing (NLP) technology to identify the type of problem. The server then searches for similar problems and their solutions from past case studies in a database. The server then extracts the optimal solution and provides advice to the user.

[0522] For example, a user may report that "the TV cannot connect to the Internet," and the server may advise, "Please turn the router off and on again and connect the LAN cable correctly."

[0523] Bug debugging support

[0524] The user uses the debug reporting function in the system to enter details of the problem they are experiencing and report it. The device sends the user's report to the server. The server generates a basic diagnostic flow and presents it to the user. For example, it might show a procedure such as "Please check if the power cord is connected properly." If the user performs an initial diagnosis and the problem is still not resolved, they enter additional details into a form in the system. The device then sends the details back to the server. The server performs a deeper analysis of the received details and generates more advanced diagnostic procedures.

[0525] An example of a prompt sentence is when you report that "the air conditioner is not cooling" and the server advises you to "clean the internal filter."

[0526] Expired maintenance / replacement support

[0527] The server periodically scans the user's device list and checks the maintenance deadline for each device. The server detects devices whose maintenance deadline is approaching and generates a notification message. The device displays the notification message to the user. The server searches for new device candidates suitable for the user based on market information, and the device displays them as recommendations.

[0528] As a specific example, when the maintenance deadline for a user's washing machine is approaching, the server notifies the user that "(Manufacturer name) (Model name) is a good candidate for replacement."

[0529] Emotional engine regulation

[0530] The device collects the user's input, facial expressions, and voice during operation and sends them to the server. The server then analyzes the received data using an emotion engine to recognize the user's emotional state. For example, it may recognize that the user is feeling stressed. The server then adjusts the advice and response it provides based on the emotions recognized by the emotion engine. For example, it may provide simpler, gentler advice to a user who is feeling stressed.

[0531] As an example of a prompt sentence, for a user who is stressed because "my TV can't connect to the Internet," a question is displayed in a soft tone: "Is the light on your router lit normally?"

[0532] The present invention is embodied in the above-described manner, which allows users to effectively manage their home electronic devices and receive prompt and accurate support. Furthermore, flexible responses based on emotional states improve the user experience.

[0533] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0534] Step 1:

[0535] System Access and Login

[0536] A user launches a web browser or mobile app and attempts to access the system.

[0537] The device receives the access request and displays the login screen.

[0538] Input: User ID, Password

[0539] Output: Login screen displayed

[0540] Specific operation: Enter the user ID and password into the input form.

[0541] Step 2:

[0542] Authentication process

[0543] The device sends the login information to the server.

[0544] The server checks the received login information against the registered information in the database and performs authentication.

[0545] If the authentication is successful, the server initiates a session, generates an authentication token, and returns it to the terminal.

[0546] Displays the dashboard to the user with a success message.

[0547] If the authentication fails, the server generates an error message and sends it to the terminal.

[0548] Input: Login information (user ID, password)

[0549] Output: Authentication token, success or error message

[0550] Specific behavior: The server performs authentication by referencing the database.

[0551] Step 3:

[0552] Entering device information

[0553] The user registers information about a new or existing device using the system's input form.

[0554] The information entered includes the device name, manufacturer, model, serial number, purchase date, and warranty period.

[0555] The terminal transmits the input information to the server.

[0556] Input: Device information (device name, manufacturer, model, serial number, purchase date, warranty period)

[0557] Output: Send device information to the server

[0558] What happens: A user fills in the form with details and clicks the submit button.

[0559] Step 4:

[0560] Saving and checking device information

[0561] The server analyzes the received device information and stores it in a database in an appropriate format.

[0562] Once the save is complete, the server generates a success message and sends it to the device.

[0563] The terminal notifies the user of the successful save.

[0564] Input: Device information, data to send to server

[0565] Output: Database updated, success message

[0566] What happens: The server saves the information in the corresponding field in the database.

[0567] Step 5:

[0568] Problem report

[0569] A user enters a problem or question about a specific device into a reporting form within the system and submits it.

[0570] The terminal sends the report to the server.

[0571] Input: Problem Report (user-entered text)

[0572] Output: Send problem report to server

[0573] Specific behavior: The user enters the problem description and clicks the report button.

[0574] Step 6:

[0575] Analyzing the problem

[0576] The server analyzes the report received using natural language processing (NLP) technology to identify the type of problem.

[0577] The server searches a database of past case studies for similar problems and their solutions.

[0578] Input: Problem report text

[0579] Output: Identified problem types, search results for similar problems

[0580] Specific operation: The server performs text analysis using NLP technology.

[0581] Step 7:

[0582] Providing advice

[0583] The server extracts the best solution and provides advice to the user.

[0584] The terminal displays the solution to the user.

[0585] Input: Search results, solution data

[0586] Output: Advisory message

[0587] Specific operation: The server generates an advisory message and sends it to the terminal.

[0588] Step 8:

[0589] Maintenance expiration check and notification

[0590] The server periodically scans the user's device list and detects devices that are close to expiring for maintenance.

[0591] The server generates a notification message for a device whose maintenance deadline is approaching and sends it to the terminal.

[0592] The terminal displays a notification message to the user.

[0593] Input: Equipment list data

[0594] Output: Information message

[0595] Specific operation: The server scans the device list data and checks the maintenance deadline.

[0596] Step 9:

[0597] Replacement recommendation

[0598] The server searches for new device candidates based on market information and generates appropriate recommendations.

[0599] The device displays the recommendations to the user.

[0600] Input: Market information, equipment information

[0601] Output: Recommendation message

[0602] What happens: The server searches the market database for new models.

[0603] Step 10:

[0604] Recognition of emotional states

[0605] The device collects the user's input, facial expressions and voice during operation, and sends them to the server.

[0606] The server analyzes the received data using an emotion engine to recognize the user's emotional state.

[0607] Input: facial expression data, voice data, input content

[0608] Output: Emotional state data

[0609] Specific operation: The device collects data using the camera and microphone and sends it to the server.

[0610] Step 11:

[0611] Adjusting advice based on emotions

[0612] The server tailors advice and responses based on the recognized emotion.

[0613] The terminal displays the adjusted advice to the user.

[0614] Input: Emotional state data, advice data

[0615] Output: Adjusted advisory message

[0616] Specific behavior: The server changes the tone and content of advice depending on the emotional state.

[0617] (Application example 2)

[0618] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0619] Modern brick-and-mortar stores use many electronic devices and equipment, and managing and maintaining them is a key issue for store operations. However, managing the maintenance deadlines for these devices and equipment and responding to breakdowns places a heavy burden on store staff. Furthermore, efficient problem solving can be difficult depending on the emotional state of staff. Furthermore, there is a need to visualize the usage and management status of in-store equipment to improve customer service.

[0620] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to input device information, a means for saving the input device information in a database, a means for analyzing the saved device information and providing advice for solving problems, a means for periodically checking the device information and notifying the user of maintenance deadlines and recommended replacement times, a means for analyzing the user's emotional state using an emotion recognition engine and adjusting the advice content, and a means for visualizing the management and usage status of various in-store facilities. This enables centralized management of electronic devices and facilities in the store and further enables flexible responses tailored to the emotional states of staff. Furthermore, visualization of in-store equipment usage is expected to improve operational efficiency and customer service.

[0621] Key word definitions

[0622] "Device information" refers to detailed information about electronic devices and equipment, such as the name, model, manufacturer, serial number, purchase date, and warranty period.

[0623] A "database" is an information repository designed to efficiently store, manage, and search input device information.

[0624] "Analysis" is the process of analyzing stored information to understand a particular problem or situation and derive a solution.

[0625] "Advice" refers to specific solutions or methods of dealing with problems that arise.

[0626] "Maintenance period" refers to the warranty and maintenance service period for electronic equipment and facilities.

[0627] The "recommended time for replacement" is the time when it is recommended to replace the current equipment with a new one, taking into consideration the lifespan and performance of the current equipment.

[0628] An "emotion recognition engine" is a software component that has the technology to analyze a user's emotional state from their facial expressions and voice.

[0629] The "means for adjusting the content of advice" is a function for changing the tone and content of the advice provided depending on the emotional state of the user analyzed by the emotion recognition engine.

[0630] "Various in-store equipment" refers to all electronic devices and equipment necessary for store operation, such as POS systems, security cameras, air conditioning, and lighting.

[0631] "Visualization" refers to the graphical display of the management and usage status of various equipment used within the store.

[0632] MODE FOR CARRYING OUT THE INVENTION

[0633] The following describes in detail the mode for carrying out the present invention. This system supports the management, maintenance, and troubleshooting of various electronic devices and equipment installed in brick-and-mortar stores, and also uses an emotion recognition engine to provide appropriate advice according to the emotional state of staff.

[0634] Hardware and Software Configuration

[0635] Hardware

[0636] Smartphone (iOS, ANDROID (registered trademark))

[0637] Tablet (iOS, Android)

[0638] Smart glasses (e.g., Google Glass (registered trademark), Vuzix Blade)

[0639] Sensors (face recognition camera, voice input microphone)

[0640] software

[0641] Natural Language Processing (NLP): Google Cloud Natural Language API, IBM Watson(R) NLP

[0642] Emotion recognition engine: Affectiva SDK, Microsoft® Azure® Face API

[0643] Database: Firebase, MySQL (registered trademark)

[0644] Web Server: Node.js, Express

[0645] Specific Embodiments

[0646] Registering and managing device information

[0647] Using a device (smartphone or tablet), store staff enter information about various electronic devices (device name, model, manufacturer, serial number, purchase date, warranty period, etc.) and store staff saves the information in a database (Firebase). This information is managed via a web server (Node.js, Express), and store staff can access it as needed from a web dashboard (browser interface).

[0648] Problem Reporting and Analysis

[0649] When a user (store staff member) reports a problem, the terminal sends the information to the server. The server uses a natural language processing (NLP) engine to analyze the report and search a database for past case studies. The server finds the optimal solution, sends it back to the terminal, and displays it to the staff member. For example, if a problem occurs with the POS system, the terminal will display the advice, "Please reconnect the power cord of the POS system."

[0650] Use of emotion recognition engine

[0651] The device's built-in camera and microphone collect the staff's facial expressions and voice, which are then analyzed by an emotion recognition engine (Affectiva SDK, Microsoft Azure Face API). Based on the analysis results, the server adjusts the tone and content of the advice. For example, if the staff member is irritated, the advice will be changed to a simpler, softer tone.

[0652] Maintenance and replacement support

[0653] The server periodically scans the database for device information and generates notifications for devices that are nearing the end of their maintenance period or have reached the recommended replacement date. The terminal displays these notifications to staff and also recommends new devices based on market information. For example, if an air conditioner's maintenance period is approaching, the system will display a message saying, "We recommend the XX model of XX manufacturer for your new air conditioner."

[0654] Specific examples

[0655] Example 1: Your POS system isn't working

[0656] Staff report issues:

[0657] A staff member reports that the POS system is not responding.

[0658] Emotion engine recognizes frustration:

[0659] The system recognizes irritation from staff members' facial expressions.

[0660] Softly presented advice:

[0661] "Don't worry. Let's check the power cord first," the message read in a gentle tone.

[0662] Example 2: Air conditioner maintenance deadline notification

[0663] Maintenance Expiration Notification:

[0664] The system detects that the air conditioner's maintenance deadline is approaching and notifies you, "Your air conditioner's maintenance deadline will expire at the end of this month."

[0665] Recommendations for tired staff:

[0666] Recognizing that the staff member is tired, the system displays in a soft tone, "We recommend (manufacturer name) (model name) for your new air conditioner."

[0667] Prompt Sentence Examples

[0668] 1. If your POS system isn't working:

[0669] A user says, "The POS system is not responding." What is the best advice and how can you provide a gentle solution to the user's frustration?

[0670] 2. Maintenance Expiration Notifications and Recommendations:

[0671] When notifying a user that their air conditioner's maintenance is due at the end of this month, gently recommend a new air conditioner model to a tired user.

[0672] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0673] Program processing steps

[0674] Step 1: Register device information

[0675] 1. User Input

[0676] A user (store staff member) uses a terminal (smartphone or tablet) to input information about various electronic devices, such as the device name, model, manufacturer, serial number, purchase date, and warranty period.

[0677] 2. Data Transmission

[0678] The device sends the entered device information to the server. The entered data is packaged in JSON format and sent using a secure communication protocol (HTTPS).

[0679] 3. Database storage

[0680] The server receives the input data and saves it in the database (Firebase). The server converts the received data into the appropriate format and saves it. A success message is sent to the device.

[0681] Step 2: Report and analyze the problem

[0682] 1. User Reports

[0683] A user (staff member) uses a terminal to report a problem with a specific device. The report content is entered in natural language.

[0684] 2. Data Transmission

[0685] The device sends the report to the server, also in JSON format using a secure communication protocol.

[0686] 3. Natural Language Processing

[0687] The server passes the received report to a natural language processing (NLP) engine (Google Cloud Natural Language API, IBM Watson NLP) to analyze the type of problem. The analyzed data is stored in an internal database.

[0688] 4. Search for a solution

[0689] The server searches an internal database for past case studies to find similar problems and their solutions.

[0690] 5. Generating Advice

[0691] The server generates an optimal solution and sends it to the terminal as advice to the user, for example, "Please reconnect the power cord of the POS system."

[0692] Step 3: Emotion recognition and advice adjustment

[0693] 1. Emotional data collection

[0694] The camera and microphone installed on the device are used to collect the facial expressions and voices of users (staff) in real time.

[0695] 2. Data Transmission

[0696] The collected emotion data is sent to the server in an appropriate format (e.g., as image data or audio data).

[0697] 3. Emotion analysis

[0698] The server analyzes the received data using an emotion recognition engine (Affectiva SDK, Microsoft Azure Face API) to identify the user's emotional state.

[0699] 4. Coordination of advice

[0700] The server adjusts the tone and content of the advice it provides based on the analyzed emotional state. For example, if the user is annoyed, it changes the advice to a simpler, more gentle tone.

[0701] 5. Submitting Advice

[0702] The adjusted advice is sent to the terminal and displayed in an appropriate form to the user.

[0703] Step 4: Maintenance and replacement support

[0704] 1. Regular scans

[0705] The server periodically scans the database to check the maintenance deadlines for each device.

[0706] 2. Notification generation

[0707] The server detects devices that are nearing the end of their maintenance period and generates a notification message that includes the expiration date and the recommended replacement time.

[0708] 3. Data Transmission

[0709] A notification message is sent to the terminal and displayed to the user.

[0710] 4. Recommendation Generation

[0711] The server searches for suitable new device candidates based on market information and generates a recommendation model.

[0712] 5. Sending Recommendations

[0713] The device displays new device candidates to the user. For example, a recommendation such as "We recommend the XX model of XX manufacturer for your new air conditioner" is displayed.

[0714] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0715] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0716] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0717] [Second embodiment]

[0718] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0719] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0720] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0721] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0722] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0723] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0724] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0725] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0726] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0727] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0728] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0729] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0730] The following describes in detail the mode for carrying out the present invention: This system allows a user to manage information about electronic devices in the home and receive support for solving problems.

[0731] Building the user interface

[0732] 1. Access and Dashboard View

[0733] A user accesses the system using a web browser or a mobile app.

[0734] The device displays a list of the user's registered devices on a dashboard, along with visual status information for each device, such as which devices are currently in operation, which devices are nearing the end of their maintenance period, and which devices are nearing the recommended replacement date.

[0735] Input and analysis of equipment information

[0736] 1. Registering device information

[0737] Users register information about newly purchased or existing devices using an input form within the system, including details such as device name, manufacturer, model, serial number, purchase date, and warranty period.

[0738] 2. Saving to the database

[0739] The terminal transmits the input device information to the server.

[0740] The server analyzes the received information and stores it in a database in an appropriate format.

[0741] Device information analysis and problem solving

[0742] 1. Reporting and analyzing problems

[0743] Users report problems or questions about specific devices by filling out a form within the system.

[0744] The server scans the reports it receives and analyzes them according to the type of problem, using natural language processing (NLP) techniques to understand the details of the problem.

[0745] 2. Providing advice

[0746] The server searches for similar problems from past case studies in a database and extracts the best solutions.

[0747] The device will display solutions to the user, such as "If the network connection is unstable, please restart the router."

[0748] Bug debugging support

[0749] 1. Reporting a Bug

[0750] Users report in detail any issues they are experiencing with a particular feature within the system.

[0751] 2. Initial diagnosis

[0752] The server provides a basic diagnostic flow to the user, for example, by providing instructions such as "Make sure the power cord is properly connected."

[0753] 3. Perform detailed diagnostics

[0754] If the user is unable to resolve the problem through initial diagnosis, the terminal collects additional details from the user and sends them to the server.

[0755] The server then analyzes the details received and suggests more advanced diagnostic steps, such as "The internal filter may be clogged. Please clean the filter."

[0756] Expired maintenance / replacement support

[0757] 1. Check the maintenance period

[0758] The server periodically scans the user's device list and checks the maintenance deadline for each device.

[0759] 2. Generate and send notifications

[0760] The server detects devices whose maintenance period is approaching and generates a notification to the user. For example, it generates a notification saying, "The maintenance period for your washing machine will expire at the end of next month," and displays it on the device.

[0761] 3. Providing recommendations

[0762] The server searches for new device candidates suitable for the user based on market information.

[0763] The device displays a recommendation that reads, "Your washing machine's maintenance deadline is approaching, so consider purchasing a new washing machine (manufacturer name) (model name)."

[0764] Specific examples

[0765] Example 1: When a wiring problem occurs

[0766] 1. Report a problem

[0767] User: Reports "TV won't connect to the Internet."

[0768] 2. Analysis and Advice

[0769] The device will ask the user the question, "Are the lights on your router working properly?"

[0770] User: "The green light is not on."

[0771] The server will analyze the problem and provide specific advice such as "Turn the router off and on again and connect the LAN cable correctly."

[0772] Example 2: Notification of expiration of maintenance period and recommendation to purchase a new product

[0773] 1. Notification of Maintenance Expiration

[0774] The server periodically checks to detect when a particular router is nearing the end of its maintenance period.

[0775] 2. Providing notifications and recommendations

[0776] The server generates a notification that says, "Your router's maintenance will expire at the end of this month," and the device displays it to the user.

[0777] Based on market information, the server suggests new router candidates to the user, such as "(Manufacturer name) (Model name) is recommended."

[0778] The present invention can be implemented in the above manner, and can perform device management, problem solving, and maintenance management for users in a unified manner. This allows users to use their electronic devices with peace of mind and receive prompt and accurate support when a problem occurs.

[0779] The processing flow will be explained below.

[0780] Building the user interface

[0781] Step 1:

[0782] A user opens a web browser or mobile app and accesses the system.

[0783] Step 2:

[0784] The terminal sends the user's login information to the server for authentication.

[0785] Step 3:

[0786] The server verifies the user's credentials and starts a session to allow access to the dashboard.

[0787] Step 4:

[0788] The device displays a dashboard that provides a visual list of the devices the user has registered and their respective status information.

[0789] Input and analysis of equipment information

[0790] Step 1:

[0791] When a user purchases or installs new equipment, they enter the equipment information into an input form within the system.

[0792] Step 2:

[0793] The device sends the entered device information (device name, manufacturer, model, serial number, purchase date, warranty period, etc.) to the server.

[0794] Step 3:

[0795] The server analyzes the received device information and stores it in a database in an appropriate format.

[0796] Step 4:

[0797] The server checks the saved device information and notifies the user that it has been correctly stored in the database.

[0798] Device information analysis and problem solving

[0799] Step 1:

[0800] Users submit problems or questions about specific devices by filling out a report form within the system.

[0801] Step 2:

[0802] The terminal transmits the user's report to the server.

[0803] Step 3:

[0804] The server analyzes the report received using natural language processing (NLP) technology to identify the type of problem.

[0805] Step 4:

[0806] The server searches a database of past case studies for similar problems and their solutions.

[0807] Step 5:

[0808] The server will extract the optimal solution and provide advice to the user, for example, "If your Internet connection is unstable, please reboot your router."

[0809] Bug debugging support

[0810] Step 1:

[0811] The user utilizes the debug reporting feature within the system to enter details of the problem they are experiencing and report it.

[0812] Step 2:

[0813] The terminal transmits the user's report to the server.

[0814] Step 3:

[0815] The server generates a basic diagnostic flow and presents it to the user, for example, showing a procedure such as "Check if the power cord is properly connected."

[0816] Step 4:

[0817] If the user performs an initial diagnosis and the problem persists, they enter additional details into a form within the system.

[0818] Step 5:

[0819] The terminal sends the detailed information back to the server.

[0820] Step 6:

[0821] The server then performs deep analysis of the details received and generates more advanced diagnostic procedures, such as displaying advice such as "The internal filter may be clogged. Please clean the filter."

[0822] Expired maintenance / replacement support

[0823] Step 1:

[0824] The server periodically scans the user's device list and checks the maintenance expiration date for each device.

[0825] Step 2:

[0826] The server detects devices that are close to expiring for maintenance and generates a notification message.

[0827] Step 3:

[0828] The device displays a notification message to the user, for example, "Your washing machine's maintenance period expires at the end of next month."

[0829] Step 4:

[0830] The server searches for new device candidates suitable for the user based on market information.

[0831] Step 5:

[0832] The device displays a recommendation to the user saying, "Your washing machine's maintenance deadline is approaching, so consider purchasing a new washing machine (manufacturer name) (model name)."

[0833] Example 1

[0834] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0835] Management of electronic devices in the home is generally done on a per-device basis, and when a problem occurs, the response varies from device to device, which can be time-consuming and difficult for users to understand. Furthermore, managing maintenance deadlines and replacement times manually can easily lead to errors, and there is a risk that the device will not last the full lifespan of the device. A system is needed to centrally manage these issues, reduce the burden on users, and enable rapid problem resolution.

[0836] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0837] In this invention, the server includes means for a user to input device information, means for saving the input device information in a database, means for analyzing the saved device information and providing advice for solving problems, means for periodically checking device information and notifying the user of maintenance deadlines and recommended replacement times, means for a user to report problems, means for analyzing reported problems and providing appropriate solutions, means for a user to report malfunctions, means for providing initial and detailed diagnoses, and means for displaying detailed advice. This allows for centralized management of electronic devices in the home, enabling confirmation of maintenance deadlines, notifications of recommended replacements, and rapid problem resolution.

[0838] A "user" is a person who uses this system to manage information about devices in the home.

[0839] "Device information" is detailed information about electronic devices in the home, including, for example, the device name, manufacturer, model, serial number, purchase date, warranty period, and so on.

[0840] "Input means" refers to the interface through which users input device information and problem details into the system, and includes web browsers and mobile apps.

[0841] A "database" is a collection of data managed by a server, and stores registered device information and data for problem solving.

[0842] "Analysis means" is a function in which the server analyzes device information and reports from users, and performs processing to provide appropriate advice and solutions.

[0843] The "advice providing means" is a function that provides the user with the solution obtained by the analysis means, and displays specific operating methods and countermeasures.

[0844] "Maintenance expiration date" means the end of the equipment warranty or maintenance contract period.

[0845] The "notification means" is a function that notifies the user of devices whose maintenance deadlines are approaching or when replacement is recommended.

[0846] The "recommendation means" is a function that suggests suitable new device candidates to the user, for example, by displaying recommended devices based on the latest market information.

[0847] The "problem reporting means" is a function for a user to report a device problem they are experiencing to the system, and includes an input form and a feedback function.

[0848] The "initial diagnosis means" is a function for performing basic diagnosis of a problem reported by the problem reporting means.

[0849] The "detailed diagnosis means" is a function for collecting more detailed information and conducting in-depth analysis of problems that could not be resolved through initial diagnosis.

[0850] "Advice display means" is a function that displays specific advice or solutions to the user. For example, it may display on the terminal screen, "Please turn the router off and on again, and connect the LAN cable correctly."

[0851] A specific embodiment for carrying out this invention will be described in detail below. This system allows users to manage information about electronic devices in their homes and receive support for solving problems. The system is designed to allow users to register and manage device information, and report and diagnose problems using a web browser or mobile app.

[0852] Building the user interface

[0853] Access and Dashboard Viewing

[0854] A user accesses the system using a web browser or mobile app. The device sends the authentication information entered by the user to the server, which then authenticates the user based on that information. Once authentication is complete, the server obtains the user's device information and displays it on the device in dashboard format. For example, it visually provides information such as devices currently in operation, devices nearing the end of their maintenance period, and devices nearing the recommended replacement date.

[0855] Input and analysis of equipment information

[0856] Registering device information

[0857] Users register information about newly purchased or existing devices through an input form, including details such as device name, manufacturer, model, serial number, purchase date, warranty period, etc. The device sends this information to the server, which stores it in a database in the appropriate format.

[0858] Saving to a database

[0859] The server analyzes the device information received from the terminal and stores it in a database. For example, the server normalizes the device information and stores it in a database to enable efficient searches. This process uses a database management system (DBMS) or similar.

[0860] Device information analysis and problem solving

[0861] Problem reporting and analysis

[0862] Users report problems or questions about a specific device by filling out a form within the system. For example, if a user reports that their TV cannot connect to the Internet, the server analyzes the report using natural language processing (NLP) techniques, such as the Google NLP API and other generative AI models.

[0863] Providing advice

[0864] The server searches for similar problems from past case studies in a database and extracts the optimal solution. For example, it displays a question on the device, such as "Are the lights on your router lit normally?", and provides specific advice based on the user's answer. The server generates advice such as "If your network connection is unstable, please turn the router off and on again and properly connect the LAN cable," and the advice is displayed on the device.

[0865] Bug debugging support

[0866] Reporting a Bug

[0867] A user reports a problem in detail using a specific function within the system. For example, if a user reports that their washing machine has stopped working, the details of the report are collected.

[0868] Initial diagnosis

[0869] The server analyzes the problem and provides a basic diagnostic flow, such as prompting the user to "Make sure the power cord is properly connected."

[0870] Performing detailed diagnostics

[0871] If the initial diagnosis doesn't solve the problem, the device collects additional details from the user and sends them to the server, which then performs a more detailed diagnosis and displays advanced advice, such as "The filter may be clogged, so please clean it."

[0872] Expired maintenance / replacement support

[0873] Check maintenance expiration date

[0874] The server periodically scans the user's device list and checks the maintenance expiration date for each device. For example, it detects information such as "The maintenance expiration date for the washing machine will expire in one month."

[0875] Generate and send notifications

[0876] The server detects approaching maintenance deadlines and generates notifications such as, "Your washing machine's maintenance deadline will expire at the end of next month." The generated notifications are displayed on the device.

[0877] Providing recommendations

[0878] The server searches for new appliance candidates suitable for the user based on market information. For example, it displays a recommendation message on the device saying, "A new washing machine from (manufacturer name) (model name) is recommended."

[0879] Specific examples

[0880] Below are some examples of prompt sentences.

[0881] Example prompt 1:

[0882] "Please explain the programming process for a home electronics information management system."

[0883] Example prompt 2:

[0884] "Could you please explain in detail what the process is when a user reports an issue?"

[0885] Example prompt 3:

[0886] "Please give specific examples of support for replacement of electronic devices in the home when their maintenance expires."

[0887] As described above, this system realizes the unification of management, problem solving, and maintenance of electronic devices used in the home, thereby greatly improving user convenience.

[0888] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0889] Step 1:

[0890] Access and Authentication

[0891] A user accesses the system using a web browser or mobile app. The device enters the user's authentication information (username and password) and sends it to the server. The server checks the authentication information against a database and sends the authentication result back to the device.

[0892] Input: Username, Password

[0893] Data processing: Authentication verification

[0894] Output: Authentication success / failure message

[0895] Step 2:

[0896] Viewing the Dashboard

[0897] If authentication is successful, the device displays the dashboard. The server retrieves the user's device information from the database and sends it to the device. The device displays the device information received in a list on the dashboard.

[0898] Input: User ID

[0899] Data processing: Acquisition of device information

[0900] Output: Dashboard screen

[0901] Step 3:

[0902] Registering device information

[0903] The user registers information about a newly purchased or existing device using an input form. The device enters the device information (device name, manufacturer, model, serial number, purchase date, warranty period, etc.) and sends it to the server. The server saves the information in a database.

[0904] Input: Device information (device name, manufacturer, model, serial number, purchase date, warranty period, etc.)

[0905] Data processing: Saving device information

[0906] Output: Registration complete message

[0907] Step 4:

[0908] Reporting an issue

[0909] Users fill out a form about a specific device issue and send it from their device to a server, which then analyzes the report using natural language processing (NLP) techniques.

[0910] Input: Problem report

[0911] Data processing: Problem analysis using NLP

[0912] Output: Problem analysis results

[0913] Step 5:

[0914] Providing advice

[0915] The server searches for similar problems from past case studies in a database and extracts appropriate solutions. The device displays the received solutions to the user. For example, for the problem "My TV can't connect to the Internet," the device displays the advice "Power cycle my router."

[0916] Input: Problem analysis results

[0917] Data processing: Finding case studies and extracting solutions

[0918] Output: Show solution

[0919] Step 6:

[0920] Reporting a Bug

[0921] A user reports a problem using a specific function within the system, and the report is sent from the terminal to the server.

[0922] Input: Bug report

[0923] Data processing: Saving report contents

[0924] Output: Report confirmation message

[0925] Step 7:

[0926] Providing an initial diagnosis

[0927] The server provides the user with a basic diagnostic flow. The terminal displays the diagnostic procedure received from the server to the user. For example, it displays an instruction such as "Please check if the power cord is connected properly."

[0928] Input: Bug report

[0929] Data processing: Basic diagnostic flow generation

[0930] Output: Display of diagnostic steps

[0931] Step 8:

[0932] Performing detailed diagnostics

[0933] If the user is unable to resolve the issue through initial diagnosis, the device will collect additional details from the user and send them to the server, which will then perform further analysis and provide advanced diagnostic procedures, such as displaying advice such as "Please clean the internal filter."

[0934] Input: More information

[0935] Data processing: Detailed diagnostic analysis

[0936] Output: Display of advanced diagnostic procedures

[0937] Step 9:

[0938] Check maintenance expiration date

[0939] The server periodically scans the database to check the maintenance deadlines of each device, detects devices that are close to the end of their maintenance period, and generates a notification to the user.

[0940] Input: Device information

[0941] Data processing: Check maintenance period

[0942] Output: Maintenance expiration notice

[0943] Step 10:

[0944] Providing recommendations

[0945] The server searches for new appliance candidates suitable for the user based on market information. The device displays a recommendation message to the user. For example, it displays a message saying, "We recommend a new washing machine from (manufacturer name) (model name)."

[0946] Input: Maintenance expiration information

[0947] Data processing: recommendation generation

[0948] Output: Recommendation notification

[0949] Through these steps, this system can centrally manage all electronic devices in the home, and provide quick problem resolution and appropriate maintenance support.

[0950] (Application example 1)

[0951] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0952] Conventional electronic device management systems for homes and brick-and-mortar stores have the problem of taking time and effort to monitor the status of devices and manage maintenance deadlines. Furthermore, it is difficult to quickly address problems when they occur, making rapid response particularly important in brick-and-mortar stores.

[0953] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0954] In this invention, the server includes: a means for a user to input device information; a means for saving the input device information in a database; a means for analyzing the saved device information and providing advice for resolving problems; a means for periodically checking device information and notifying users of maintenance deadlines and recommended replacement times; a means for providing notifications of maintenance deadlines and recommended replacement times using a visual display device; and a means for reporting device malfunctions and providing rapid problem resolution in real time via the visual display device. This makes it possible to monitor the status of electronic devices in real time and notify users of appropriate timing for maintenance or replacement. Furthermore, rapid problem resolution enables significant efficiency improvements in electronic device management, especially in physical stores.

[0955] The "means for users to input device information" is an interface that allows users to register information about newly purchased devices and existing devices in the system.

[0956] The "means for saving input device information in a database" is a system function that saves device information entered by a user in a database in an appropriate format.

[0957] The "means for analyzing stored device information" is a function for analyzing device information in the database and understanding the device status, maintenance deadlines, etc.

[0958] "Means for providing advice for solving problems" is a system function that presents appropriate solutions to users who report equipment malfunctions or problems.

[0959] "Means for periodically checking device information" refers to a function in which the server periodically scans the device information in the database to check things like maintenance deadlines and recommended replacement times.

[0960] The "means for notifying the user of the expiration date of maintenance or the recommended time for replacement" is a function for notifying the user of devices whose maintenance deadline is approaching or whose replacement is recommended.

[0961] "Means for providing notifications of maintenance deadlines and recommended replacement times using a visual display device" refers to a function that uses a visual display device (e.g., smart glasses, head-mounted display) to visually notify the user of maintenance deadlines and recommended replacement times.

[0962] "Means for reporting equipment malfunctions and quickly resolving problems in real time through a visual display device" is a function that analyzes equipment malfunctions reported by users in real time through a visual display device and provides quick solutions.

[0963] A "server" is a central computer that manages and analyzes data for the entire system.

[0964] The following describes in detail an embodiment of the present invention. This system allows users to manage electronic device information in a store and receive support for problem-solving. The users are primarily store clerks who use smart glasses or a head-mounted display to manage the devices.

[0965] Building the user interface

[0966] 1. Access and Dashboard View

[0967] Store associates access the system using smart glasses or a head-mounted display.

[0968] The device displays a list of the user's registered devices on a dashboard, along with visual status information for each device, such as which devices are currently in operation, which devices are nearing the end of their maintenance period, and which devices are nearing the recommended replacement date.

[0969] Input and analysis of equipment information

[0970] 1. Registering device information

[0971] The store clerk registers information about newly purchased or existing devices using an input form in the system, entering details such as the device name, manufacturer, model, serial number, purchase date, and warranty period.

[0972] 2. Saving to the database

[0973] The device sends the entered device information to the server, which analyzes the received information and stores it in a database in an appropriate format.

[0974] Device information analysis and problem solving

[0975] 1. Reporting and analyzing problems

[0976] A store clerk reports a problem or question about a specific device by filling out a form in the system. The server scans the report and analyzes it based on the type of problem. It then uses natural language processing (NLP) technology to understand the details of the problem.

[0977] 2. Providing advice

[0978] The server searches for similar problems from past case studies in a database and extracts the optimal solution. The device displays the solution to the user, providing specific advice such as "If your network connection is unstable, please restart your router."

[0979] Bug debugging support

[0980] 1. Reporting a Bug

[0981] The store associate uses a specific feature within the system to report in detail the problem they are experiencing.

[0982] 2. Initial diagnosis

[0983] The server provides a basic diagnostic flow to the user, for example, by providing instructions such as "Make sure the power cord is properly connected."

[0984] 3. Perform detailed diagnostics

[0985] If the store attendant is unable to resolve the problem through initial diagnosis, the terminal collects additional details from the store attendant and transmits them to the server.

[0986] The server then analyzes the details received and suggests more advanced diagnostic steps, such as "The internal filter may be clogged. Please clean the filter."

[0987] Expired maintenance / replacement support

[0988] 1. Check the maintenance period

[0989] The server periodically scans the user's device list and checks the maintenance deadline for each device.

[0990] 2. Generate and send notifications

[0991] The server detects devices whose maintenance period is approaching and generates a notification to the user. Specifically, it generates a notification saying, "The maintenance period for your washing machine will expire at the end of next month," and displays it on the device.

[0992] 3. Providing recommendations

[0993] The server searches for suitable new appliances for the user based on market information, and the device displays a recommendation such as, "Your current washing machine's maintenance deadline is approaching, so consider (manufacturer name) (model name) as your new washing machine."

[0994] Specific examples

[0995] Example 1: When a wiring problem occurs

[0996] 1. Report a problem

[0997] Store Clerk: Report "TV won't connect to the internet."

[0998] 2. Analysis and Advice

[0999] The device will ask the user the question, "Are the lights on your router working properly?"

[1000] Store Clerk: "The green light is not on."

[1001] The server will analyze the problem and provide specific advice such as "Turn the router off and on again and connect the LAN cable correctly."

[1002] Examples of prompt statements

[1003] Shop Electronics Problem Report: The register display is not working. Please identify the cause and provide a solution.

[1004] This system ensures thorough management of electronic devices in stores, ensuring adherence to maintenance deadlines and rapid troubleshooting. Real-time status monitoring and information provided through visual display devices dramatically improves user convenience.

[1005] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1006] Step 1:

[1007] The user enters the device information

[1008] Input: The sales associate enters details of the newly purchased or existing device (device name, manufacturer, model, serial number, purchase date, warranty period) into an input form.

[1009] How it works: This information is sent to the system via smart glasses or a head-mounted display.

[1010] Output: The entered data is sent to the system.

[1011] Step 2:

[1012] Save the entered device information in the database

[1013] Input: The device details obtained in step 1.

[1014] How it works: The server parses the information it receives and stores it in a database (such as SQLite) in an appropriate format.

[1015] Output: New device information is added to the database.

[1016] Step 3:

[1017] Dashboard View

[1018] Input: Equipment information stored in the database.

[1019] How it works: The device (smart glasses or head-mounted display) displays a list of device information on a dashboard, providing visual information on the status of each device (operating, approaching maintenance deadline, recommended replacement, etc.).

[1020] Output: Store staff can check the status of the equipment.

[1021] Step 4:

[1022] Problem reporting and analysis

[1023] Input: The store clerk fills out a problem report form to describe the equipment malfunction or problem.

[1024] How it works: The server analyzes the report using natural language processing (NLP) to understand the details of the problem. It uses a generative AI model to extract the type and details of the problem.

[1025] Output: Provides advice on next steps based on the extracted problem details.

[1026] Step 5:

[1027] Providing advice

[1028] Input: Detailed information about the problem analyzed in step 4.

[1029] How it works: The server searches a database of past case studies for similar problems and extracts the best solution. The generated advice is then displayed to the salesperson on the terminal.

[1030] Output: The store clerk receives specific advice, for example, "If your network connection is unstable, please power cycle your router."

[1031] Step 6:

[1032] Check maintenance expiration date

[1033] Input: Equipment information stored in the database.

[1034] How it works: The server periodically scans device information and checks the maintenance expiration date and usage period of each device.

[1035] Output: A list of devices due for maintenance soon is generated.

[1036] Step 7:

[1037] Notification of maintenance deadlines and recommended replacement times

[1038] Input: The list of equipment due for maintenance soon, generated in step 6.

[1039] How it works: The server generates notifications for devices that are due for maintenance soon, and the device (smart glasses or head-mounted display) visually displays them to the store clerk.

[1040] Output: The store clerk sees a notification that says, "Your washing machine's maintenance deadline will expire at the end of next month."

[1041] Step 8:

[1042] Providing recommendations

[1043] Input: New device candidates generated by the server based on market information.

[1044] How it works: The server generates a recommendation that reads, "Your current washing machine's maintenance deadline is approaching, so please consider purchasing a new washing machine (manufacturer name) (model name)," and displays it to the store clerk on the terminal.

[1045] Output: The store clerk reviews the new equipment candidates.

[1046] Step 9:

[1047] Detailed diagnosis of the problem

[1048] Input: Detailed information if the store attendant was unable to resolve the issue with initial diagnosis.

[1049] How it works: The device collects additional details from the store attendant and sends them to the server, which analyzes them and suggests more advanced diagnostic steps, such as "The internal filter may be clogged. Please clean the filter."

[1050] Output: The store associate receives detailed diagnostic results and instructions.

[1051] In this way, the entire system can efficiently support a series of processes from registering device information to problem solving, maintenance management, and replacement recommendations.

[1052] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1053] The following describes in detail the mode for carrying out the present invention. This system allows users to manage information about electronic devices in their homes and receive support for solving problems. It also aims to flexibly adjust advice and responses by combining it with an emotion engine that recognizes the user's emotions.

[1054] Building the user interface

[1055] 1. Access and Dashboard View

[1056] A user accesses the system using a web browser or a mobile app.

[1057] The terminal sends the user's login information to the server for authentication.

[1058] The server verifies the user's credentials and starts a session to allow access to the dashboard.

[1059] The device displays a dashboard that visually lists the devices registered by the user and their status information, such as which devices are currently in operation, which devices are nearing the end of their maintenance period, and which devices are nearing the recommended replacement date.

[1060] Input and analysis of equipment information

[1061] 1. Registering device information

[1062] Users register information about newly purchased or existing devices using an input form within the system, including details such as device name, manufacturer, model, serial number, purchase date, and warranty period.

[1063] The terminal transmits the input device information to the server.

[1064] The server analyzes the received device information and stores it in a database in an appropriate format.

[1065] The server checks the saved device information and notifies the user that it has been correctly stored in the database.

[1066] Device information analysis and problem solving

[1067] 1. Reporting and analyzing problems

[1068] Users submit problems or questions about specific devices by filling out a report form within the system.

[1069] The terminal transmits the user's report to the server.

[1070] The server analyzes the report received using natural language processing (NLP) technology to identify the type of problem.

[1071] The server searches a database of past case studies for similar problems and their solutions.

[1072] The server will extract the optimal solution and provide advice to the user, for example, "If your Internet connection is unstable, please reboot your router."

[1073] Bug debugging support

[1074] 1. Reporting a Bug

[1075] The user utilizes the debug reporting feature within the system to enter details of the problem they are experiencing and report it.

[1076] The terminal transmits the user's report to the server.

[1077] The server generates a basic diagnostic flow and presents it to the user, for example, showing a procedure such as "Check if the power cord is properly connected."

[1078] If the user performs an initial diagnosis and the problem persists, they enter additional details into a form within the system.

[1079] The terminal sends the detailed information back to the server.

[1080] The server then performs deep analysis of the details received and generates more advanced diagnostic procedures, such as displaying advice such as "The internal filter may be clogged. Please clean the filter."

[1081] Expired maintenance / replacement support

[1082] 1. Check the maintenance period

[1083] The server periodically scans the user's device list and checks the maintenance expiration date for each device.

[1084] The server detects devices that are close to expiring for maintenance and generates a notification message.

[1085] The device displays a notification message to the user, for example, "Your washing machine's maintenance period expires at the end of next month."

[1086] 2. Providing recommendations

[1087] The server searches for new device candidates suitable for the user based on market information.

[1088] The device displays a recommendation to the user saying, "Your washing machine's maintenance deadline is approaching, so consider purchasing a new washing machine (manufacturer name) (model name)."

[1089] Emotional engine regulation

[1090] 1. Emotional awareness

[1091] The device collects the user's input, facial expressions and voice during operation, and sends them to the server.

[1092] The server uses an emotion engine to analyze the received data and recognize the user's emotional state. For example, the server may recognize that the user is feeling stressed.

[1093] 2. Coordination of advice

[1094] The server adjusts the advice and responses it provides based on the emotions recognized by the emotion engine, for example providing simpler, gentler advice to a user who is feeling stressed.

[1095] Specific examples

[1096] Example 1: When a wiring problem occurs

[1097] 1. Report a problem

[1098] User: Reports "TV won't connect to the Internet."

[1099] Emotion engine: Recognizes irritation from the user's facial expression.

[1100] 2. Analysis and Advice

[1101] The device will ask the user in a soft tone, "Are the router lights up properly?"

[1102] User: "The green light is not on."

[1103] The server will analyze the problem and provide gentle advice such as, "Please turn the router off and on again and connect the LAN cable correctly."

[1104] Example 2: Notification of expiration of maintenance period and recommendation to purchase a new product

[1105] 1. Notification of Maintenance Expiration

[1106] The server periodically checks to detect when a particular router is nearing the end of its maintenance period.

[1107] Emotion engine: Recognizes that the user receiving the notification is tired.

[1108] 2. Providing notifications and recommendations

[1109] The server generates a notification that says, "Your router's maintenance will expire at the end of this month," and the device displays it to the user in a soft tone.

[1110] Based on market information, the server presents new router candidates, such as "(Manufacturer name) (Model name) is recommended," to tired users.

[1111] The present invention, embodied in the above-described manner, allows users to centrally manage their devices, solve problems, and manage maintenance. This allows users to use their electronic devices with peace of mind and receive prompt and accurate support when problems occur. Furthermore, the use of an emotion engine enables flexible responses tailored to the user's emotional state, improving the user experience.

[1112] The processing flow will be explained below.

[1113] Building the user interface

[1114] Step 1:

[1115] A user opens a web browser or mobile app and accesses the system.

[1116] Step 2:

[1117] The terminal sends the user's login information to the server for authentication.

[1118] Step 3:

[1119] The server verifies the user's credentials and starts a session to allow access to the dashboard.

[1120] Step 4:

[1121] The device displays a dashboard that visually provides information about the user's registered devices and their status.

[1122] Input and analysis of equipment information

[1123] Step 1:

[1124] When a user purchases or installs new equipment, they enter the equipment information into an input form within the system.

[1125] Step 2:

[1126] The device sends the entered device information (device name, manufacturer, model, serial number, purchase date, warranty period, etc.) to the server.

[1127] Step 3:

[1128] The server analyzes the received device information and stores it in a database in an appropriate format.

[1129] Step 4:

[1130] The server checks the saved device information and notifies the user that it has been correctly stored in the database.

[1131] Device information analysis and problem solving

[1132] Step 1:

[1133] Users submit problems or questions about specific devices by filling out a report form within the system.

[1134] Step 2:

[1135] The terminal transmits the user's report to the server.

[1136] Step 3:

[1137] The server analyzes the report received using natural language processing (NLP) technology to identify the type of problem.

[1138] Step 4:

[1139] The server searches a database of past case studies for similar problems and their solutions.

[1140] Step 5:

[1141] The server will extract the optimal solution and provide advice to the user, for example, "If your Internet connection is unstable, please reboot your router."

[1142] Bug debugging support

[1143] Step 1:

[1144] The user reports the problem they are experiencing by entering it using the debug reporting function within the system.

[1145] Step 2:

[1146] The terminal transmits the user's report to the server.

[1147] Step 3:

[1148] The server generates a basic diagnostic flow and presents it to the user, for example, showing a procedure such as "Check if the power cord is properly connected."

[1149] Step 4:

[1150] If the user performs an initial diagnosis and the problem persists, they enter additional details into a form within the system.

[1151] Step 5:

[1152] The terminal sends the detailed information back to the server.

[1153] Step 6:

[1154] The server then performs deep analysis of the details received and generates more advanced diagnostic procedures, such as displaying advice such as "The internal filter may be clogged. Please clean the filter."

[1155] Expired maintenance / replacement support

[1156] Step 1:

[1157] The server periodically scans the user's device list and checks the maintenance expiration date for each device.

[1158] Step 2:

[1159] The server detects devices that are close to expiring for maintenance and generates a notification message.

[1160] Step 3:

[1161] The device displays a notification message to the user, for example, "Your washing machine's maintenance period expires at the end of next month."

[1162] Step 4:

[1163] The server searches for new device candidates suitable for the user based on market information.

[1164] Step 5:

[1165] The device displays a recommendation to the user saying, "Your washing machine's maintenance deadline is approaching, so consider purchasing a new washing machine (manufacturer name) (model name)."

[1166] Emotional engine regulation

[1167] Step 1:

[1168] The device uses an emotion engine to collect the user's input, facial expressions, and voice during operation, and transmits these to the server.

[1169] Step 2:

[1170] The server uses an emotion engine to analyze the received data and recognize the user's emotional state, for example, recognizing that the user is feeling stressed.

[1171] Step 3:

[1172] The server adjusts the advice and response it provides based on the emotions recognized by the emotion engine.

[1173] Step 4:

[1174] The device will then display tailored advice and solutions to the user, such as "Please power cycle the router and properly connect the LAN cable" in a soft tone.

[1175] Specific examples

[1176] Example 1: When a wiring problem occurs

[1177] Step 1:

[1178] A user reports that their TV cannot connect to the Internet.

[1179] Step 2:

[1180] The emotion engine recognizes irritation from the user's facial expression.

[1181] Step 3:

[1182] The device will ask the user in a soft tone, "Are the router lights up properly?"

[1183] Step 4:

[1184] The user responds, "The green light is not on."

[1185] Step 5:

[1186] The server will analyze the problem and provide gentle advice such as, "Please turn the router off and on again and connect the LAN cable correctly."

[1187] Example 2: Notification of expiration of maintenance period and recommendation to purchase a new product

[1188] Step 1:

[1189] The server periodically checks to detect when a particular router is nearing the end of its maintenance period.

[1190] Step 2:

[1191] The emotion engine recognizes that the user receiving the notification is tired.

[1192] Step 3:

[1193] The server generates a notification that says, "Your router's maintenance will expire at the end of this month," and the device displays it to the user in a soft tone.

[1194] Step 4:

[1195] Based on market information, the server presents new router candidates, such as "(Manufacturer name) (Model name) is recommended," to tired users.

[1196] The present invention, embodied in the above-described manner, allows users to centrally manage their devices, solve problems, and manage maintenance. This allows users to use their electronic devices with peace of mind and receive prompt and accurate support when problems occur. Furthermore, the use of an emotion engine enables flexible responses tailored to the user's emotional state, improving the user experience.

[1197] Example 2

[1198] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1199] In conventional home electronic device management systems, it is difficult to manually input device information and determine when maintenance or replacement is required. Furthermore, when a problem occurs with an electronic device, it is often difficult for the user to resolve the problem themselves, requiring time and effort. Furthermore, when users experience a problem, they often experience emotional stress, which can remain as stress. The purpose of this invention is to solve these problems and enable users to use electronic devices with peace of mind.

[1200] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for a user to input device information, means for saving the input device information in a database, means for analyzing the saved device information and providing advice for solving problems, means for periodically checking the device information and notifying the user of maintenance deadlines and recommended replacement times, and emotion engine means for recognizing the user's emotional state and adjusting the advice content based on the recognized emotion. This enables the user to efficiently manage electronic devices in their home, receive prompt and accurate support when a problem occurs, and realize flexible responses tailored to the user's emotional state.

[1201] "Means for users to input device information" refers to a function that allows users to input information about new or existing devices through an interface within the system and send it to the server.

[1202] The "means for saving input device information in a database" is a function for receiving device information sent from a terminal and storing it in a database in an appropriate format.

[1203] The "means for analyzing stored device information and providing advice for resolving problems" is a function for analyzing device information stored in a database and providing appropriate advice and solutions for specific problems.

[1204] "Means to periodically check device information and notify users of maintenance deadlines and recommended replacement times" is a function that periodically scans the device information in the database, detects devices that are approaching their maintenance deadlines or devices that are recommended for replacement, and notifies the user.

[1205] "Emotion engine means for recognizing the user's emotional state and adjusting advice content based on the recognized emotion" is a function for recognizing the user's emotional state by analyzing the user's input content and facial expressions and voice during operation, and adjusting the advice and response provided based on the recognition results.

[1206] The present invention is a system for allowing users to manage information about electronic devices in their homes and receive assistance in solving problems, and further combines it with an emotion engine that recognizes the user's emotional state. Detailed embodiments of the present invention are described below.

[1207] System Configuration

[1208] This system consists of a user device (e.g., PC, smartphone, tablet), a central server, and a database. Users access the system using their device to enter device information and report problems. The server analyzes the received information and generates appropriate advice and notifications. The database stores device information and past case studies.

[1209] Building the user interface

[1210] A user accesses the system using a web browser or mobile app. The device sends the user's login information to the server for authentication. The server verifies the user's authentication information and initiates a session to access the dashboard. The device displays the dashboard, providing a visual list of the devices the user has registered and their respective status information.

[1211] For example, a user launches a smartphone app, logs in, and accesses the dashboard, which displays the operating status of their refrigerator or washing machine, maintenance deadlines, and recommended replacement times.

[1212] Input and analysis of equipment information

[1213] Users register information about newly purchased or existing devices using an input form within the system. Detailed information such as the device name, manufacturer, model, serial number, purchase date, and warranty period is entered. The terminal sends the entered device information to the server. The server analyzes the received device information and stores it in the database in the appropriate format. The server verifies the saved device information and notifies the user that it has been correctly stored in the database.

[1214] An example prompt is, "Please enter information about your newly purchased refrigerator. Include details such as the manufacturer, model, purchase date, serial number, and warranty expiration date."

[1215] Device information analysis and problem solving

[1216] The user enters a problem or question about a specific device into a report form within the system and submits it. The device then sends the user's report to the server. The server then analyzes the received report using natural language processing (NLP) technology to identify the type of problem. The server then searches for similar problems and their solutions from past case studies in a database. The server then extracts the optimal solution and provides advice to the user.

[1217] For example, a user may report that "the TV cannot connect to the Internet," and the server may advise, "Please turn the router off and on again and connect the LAN cable correctly."

[1218] Bug debugging support

[1219] The user uses the debug reporting function in the system to enter details of the problem they are experiencing and report it. The device sends the user's report to the server. The server generates a basic diagnostic flow and presents it to the user. For example, it might show a procedure such as "Please check if the power cord is connected properly." If the user performs an initial diagnosis and the problem is still not resolved, they enter additional details into a form in the system. The device then sends the details back to the server. The server performs a deeper analysis of the received details and generates more advanced diagnostic procedures.

[1220] An example of a prompt sentence is when you report that "the air conditioner is not cooling" and the server advises you to "clean the internal filter."

[1221] Expired maintenance / replacement support

[1222] The server periodically scans the user's device list and checks the maintenance deadline for each device. The server detects devices whose maintenance deadline is approaching and generates a notification message. The device displays the notification message to the user. The server searches for new device candidates suitable for the user based on market information, and the device displays them as recommendations.

[1223] As a specific example, when the maintenance deadline for a user's washing machine is approaching, the server notifies the user that "(Manufacturer name) (Model name) is a good candidate for replacement."

[1224] Emotional engine regulation

[1225] The device collects the user's input, facial expressions, and voice during operation and sends them to the server. The server then analyzes the received data using an emotion engine to recognize the user's emotional state. For example, it may recognize that the user is feeling stressed. The server then adjusts the advice and response it provides based on the emotions recognized by the emotion engine. For example, it may provide simpler, gentler advice to a user who is feeling stressed.

[1226] As an example of a prompt sentence, for a user who is stressed because "my TV can't connect to the Internet," a question is displayed in a soft tone: "Is the light on your router lit normally?"

[1227] The present invention is embodied in the above-described manner, which allows users to effectively manage their home electronic devices and receive prompt and accurate support. Furthermore, flexible responses based on emotional states improve the user experience.

[1228] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1229] Step 1:

[1230] System Access and Login

[1231] A user launches a web browser or mobile app and attempts to access the system.

[1232] The device receives the access request and displays the login screen.

[1233] Input: User ID, Password

[1234] Output: Login screen displayed

[1235] Specific operation: Enter the user ID and password into the input form.

[1236] Step 2:

[1237] Authentication process

[1238] The device sends the login information to the server.

[1239] The server checks the received login information against the registered information in the database and performs authentication.

[1240] If the authentication is successful, the server initiates a session, generates an authentication token, and returns it to the terminal.

[1241] Displays the dashboard to the user with a success message.

[1242] If the authentication fails, the server generates an error message and sends it to the terminal.

[1243] Input: Login information (user ID, password)

[1244] Output: Authentication token, success or error message

[1245] Specific behavior: The server performs authentication by referencing the database.

[1246] Step 3:

[1247] Entering device information

[1248] The user registers information about a new or existing device using the system's input form.

[1249] The information entered includes the device name, manufacturer, model, serial number, purchase date, and warranty period.

[1250] The terminal transmits the input information to the server.

[1251] Input: Device information (device name, manufacturer, model, serial number, purchase date, warranty period)

[1252] Output: Send device information to the server

[1253] What happens: A user fills in the form with details and clicks the submit button.

[1254] Step 4:

[1255] Saving and checking device information

[1256] The server analyzes the received device information and stores it in a database in an appropriate format.

[1257] Once the save is complete, the server generates a success message and sends it to the device.

[1258] The terminal notifies the user of the successful save.

[1259] Input: Device information, data to send to server

[1260] Output: Database updated, success message

[1261] What happens: The server saves the information in the corresponding field in the database.

[1262] Step 5:

[1263] Problem report

[1264] A user enters a problem or question about a specific device into a reporting form within the system and submits it.

[1265] The terminal sends the report to the server.

[1266] Input: Problem Report (user-entered text)

[1267] Output: Send problem report to server

[1268] Specific behavior: The user enters the problem description and clicks the report button.

[1269] Step 6:

[1270] Analyzing the problem

[1271] The server analyzes the report received using natural language processing (NLP) technology to identify the type of problem.

[1272] The server searches a database of past case studies for similar problems and their solutions.

[1273] Input: Problem report text

[1274] Output: Identified problem types, search results for similar problems

[1275] Specific operation: The server performs text analysis using NLP technology.

[1276] Step 7:

[1277] Providing advice

[1278] The server extracts the best solution and provides advice to the user.

[1279] The terminal displays the solution to the user.

[1280] Input: Search results, solution data

[1281] Output: Advisory message

[1282] Specific operation: The server generates an advisory message and sends it to the terminal.

[1283] Step 8:

[1284] Maintenance expiration check and notification

[1285] The server periodically scans the user's device list and detects devices that are close to expiring for maintenance.

[1286] The server generates a notification message for a device whose maintenance deadline is approaching and sends it to the terminal.

[1287] The terminal displays a notification message to the user.

[1288] Input: Equipment list data

[1289] Output: Information message

[1290] Specific operation: The server scans the device list data and checks the maintenance deadline.

[1291] Step 9:

[1292] Replacement recommendation

[1293] The server searches for new device candidates based on market information and generates appropriate recommendations.

[1294] The device displays the recommendations to the user.

[1295] Input: Market information, equipment information

[1296] Output: Recommendation message

[1297] What happens: The server searches the market database for new models.

[1298] Step 10:

[1299] Recognition of emotional states

[1300] The device collects the user's input, facial expressions and voice during operation, and sends them to the server.

[1301] The server analyzes the received data using an emotion engine to recognize the user's emotional state.

[1302] Input: facial expression data, voice data, input content

[1303] Output: Emotional state data

[1304] Specific operation: The device collects data using the camera and microphone and sends it to the server.

[1305] Step 11:

[1306] Adjusting advice based on emotions

[1307] The server tailors advice and responses based on the recognized emotion.

[1308] The terminal displays the adjusted advice to the user.

[1309] Input: Emotional state data, advice data

[1310] Output: Adjusted advisory message

[1311] Specific behavior: The server changes the tone and content of advice depending on the emotional state.

[1312] (Application example 2)

[1313] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1314] Modern brick-and-mortar stores use many electronic devices and equipment, and managing and maintaining them is a key issue for store operations. However, managing the maintenance deadlines for these devices and equipment and responding to breakdowns places a heavy burden on store staff. Furthermore, efficient problem solving can be difficult depending on the emotional state of staff. Furthermore, there is a need to visualize the usage and management status of in-store equipment to improve customer service.

[1315] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to input device information, a means for saving the input device information in a database, a means for analyzing the saved device information and providing advice for solving problems, a means for periodically checking the device information and notifying the user of maintenance deadlines and recommended replacement times, a means for analyzing the user's emotional state using an emotion recognition engine and adjusting the advice content, and a means for visualizing the management and usage status of various in-store facilities. This enables centralized management of electronic devices and facilities in the store and further enables flexible responses tailored to the emotional states of staff. Furthermore, visualization of in-store equipment usage is expected to improve operational efficiency and customer service.

[1316] Key word definitions

[1317] "Device information" refers to detailed information about electronic devices and equipment, such as the name, model, manufacturer, serial number, purchase date, and warranty period.

[1318] A "database" is an information repository designed to efficiently store, manage, and search input device information.

[1319] "Analysis" is the process of analyzing stored information to understand a particular problem or situation and derive a solution.

[1320] "Advice" refers to specific solutions or methods of dealing with problems that arise.

[1321] "Maintenance period" refers to the warranty and maintenance service period for electronic equipment and facilities.

[1322] The "recommended time for replacement" is the time when it is recommended to replace the current equipment with a new one, taking into consideration the lifespan and performance of the current equipment.

[1323] An "emotion recognition engine" is a software component that has the technology to analyze a user's emotional state from their facial expressions and voice.

[1324] The "means for adjusting the content of advice" is a function for changing the tone and content of the advice provided depending on the emotional state of the user analyzed by the emotion recognition engine.

[1325] "Various in-store equipment" refers to all electronic devices and equipment necessary for store operation, such as POS systems, security cameras, air conditioning, and lighting.

[1326] "Visualization" refers to the graphical display of the management and usage status of various equipment used within the store.

[1327] MODE FOR CARRYING OUT THE INVENTION

[1328] The following describes in detail the mode for carrying out the present invention. This system supports the management, maintenance, and troubleshooting of various electronic devices and equipment installed in brick-and-mortar stores, and also uses an emotion recognition engine to provide appropriate advice according to the emotional state of staff.

[1329] Hardware and Software Configuration

[1330] Hardware

[1331] Smartphone (iOS, Android)

[1332] Tablet (iOS, Android)

[1333] Smart glasses (e.g. Google Glass, Vuzix Blade)

[1334] Sensors (face recognition camera, voice input microphone)

[1335] software

[1336] Natural Language Processing (NLP): Google Cloud Natural Language API, IBM Watson NLP

[1337] Emotion recognition engine: Affectiva SDK, Microsoft Azure Face API

[1338] Database: Firebase, MySQL

[1339] Web Server: Node.js, Express

[1340] Specific Embodiments

[1341] Registering and managing device information

[1342] Using a device (smartphone or tablet), store staff enter information about various electronic devices (device name, model, manufacturer, serial number, purchase date, warranty period, etc.) and store staff saves the information in a database (Firebase). This information is managed via a web server (Node.js, Express), and store staff can access it as needed from a web dashboard (browser interface).

[1343] Problem Reporting and Analysis

[1344] When a user (store staff member) reports a problem, the terminal sends the information to the server. The server uses a natural language processing (NLP) engine to analyze the report and search a database for past case studies. The server finds the optimal solution, sends it back to the terminal, and displays it to the staff member. For example, if a problem occurs with the POS system, the terminal will display the advice, "Please reconnect the power cord of the POS system."

[1345] Use of emotion recognition engine

[1346] The device's built-in camera and microphone collect the staff's facial expressions and voice, which are then analyzed by an emotion recognition engine (Affectiva SDK, Microsoft Azure Face API). Based on the analysis results, the server adjusts the tone and content of the advice. For example, if the staff member is irritated, the advice will be changed to a simpler, softer tone.

[1347] Maintenance and replacement support

[1348] The server periodically scans the database for device information and generates notifications for devices that are nearing the end of their maintenance period or have reached the recommended replacement date. The terminal displays these notifications to staff and also recommends new devices based on market information. For example, if an air conditioner's maintenance period is approaching, the system will display a message saying, "We recommend the XX model of XX manufacturer for your new air conditioner."

[1349] Specific examples

[1350] Example 1: Your POS system isn't working

[1351] Staff report issues:

[1352] A staff member reports that the POS system is not responding.

[1353] Emotion engine recognizes frustration:

[1354] The system recognizes irritation from staff members' facial expressions.

[1355] Softly presented advice:

[1356] "Don't worry. Let's check the power cord first," the message read in a gentle tone.

[1357] Example 2: Air conditioner maintenance deadline notification

[1358] Maintenance Expiration Notification:

[1359] The system detects that the air conditioner's maintenance deadline is approaching and notifies you, "Your air conditioner's maintenance deadline will expire at the end of this month."

[1360] Recommendations for tired staff:

[1361] Recognizing that the staff member is tired, the system displays in a soft tone, "We recommend (manufacturer name) (model name) for your new air conditioner."

[1362] Prompt Sentence Examples

[1363] 1. If your POS system isn't working:

[1364] A user says, "The POS system is not responding." What is the best advice and how can you provide a gentle solution to the user's frustration?

[1365] 2. Maintenance Expiration Notifications and Recommendations:

[1366] When notifying a user that their air conditioner's maintenance is due at the end of this month, gently recommend a new air conditioner model to a tired user.

[1367] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1368] Program processing steps

[1369] Step 1: Register device information

[1370] 1. User Input

[1371] A user (store staff member) uses a terminal (smartphone or tablet) to input information about various electronic devices, such as the device name, model, manufacturer, serial number, purchase date, and warranty period.

[1372] 2. Data Transmission

[1373] The device sends the entered device information to the server. The entered data is packaged in JSON format and sent using a secure communication protocol (HTTPS).

[1374] 3. Database storage

[1375] The server receives the input data and saves it in the database (Firebase). The server converts the received data into the appropriate format and saves it. A success message is sent to the device.

[1376] Step 2: Report and analyze the problem

[1377] 1. User Reports

[1378] A user (staff member) uses a terminal to report a problem with a specific device. The report content is entered in natural language.

[1379] 2. Data Transmission

[1380] The device sends the report to the server, also in JSON format using a secure communication protocol.

[1381] 3. Natural Language Processing

[1382] The server passes the received report to a natural language processing (NLP) engine (Google Cloud Natural Language API, IBM Watson NLP) to analyze the type of problem. The analyzed data is stored in an internal database.

[1383] 4. Search for a solution

[1384] The server searches an internal database for past case studies to find similar problems and their solutions.

[1385] 5. Generating Advice

[1386] The server generates an optimal solution and sends it to the terminal as advice to the user, for example, "Please reconnect the power cord of the POS system."

[1387] Step 3: Emotion recognition and advice adjustment

[1388] 1. Emotional data collection

[1389] The camera and microphone installed on the device are used to collect the facial expressions and voices of users (staff) in real time.

[1390] 2. Data Transmission

[1391] The collected emotion data is sent to the server in an appropriate format (e.g., as image data or audio data).

[1392] 3. Emotion analysis

[1393] The server analyzes the received data using an emotion recognition engine (Affectiva SDK, Microsoft Azure Face API) to identify the user's emotional state.

[1394] 4. Coordination of advice

[1395] The server adjusts the tone and content of the advice it provides based on the analyzed emotional state. For example, if the user is annoyed, it changes the advice to a simpler, more gentle tone.

[1396] 5. Submitting Advice

[1397] The adjusted advice is sent to the terminal and displayed in an appropriate form to the user.

[1398] Step 4: Maintenance and replacement support

[1399] 1. Regular scans

[1400] The server periodically scans the database to check the maintenance deadlines for each device.

[1401] 2. Notification generation

[1402] The server detects devices that are nearing the end of their maintenance period and generates a notification message that includes the expiration date and the recommended replacement time.

[1403] 3. Data Transmission

[1404] A notification message is sent to the terminal and displayed to the user.

[1405] 4. Recommendation Generation

[1406] The server searches for suitable new device candidates based on market information and generates a recommendation model.

[1407] 5. Sending Recommendations

[1408] The device displays new device candidates to the user. For example, a recommendation such as "We recommend the XX model of XX manufacturer for your new air conditioner" is displayed.

[1409] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1410] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1411] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1412] [Third embodiment]

[1413] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1414] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1415] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1416] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1417] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1418] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1419] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1420] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1421] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1422] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1423] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1424] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1425] The following describes in detail the mode for carrying out the present invention: This system allows a user to manage information about electronic devices in the home and receive support for solving problems.

[1426] Building the user interface

[1427] 1. Access and Dashboard View

[1428] A user accesses the system using a web browser or a mobile app.

[1429] The device displays a list of the user's registered devices on a dashboard, along with visual status information for each device, such as which devices are currently in operation, which devices are nearing the end of their maintenance period, and which devices are nearing the recommended replacement date.

[1430] Input and analysis of equipment information

[1431] 1. Registering device information

[1432] Users register information about newly purchased or existing devices using an input form within the system, including details such as device name, manufacturer, model, serial number, purchase date, and warranty period.

[1433] 2. Saving to the database

[1434] The terminal transmits the input device information to the server.

[1435] The server analyzes the received information and stores it in a database in an appropriate format.

[1436] Device information analysis and problem solving

[1437] 1. Reporting and analyzing problems

[1438] Users report problems or questions about specific devices by filling out a form within the system.

[1439] The server scans the reports it receives and analyzes them according to the type of problem, using natural language processing (NLP) techniques to understand the details of the problem.

[1440] 2. Providing advice

[1441] The server searches for similar problems from past case studies in a database and extracts the best solutions.

[1442] The device will display solutions to the user, such as "If the network connection is unstable, please restart the router."

[1443] Bug debugging support

[1444] 1. Reporting a Bug

[1445] Users report in detail any issues they are experiencing with a particular feature within the system.

[1446] 2. Initial diagnosis

[1447] The server provides a basic diagnostic flow to the user, for example, by providing instructions such as "Make sure the power cord is properly connected."

[1448] 3. Perform detailed diagnostics

[1449] If the user is unable to resolve the problem through initial diagnosis, the terminal collects additional details from the user and sends them to the server.

[1450] The server then analyzes the details received and suggests more advanced diagnostic steps, such as "The internal filter may be clogged. Please clean the filter."

[1451] Expired maintenance / replacement support

[1452] 1. Check the maintenance period

[1453] The server periodically scans the user's device list and checks the maintenance deadline for each device.

[1454] 2. Generate and send notifications

[1455] The server detects devices whose maintenance period is approaching and generates a notification to the user. For example, it generates a notification saying, "The maintenance period for your washing machine will expire at the end of next month," and displays it on the device.

[1456] 3. Providing recommendations

[1457] The server searches for new device candidates suitable for the user based on market information.

[1458] The device displays a recommendation that reads, "Your washing machine's maintenance deadline is approaching, so consider purchasing a new washing machine (manufacturer name) (model name)."

[1459] Specific examples

[1460] Example 1: When a wiring problem occurs

[1461] 1. Report a problem

[1462] User: Reports "TV won't connect to the Internet."

[1463] 2. Analysis and Advice

[1464] The device will ask the user the question, "Are the lights on your router working properly?"

[1465] User: "The green light is not on."

[1466] The server will analyze the problem and provide specific advice such as "Turn the router off and on again and connect the LAN cable correctly."

[1467] Example 2: Notification of expiration of maintenance period and recommendation to purchase a new product

[1468] 1. Notification of Maintenance Expiration

[1469] The server periodically checks to detect when a particular router is nearing the end of its maintenance period.

[1470] 2. Providing notifications and recommendations

[1471] The server generates a notification that says, "Your router's maintenance will expire at the end of this month," and the device displays it to the user.

[1472] Based on market information, the server suggests new router candidates to the user, such as "(Manufacturer name) (Model name) is recommended."

[1473] The present invention can be implemented in the above manner, and can perform device management, problem solving, and maintenance management for users in a unified manner. This allows users to use their electronic devices with peace of mind and receive prompt and accurate support when a problem occurs.

[1474] The processing flow will be explained below.

[1475] Building the user interface

[1476] Step 1:

[1477] A user opens a web browser or mobile app and accesses the system.

[1478] Step 2:

[1479] The terminal sends the user's login information to the server for authentication.

[1480] Step 3:

[1481] The server verifies the user's credentials and starts a session to allow access to the dashboard.

[1482] Step 4:

[1483] The device displays a dashboard that provides a visual list of the devices the user has registered and their respective status information.

[1484] Input and analysis of equipment information

[1485] Step 1:

[1486] When a user purchases or installs new equipment, they enter the equipment information into an input form within the system.

[1487] Step 2:

[1488] The device sends the entered device information (device name, manufacturer, model, serial number, purchase date, warranty period, etc.) to the server.

[1489] Step 3:

[1490] The server analyzes the received device information and stores it in a database in an appropriate format.

[1491] Step 4:

[1492] The server checks the saved device information and notifies the user that it has been correctly stored in the database.

[1493] Device information analysis and problem solving

[1494] Step 1:

[1495] Users submit problems or questions about specific devices by filling out a report form within the system.

[1496] Step 2:

[1497] The terminal transmits the user's report to the server.

[1498] Step 3:

[1499] The server analyzes the report received using natural language processing (NLP) technology to identify the type of problem.

[1500] Step 4:

[1501] The server searches a database of past case studies for similar problems and their solutions.

[1502] Step 5:

[1503] The server will extract the optimal solution and provide advice to the user, for example, "If your Internet connection is unstable, please reboot your router."

[1504] Bug debugging support

[1505] Step 1:

[1506] The user utilizes the debug reporting feature within the system to enter details of the problem they are experiencing and report it.

[1507] Step 2:

[1508] The terminal transmits the user's report to the server.

[1509] Step 3:

[1510] The server generates a basic diagnostic flow and presents it to the user, for example, showing a procedure such as "Check if the power cord is properly connected."

[1511] Step 4:

[1512] If the user performs an initial diagnosis and the problem persists, they enter additional details into a form within the system.

[1513] Step 5:

[1514] The terminal sends the detailed information back to the server.

[1515] Step 6:

[1516] The server then performs deep analysis of the details received and generates more advanced diagnostic procedures, such as displaying advice such as "The internal filter may be clogged. Please clean the filter."

[1517] Expired maintenance / replacement support

[1518] Step 1:

[1519] The server periodically scans the user's device list and checks the maintenance expiration date for each device.

[1520] Step 2:

[1521] The server detects devices that are close to expiring for maintenance and generates a notification message.

[1522] Step 3:

[1523] The device displays a notification message to the user, for example, "Your washing machine's maintenance period expires at the end of next month."

[1524] Step 4:

[1525] The server searches for new device candidates suitable for the user based on market information.

[1526] Step 5:

[1527] The device displays a recommendation to the user saying, "Your washing machine's maintenance deadline is approaching, so consider purchasing a new washing machine (manufacturer name) (model name)."

[1528] Example 1

[1529] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1530] Management of electronic devices in the home is generally done on a per-device basis, and when a problem occurs, the response varies from device to device, which can be time-consuming and difficult for users to understand. Furthermore, managing maintenance deadlines and replacement times manually can easily lead to errors, and there is a risk that the device will not last the full lifespan of the device. A system is needed to centrally manage these issues, reduce the burden on users, and enable rapid problem resolution.

[1531] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1532] In this invention, the server includes means for a user to input device information, means for saving the input device information in a database, means for analyzing the saved device information and providing advice for solving problems, means for periodically checking device information and notifying the user of maintenance deadlines and recommended replacement times, means for a user to report problems, means for analyzing reported problems and providing appropriate solutions, means for a user to report malfunctions, means for providing initial and detailed diagnoses, and means for displaying detailed advice. This allows for centralized management of electronic devices in the home, enabling confirmation of maintenance deadlines, notifications of recommended replacements, and rapid problem resolution.

[1533] A "user" is a person who uses this system to manage information about devices in the home.

[1534] "Device information" is detailed information about electronic devices in the home, including, for example, the device name, manufacturer, model, serial number, purchase date, warranty period, and so on.

[1535] "Input means" refers to the interface through which users input device information and problem details into the system, and includes web browsers and mobile apps.

[1536] A "database" is a collection of data managed by a server, and stores registered device information and data for problem solving.

[1537] "Analysis means" is a function in which the server analyzes device information and reports from users, and performs processing to provide appropriate advice and solutions.

[1538] The "advice providing means" is a function that provides the user with the solution obtained by the analysis means, and displays specific operating methods and countermeasures.

[1539] "Maintenance expiration date" means the end of the equipment warranty or maintenance contract period.

[1540] The "notification means" is a function that notifies the user of devices whose maintenance deadlines are approaching or when replacement is recommended.

[1541] The "recommendation means" is a function that suggests suitable new device candidates to the user, for example, by displaying recommended devices based on the latest market information.

[1542] The "problem reporting means" is a function for a user to report a device problem they are experiencing to the system, and includes an input form and a feedback function.

[1543] The "initial diagnosis means" is a function for performing basic diagnosis of a problem reported by the problem reporting means.

[1544] The "detailed diagnosis means" is a function for collecting more detailed information and conducting in-depth analysis of problems that could not be resolved through initial diagnosis.

[1545] "Advice display means" is a function that displays specific advice or solutions to the user. For example, it may display on the terminal screen, "Please turn the router off and on again, and connect the LAN cable correctly."

[1546] A specific embodiment for carrying out this invention will be described in detail below. This system allows users to manage information about electronic devices in their homes and receive support for solving problems. The system is designed to allow users to register and manage device information, and report and diagnose problems using a web browser or mobile app.

[1547] Building the user interface

[1548] Access and Dashboard Viewing

[1549] A user accesses the system using a web browser or mobile app. The device sends the authentication information entered by the user to the server, which then authenticates the user based on that information. Once authentication is complete, the server obtains the user's device information and displays it on the device in dashboard format. For example, it visually provides information such as devices currently in operation, devices nearing the end of their maintenance period, and devices nearing the recommended replacement date.

[1550] Input and analysis of equipment information

[1551] Registering device information

[1552] Users register information about newly purchased or existing devices through an input form, including details such as device name, manufacturer, model, serial number, purchase date, warranty period, etc. The device sends this information to the server, which stores it in a database in the appropriate format.

[1553] Saving to a database

[1554] The server analyzes the device information received from the terminal and stores it in a database. For example, the server normalizes the device information and stores it in a database to enable efficient searches. This process uses a database management system (DBMS) or similar.

[1555] Device information analysis and problem solving

[1556] Problem reporting and analysis

[1557] Users report problems or questions about a specific device by filling out a form within the system. For example, if a user reports that their TV cannot connect to the Internet, the server analyzes the report using natural language processing (NLP) techniques, such as the Google NLP API and other generative AI models.

[1558] Providing advice

[1559] The server searches for similar problems from past case studies in a database and extracts the optimal solution. For example, it displays a question on the device, such as "Are the lights on your router lit normally?", and provides specific advice based on the user's answer. The server generates advice such as "If your network connection is unstable, please turn the router off and on again and properly connect the LAN cable," and the advice is displayed on the device.

[1560] Bug debugging support

[1561] Reporting a Bug

[1562] A user reports a problem in detail using a specific function within the system. For example, if a user reports that their washing machine has stopped working, the details of the report are collected.

[1563] Initial diagnosis

[1564] The server analyzes the problem and provides a basic diagnostic flow, such as prompting the user to "Make sure the power cord is properly connected."

[1565] Performing detailed diagnostics

[1566] If the initial diagnosis doesn't solve the problem, the device collects additional details from the user and sends them to the server, which then performs a more detailed diagnosis and displays advanced advice, such as "The filter may be clogged, so please clean it."

[1567] Expired maintenance / replacement support

[1568] Check maintenance expiration date

[1569] The server periodically scans the user's device list and checks the maintenance expiration date for each device. For example, it detects information such as "The maintenance expiration date for the washing machine will expire in one month."

[1570] Generate and send notifications

[1571] The server detects approaching maintenance deadlines and generates notifications such as, "Your washing machine's maintenance deadline will expire at the end of next month." The generated notifications are displayed on the device.

[1572] Providing recommendations

[1573] The server searches for new appliance candidates suitable for the user based on market information. For example, it displays a recommendation message on the device saying, "A new washing machine from (manufacturer name) (model name) is recommended."

[1574] Specific examples

[1575] Below are some examples of prompt sentences.

[1576] Example prompt 1:

[1577] "Please explain the programming process for a home electronics information management system."

[1578] Example prompt 2:

[1579] "Could you please explain in detail what the process is when a user reports an issue?"

[1580] Example prompt 3:

[1581] "Please give specific examples of support for replacement of electronic devices in the home when their maintenance expires."

[1582] As described above, this system realizes the unification of management, problem solving, and maintenance of electronic devices used in the home, thereby greatly improving user convenience.

[1583] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1584] Step 1:

[1585] Access and Authentication

[1586] A user accesses the system using a web browser or mobile app. The device enters the user's authentication information (username and password) and sends it to the server. The server checks the authentication information against a database and sends the authentication result back to the device.

[1587] Input: Username, Password

[1588] Data processing: Authentication verification

[1589] Output: Authentication success / failure message

[1590] Step 2:

[1591] Viewing the Dashboard

[1592] If authentication is successful, the device displays the dashboard. The server retrieves the user's device information from the database and sends it to the device. The device displays the device information received in a list on the dashboard.

[1593] Input: User ID

[1594] Data processing: Acquisition of device information

[1595] Output: Dashboard screen

[1596] Step 3:

[1597] Registering device information

[1598] The user registers information about a newly purchased or existing device using an input form. The device enters the device information (device name, manufacturer, model, serial number, purchase date, warranty period, etc.) and sends it to the server. The server saves the information in a database.

[1599] Input: Device information (device name, manufacturer, model, serial number, purchase date, warranty period, etc.)

[1600] Data processing: Saving device information

[1601] Output: Registration complete message

[1602] Step 4:

[1603] Reporting an issue

[1604] Users fill out a form about a specific device issue and send it from their device to a server, which then analyzes the report using natural language processing (NLP) techniques.

[1605] Input: Problem report

[1606] Data processing: Problem analysis using NLP

[1607] Output: Problem analysis results

[1608] Step 5:

[1609] Providing advice

[1610] The server searches for similar problems from past case studies in a database and extracts appropriate solutions. The device displays the received solutions to the user. For example, for the problem "My TV can't connect to the Internet," the device displays the advice "Power cycle my router."

[1611] Input: Problem analysis results

[1612] Data processing: Finding case studies and extracting solutions

[1613] Output: Show solution

[1614] Step 6:

[1615] Reporting a Bug

[1616] A user reports a problem using a specific function within the system, and the report is sent from the terminal to the server.

[1617] Input: Bug report

[1618] Data processing: Saving report contents

[1619] Output: Report confirmation message

[1620] Step 7:

[1621] Providing an initial diagnosis

[1622] The server provides the user with a basic diagnostic flow. The terminal displays the diagnostic procedure received from the server to the user. For example, it displays an instruction such as "Please check if the power cord is connected properly."

[1623] Input: Bug report

[1624] Data processing: Basic diagnostic flow generation

[1625] Output: Display of diagnostic steps

[1626] Step 8:

[1627] Performing detailed diagnostics

[1628] If the user is unable to resolve the issue through initial diagnosis, the device will collect additional details from the user and send them to the server, which will then perform further analysis and provide advanced diagnostic procedures, such as displaying advice such as "Please clean the internal filter."

[1629] Input: More information

[1630] Data processing: Detailed diagnostic analysis

[1631] Output: Display of advanced diagnostic procedures

[1632] Step 9:

[1633] Check maintenance expiration date

[1634] The server periodically scans the database to check the maintenance deadlines of each device, detects devices that are close to the end of their maintenance period, and generates a notification to the user.

[1635] Input: Device information

[1636] Data processing: Check maintenance period

[1637] Output: Maintenance expiration notice

[1638] Step 10:

[1639] Providing recommendations

[1640] The server searches for new appliance candidates suitable for the user based on market information. The device displays a recommendation message to the user. For example, it displays a message saying, "We recommend a new washing machine from (manufacturer name) (model name)."

[1641] Input: Maintenance expiration information

[1642] Data processing: recommendation generation

[1643] Output: Recommendation notification

[1644] Through these steps, this system can centrally manage all electronic devices in the home, and provide quick problem resolution and appropriate maintenance support.

[1645] (Application example 1)

[1646] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1647] Conventional electronic device management systems for homes and brick-and-mortar stores have the problem of taking time and effort to monitor the status of devices and manage maintenance deadlines. Furthermore, it is difficult to quickly address problems when they occur, making rapid response particularly important in brick-and-mortar stores.

[1648] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1649] In this invention, the server includes: a means for a user to input device information; a means for saving the input device information in a database; a means for analyzing the saved device information and providing advice for resolving problems; a means for periodically checking device information and notifying users of maintenance deadlines and recommended replacement times; a means for providing notifications of maintenance deadlines and recommended replacement times using a visual display device; and a means for reporting device malfunctions and providing rapid problem resolution in real time via the visual display device. This makes it possible to monitor the status of electronic devices in real time and notify users of appropriate timing for maintenance or replacement. Furthermore, rapid problem resolution enables significant efficiency improvements in electronic device management, especially in physical stores.

[1650] The "means for users to input device information" is an interface that allows users to register information about newly purchased devices and existing devices in the system.

[1651] The "means for saving input device information in a database" is a system function that saves device information entered by a user in a database in an appropriate format.

[1652] The "means for analyzing stored device information" is a function for analyzing device information in the database and understanding the device status, maintenance deadlines, etc.

[1653] "Means for providing advice for solving problems" is a system function that presents appropriate solutions to users who report equipment malfunctions or problems.

[1654] "Means for periodically checking device information" refers to a function in which the server periodically scans the device information in the database to check things like maintenance deadlines and recommended replacement times.

[1655] The "means for notifying the user of the expiration date of maintenance or the recommended time for replacement" is a function for notifying the user of devices whose maintenance deadline is approaching or whose replacement is recommended.

[1656] "Means for providing notifications of maintenance deadlines and recommended replacement times using a visual display device" refers to a function that uses a visual display device (e.g., smart glasses, head-mounted display) to visually notify the user of maintenance deadlines and recommended replacement times.

[1657] "Means for reporting equipment malfunctions and quickly resolving problems in real time through a visual display device" is a function that analyzes equipment malfunctions reported by users in real time through a visual display device and provides quick solutions.

[1658] A "server" is a central computer that manages and analyzes data for the entire system.

[1659] The following describes in detail an embodiment of the present invention. This system allows users to manage electronic device information in a store and receive support for problem-solving. The users are primarily store clerks who use smart glasses or a head-mounted display to manage the devices.

[1660] Building the user interface

[1661] 1. Access and Dashboard View

[1662] Store associates access the system using smart glasses or a head-mounted display.

[1663] The device displays a list of the user's registered devices on a dashboard, along with visual status information for each device, such as which devices are currently in operation, which devices are nearing the end of their maintenance period, and which devices are nearing the recommended replacement date.

[1664] Input and analysis of equipment information

[1665] 1. Registering device information

[1666] The store clerk registers information about newly purchased or existing devices using an input form in the system, entering details such as the device name, manufacturer, model, serial number, purchase date, and warranty period.

[1667] 2. Saving to the database

[1668] The device sends the entered device information to the server, which analyzes the received information and stores it in a database in an appropriate format.

[1669] Device information analysis and problem solving

[1670] 1. Reporting and analyzing problems

[1671] A store clerk reports a problem or question about a specific device by filling out a form in the system. The server scans the report and analyzes it based on the type of problem. It then uses natural language processing (NLP) technology to understand the details of the problem.

[1672] 2. Providing advice

[1673] The server searches for similar problems from past case studies in a database and extracts the optimal solution. The device displays the solution to the user, providing specific advice such as "If your network connection is unstable, please restart your router."

[1674] Bug debugging support

[1675] 1. Reporting a Bug

[1676] The store associate uses a specific feature within the system to report in detail the problem they are experiencing.

[1677] 2. Initial diagnosis

[1678] The server provides a basic diagnostic flow to the user, for example, by providing instructions such as "Make sure the power cord is properly connected."

[1679] 3. Perform detailed diagnostics

[1680] If the store attendant is unable to resolve the problem through initial diagnosis, the terminal collects additional details from the store attendant and transmits them to the server.

[1681] The server then analyzes the details received and suggests more advanced diagnostic steps, such as "The internal filter may be clogged. Please clean the filter."

[1682] Expired maintenance / replacement support

[1683] 1. Check the maintenance period

[1684] The server periodically scans the user's device list and checks the maintenance deadline for each device.

[1685] 2. Generate and send notifications

[1686] The server detects devices whose maintenance period is approaching and generates a notification to the user. Specifically, it generates a notification saying, "The maintenance period for your washing machine will expire at the end of next month," and displays it on the device.

[1687] 3. Providing recommendations

[1688] The server searches for suitable new appliances for the user based on market information, and the device displays a recommendation such as, "Your current washing machine's maintenance deadline is approaching, so consider (manufacturer name) (model name) as your new washing machine."

[1689] Specific examples

[1690] Example 1: When a wiring problem occurs

[1691] 1. Report a problem

[1692] Store Clerk: Report "TV won't connect to the internet."

[1693] 2. Analysis and Advice

[1694] The device will ask the user the question, "Are the lights on your router working properly?"

[1695] Store Clerk: "The green light is not on."

[1696] The server will analyze the problem and provide specific advice such as "Turn the router off and on again and connect the LAN cable correctly."

[1697] Examples of prompt statements

[1698] Shop Electronics Problem Report: The register display is not working. Please identify the cause and provide a solution.

[1699] This system ensures thorough management of electronic devices in stores, ensuring adherence to maintenance deadlines and rapid troubleshooting. Real-time status monitoring and information provided through visual display devices dramatically improves user convenience.

[1700] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1701] Step 1:

[1702] The user enters the device information

[1703] Input: The sales associate enters details of the newly purchased or existing device (device name, manufacturer, model, serial number, purchase date, warranty period) into an input form.

[1704] How it works: This information is sent to the system via smart glasses or a head-mounted display.

[1705] Output: The entered data is sent to the system.

[1706] Step 2:

[1707] Save the entered device information in the database

[1708] Input: The device details obtained in step 1.

[1709] How it works: The server parses the information it receives and stores it in a database (such as SQLite) in an appropriate format.

[1710] Output: New device information is added to the database.

[1711] Step 3:

[1712] Dashboard View

[1713] Input: Equipment information stored in the database.

[1714] How it works: The device (smart glasses or head-mounted display) displays a list of device information on a dashboard, providing visual information on the status of each device (operating, approaching maintenance deadline, recommended replacement, etc.).

[1715] Output: Store staff can check the status of the equipment.

[1716] Step 4:

[1717] Problem reporting and analysis

[1718] Input: The store clerk fills out a problem report form to describe the equipment malfunction or problem.

[1719] How it works: The server analyzes the report using natural language processing (NLP) to understand the details of the problem. It uses a generative AI model to extract the type and details of the problem.

[1720] Output: Provides advice on next steps based on the extracted problem details.

[1721] Step 5:

[1722] Providing advice

[1723] Input: Detailed information about the problem analyzed in step 4.

[1724] How it works: The server searches a database of past case studies for similar problems and extracts the best solution. The generated advice is then displayed to the salesperson on the terminal.

[1725] Output: The store clerk receives specific advice, for example, "If your network connection is unstable, please power cycle your router."

[1726] Step 6:

[1727] Check maintenance expiration date

[1728] Input: Equipment information stored in the database.

[1729] How it works: The server periodically scans device information and checks the maintenance expiration date and usage period of each device.

[1730] Output: A list of devices due for maintenance soon is generated.

[1731] Step 7:

[1732] Notification of maintenance deadlines and recommended replacement times

[1733] Input: The list of equipment due for maintenance soon, generated in step 6.

[1734] How it works: The server generates notifications for devices that are due for maintenance soon, and the device (smart glasses or head-mounted display) visually displays them to the store clerk.

[1735] Output: The store clerk sees a notification that says, "Your washing machine's maintenance deadline will expire at the end of next month."

[1736] Step 8:

[1737] Providing recommendations

[1738] Input: New device candidates generated by the server based on market information.

[1739] How it works: The server generates a recommendation that reads, "Your current washing machine's maintenance deadline is approaching, so please consider purchasing a new washing machine (manufacturer name) (model name)," and displays it to the store clerk on the terminal.

[1740] Output: The store clerk reviews the new equipment candidates.

[1741] Step 9:

[1742] Detailed diagnosis of the problem

[1743] Input: Detailed information if the store attendant was unable to resolve the issue with initial diagnosis.

[1744] How it works: The device collects additional details from the store attendant and sends them to the server, which analyzes them and suggests more advanced diagnostic steps, such as "The internal filter may be clogged. Please clean the filter."

[1745] Output: The store associate receives detailed diagnostic results and instructions.

[1746] In this way, the entire system can efficiently support a series of processes from registering device information to problem solving, maintenance management, and replacement recommendations.

[1747] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1748] The following describes in detail the mode for carrying out the present invention. This system allows users to manage information about electronic devices in their homes and receive support for solving problems. It also aims to flexibly adjust advice and responses by combining it with an emotion engine that recognizes the user's emotions.

[1749] Building the user interface

[1750] 1. Access and Dashboard View

[1751] A user accesses the system using a web browser or a mobile app.

[1752] The terminal sends the user's login information to the server for authentication.

[1753] The server verifies the user's credentials and starts a session to allow access to the dashboard.

[1754] The device displays a dashboard that visually lists the devices registered by the user and their status information, such as which devices are currently in operation, which devices are nearing the end of their maintenance period, and which devices are nearing the recommended replacement date.

[1755] Input and analysis of equipment information

[1756] 1. Registering device information

[1757] Users register information about newly purchased or existing devices using an input form within the system, including details such as device name, manufacturer, model, serial number, purchase date, and warranty period.

[1758] The terminal transmits the input device information to the server.

[1759] The server analyzes the received device information and stores it in a database in an appropriate format.

[1760] The server checks the saved device information and notifies the user that it has been correctly stored in the database.

[1761] Device information analysis and problem solving

[1762] 1. Reporting and analyzing problems

[1763] Users submit problems or questions about specific devices by filling out a report form within the system.

[1764] The terminal transmits the user's report to the server.

[1765] The server analyzes the report received using natural language processing (NLP) technology to identify the type of problem.

[1766] The server searches a database of past case studies for similar problems and their solutions.

[1767] The server will extract the optimal solution and provide advice to the user, for example, "If your Internet connection is unstable, please reboot your router."

[1768] Bug debugging support

[1769] 1. Reporting a Bug

[1770] The user utilizes the debug reporting feature within the system to enter details of the problem they are experiencing and report it.

[1771] The terminal transmits the user's report to the server.

[1772] The server generates a basic diagnostic flow and presents it to the user, for example, showing a procedure such as "Check if the power cord is properly connected."

[1773] If the user performs an initial diagnosis and the problem persists, they enter additional details into a form within the system.

[1774] The terminal sends the detailed information back to the server.

[1775] The server then performs deep analysis of the details received and generates more advanced diagnostic procedures, such as displaying advice such as "The internal filter may be clogged. Please clean the filter."

[1776] Expired maintenance / replacement support

[1777] 1. Check the maintenance period

[1778] The server periodically scans the user's device list and checks the maintenance expiration date for each device.

[1779] The server detects devices that are close to expiring for maintenance and generates a notification message.

[1780] The device displays a notification message to the user, for example, "Your washing machine's maintenance period expires at the end of next month."

[1781] 2. Providing recommendations

[1782] The server searches for new device candidates suitable for the user based on market information.

[1783] The device displays a recommendation to the user saying, "Your washing machine's maintenance deadline is approaching, so consider purchasing a new washing machine (manufacturer name) (model name)."

[1784] Emotional engine regulation

[1785] 1. Emotional awareness

[1786] The device collects the user's input, facial expressions and voice during operation, and sends them to the server.

[1787] The server uses an emotion engine to analyze the received data and recognize the user's emotional state. For example, the server may recognize that the user is feeling stressed.

[1788] 2. Coordination of advice

[1789] The server adjusts the advice and responses it provides based on the emotions recognized by the emotion engine, for example providing simpler, gentler advice to a user who is feeling stressed.

[1790] Specific examples

[1791] Example 1: When a wiring problem occurs

[1792] 1. Report a problem

[1793] User: Reports "TV won't connect to the Internet."

[1794] Emotion engine: Recognizes irritation from the user's facial expression.

[1795] 2. Analysis and Advice

[1796] The device will ask the user in a soft tone, "Are the router lights up properly?"

[1797] User: "The green light is not on."

[1798] The server will analyze the problem and provide gentle advice such as, "Please turn the router off and on again and connect the LAN cable correctly."

[1799] Example 2: Notification of expiration of maintenance period and recommendation to purchase a new product

[1800] 1. Notification of Maintenance Expiration

[1801] The server periodically checks to detect when a particular router is nearing the end of its maintenance period.

[1802] Emotion engine: Recognizes that the user receiving the notification is tired.

[1803] 2. Providing notifications and recommendations

[1804] The server generates a notification that says, "Your router's maintenance will expire at the end of this month," and the device displays it to the user in a soft tone.

[1805] Based on market information, the server presents new router candidates, such as "(Manufacturer name) (Model name) is recommended," to tired users.

[1806] The present invention, embodied in the above-described manner, allows users to centrally manage their devices, solve problems, and manage maintenance. This allows users to use their electronic devices with peace of mind and receive prompt and accurate support when problems occur. Furthermore, the use of an emotion engine enables flexible responses tailored to the user's emotional state, improving the user experience.

[1807] The processing flow will be explained below.

[1808] Building the user interface

[1809] Step 1:

[1810] A user opens a web browser or mobile app and accesses the system.

[1811] Step 2:

[1812] The terminal sends the user's login information to the server for authentication.

[1813] Step 3:

[1814] The server verifies the user's credentials and starts a session to allow access to the dashboard.

[1815] Step 4:

[1816] The device displays a dashboard that visually provides information about the user's registered devices and their status.

[1817] Input and analysis of equipment information

[1818] Step 1:

[1819] When a user purchases or installs new equipment, they enter the equipment information into an input form within the system.

[1820] Step 2:

[1821] The device sends the entered device information (device name, manufacturer, model, serial number, purchase date, warranty period, etc.) to the server.

[1822] Step 3:

[1823] The server analyzes the received device information and stores it in a database in an appropriate format.

[1824] Step 4:

[1825] The server checks the saved device information and notifies the user that it has been correctly stored in the database.

[1826] Device information analysis and problem solving

[1827] Step 1:

[1828] Users submit problems or questions about specific devices by filling out a report form within the system.

[1829] Step 2:

[1830] The terminal transmits the user's report to the server.

[1831] Step 3:

[1832] The server analyzes the report received using natural language processing (NLP) technology to identify the type of problem.

[1833] Step 4:

[1834] The server searches a database of past case studies for similar problems and their solutions.

[1835] Step 5:

[1836] The server will extract the optimal solution and provide advice to the user, for example, "If your Internet connection is unstable, please reboot your router."

[1837] Bug debugging support

[1838] Step 1:

[1839] The user reports the problem they are experiencing by entering it using the debug reporting function within the system.

[1840] Step 2:

[1841] The terminal transmits the user's report to the server.

[1842] Step 3:

[1843] The server generates a basic diagnostic flow and presents it to the user, for example, showing a procedure such as "Check if the power cord is properly connected."

[1844] Step 4:

[1845] If the user performs an initial diagnosis and the problem persists, they enter additional details into a form within the system.

[1846] Step 5:

[1847] The terminal sends the detailed information back to the server.

[1848] Step 6:

[1849] The server then performs deep analysis of the details received and generates more advanced diagnostic procedures, such as displaying advice such as "The internal filter may be clogged. Please clean the filter."

[1850] Expired maintenance / replacement support

[1851] Step 1:

[1852] The server periodically scans the user's device list and checks the maintenance expiration date for each device.

[1853] Step 2:

[1854] The server detects devices that are close to expiring for maintenance and generates a notification message.

[1855] Step 3:

[1856] The device displays a notification message to the user, for example, "Your washing machine's maintenance period expires at the end of next month."

[1857] Step 4:

[1858] The server searches for new device candidates suitable for the user based on market information.

[1859] Step 5:

[1860] The device displays a recommendation to the user saying, "Your washing machine's maintenance deadline is approaching, so consider purchasing a new washing machine (manufacturer name) (model name)."

[1861] Emotional engine regulation

[1862] Step 1:

[1863] The device uses an emotion engine to collect the user's input, facial expressions, and voice during operation, and transmits these to the server.

[1864] Step 2:

[1865] The server uses an emotion engine to analyze the received data and recognize the user's emotional state, for example, recognizing that the user is feeling stressed.

[1866] Step 3:

[1867] The server adjusts the advice and response it provides based on the emotions recognized by the emotion engine.

[1868] Step 4:

[1869] The device will then display tailored advice and solutions to the user, such as "Please power cycle the router and properly connect the LAN cable" in a soft tone.

[1870] Specific examples

[1871] Example 1: When a wiring problem occurs

[1872] Step 1:

[1873] A user reports that their TV cannot connect to the Internet.

[1874] Step 2:

[1875] The emotion engine recognizes irritation from the user's facial expression.

[1876] Step 3:

[1877] The device will ask the user in a soft tone, "Are the router lights up properly?"

[1878] Step 4:

[1879] The user responds, "The green light is not on."

[1880] Step 5:

[1881] The server will analyze the problem and provide gentle advice such as, "Please turn the router off and on again and connect the LAN cable correctly."

[1882] Example 2: Notification of expiration of maintenance period and recommendation to purchase a new product

[1883] Step 1:

[1884] The server periodically checks to detect when a particular router is nearing the end of its maintenance period.

[1885] Step 2:

[1886] The emotion engine recognizes that the user receiving the notification is tired.

[1887] Step 3:

[1888] The server generates a notification that says, "Your router's maintenance will expire at the end of this month," and the device displays it to the user in a soft tone.

[1889] Step 4:

[1890] Based on market information, the server presents new router candidates, such as "(Manufacturer name) (Model name) is recommended," to tired users.

[1891] The present invention, embodied in the above-described manner, allows users to centrally manage their devices, solve problems, and manage maintenance. This allows users to use their electronic devices with peace of mind and receive prompt and accurate support when problems occur. Furthermore, the use of an emotion engine enables flexible responses tailored to the user's emotional state, improving the user experience.

[1892] Example 2

[1893] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1894] In conventional home electronic device management systems, it is difficult to manually input device information and determine when maintenance or replacement is required. Furthermore, when a problem occurs with an electronic device, it is often difficult for the user to resolve the problem themselves, requiring time and effort. Furthermore, when users experience a problem, they often experience emotional stress, which can remain as stress. The purpose of this invention is to solve these problems and enable users to use electronic devices with peace of mind.

[1895] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for a user to input device information, means for saving the input device information in a database, means for analyzing the saved device information and providing advice for solving problems, means for periodically checking the device information and notifying the user of maintenance deadlines and recommended replacement times, and emotion engine means for recognizing the user's emotional state and adjusting the advice content based on the recognized emotion. This enables the user to efficiently manage electronic devices in their home, receive prompt and accurate support when a problem occurs, and realize flexible responses tailored to the user's emotional state.

[1896] "Means for users to input device information" refers to a function that allows users to input information about new or existing devices through an interface within the system and send it to the server.

[1897] The "means for saving input device information in a database" is a function for receiving device information sent from a terminal and storing it in a database in an appropriate format.

[1898] The "means for analyzing stored device information and providing advice for resolving problems" is a function for analyzing device information stored in a database and providing appropriate advice and solutions for specific problems.

[1899] "Means to periodically check device information and notify users of maintenance deadlines and recommended replacement times" is a function that periodically scans the device information in the database, detects devices that are approaching their maintenance deadlines or devices that are recommended for replacement, and notifies the user.

[1900] "Emotion engine means for recognizing the user's emotional state and adjusting advice content based on the recognized emotion" is a function for recognizing the user's emotional state by analyzing the user's input content and facial expressions and voice during operation, and adjusting the advice and response provided based on the recognition results.

[1901] The present invention is a system for allowing users to manage information about electronic devices in their homes and receive assistance in solving problems, and further combines it with an emotion engine that recognizes the user's emotional state. Detailed embodiments of the present invention are described below.

[1902] System Configuration

[1903] This system consists of a user device (e.g., PC, smartphone, tablet), a central server, and a database. Users access the system using their device to enter device information and report problems. The server analyzes the received information and generates appropriate advice and notifications. The database stores device information and past case studies.

[1904] Building the user interface

[1905] A user accesses the system using a web browser or mobile app. The device sends the user's login information to the server for authentication. The server verifies the user's authentication information and initiates a session to access the dashboard. The device displays the dashboard, providing a visual list of the devices the user has registered and their respective status information.

[1906] For example, a user launches a smartphone app, logs in, and accesses the dashboard, which displays the operating status of their refrigerator or washing machine, maintenance deadlines, and recommended replacement times.

[1907] Input and analysis of equipment information

[1908] Users register information about newly purchased or existing devices using an input form within the system. Detailed information such as the device name, manufacturer, model, serial number, purchase date, and warranty period is entered. The terminal sends the entered device information to the server. The server analyzes the received device information and stores it in the database in the appropriate format. The server verifies the saved device information and notifies the user that it has been correctly stored in the database.

[1909] An example prompt is, "Please enter information about your newly purchased refrigerator. Include details such as the manufacturer, model, purchase date, serial number, and warranty expiration date."

[1910] Device information analysis and problem solving

[1911] The user enters a problem or question about a specific device into a report form within the system and submits it. The device then sends the user's report to the server. The server then analyzes the received report using natural language processing (NLP) technology to identify the type of problem. The server then searches for similar problems and their solutions from past case studies in a database. The server then extracts the optimal solution and provides advice to the user.

[1912] For example, a user may report that "the TV cannot connect to the Internet," and the server may advise, "Please turn the router off and on again and connect the LAN cable correctly."

[1913] Bug debugging support

[1914] The user uses the debug reporting function in the system to enter details of the problem they are experiencing and report it. The device sends the user's report to the server. The server generates a basic diagnostic flow and presents it to the user. For example, it might show a procedure such as "Please check if the power cord is connected properly." If the user performs an initial diagnosis and the problem is still not resolved, they enter additional details into a form in the system. The device then sends the details back to the server. The server performs a deeper analysis of the received details and generates more advanced diagnostic procedures.

[1915] An example of a prompt sentence is when you report that "the air conditioner is not cooling" and the server advises you to "clean the internal filter."

[1916] Expired maintenance / replacement support

[1917] The server periodically scans the user's device list and checks the maintenance deadline for each device. The server detects devices whose maintenance deadline is approaching and generates a notification message. The device displays the notification message to the user. The server searches for new device candidates suitable for the user based on market information, and the device displays them as recommendations.

[1918] As a specific example, when the maintenance deadline for a user's washing machine is approaching, the server notifies the user that "(Manufacturer name) (Model name) is a good candidate for replacement."

[1919] Emotional engine regulation

[1920] The device collects the user's input, facial expressions, and voice during operation and sends them to the server. The server then analyzes the received data using an emotion engine to recognize the user's emotional state. For example, it may recognize that the user is feeling stressed. The server then adjusts the advice and response it provides based on the emotions recognized by the emotion engine. For example, it may provide simpler, gentler advice to a user who is feeling stressed.

[1921] As an example of a prompt sentence, for a user who is stressed because "my TV can't connect to the Internet," a question is displayed in a soft tone: "Is the light on your router lit normally?"

[1922] The present invention is embodied in the above-described manner, which allows users to effectively manage their home electronic devices and receive prompt and accurate support. Furthermore, flexible responses based on emotional states improve the user experience.

[1923] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1924] Step 1:

[1925] System Access and Login

[1926] A user launches a web browser or mobile app and attempts to access the system.

[1927] The device receives the access request and displays the login screen.

[1928] Input: User ID, Password

[1929] Output: Login screen displayed

[1930] Specific operation: Enter the user ID and password into the input form.

[1931] Step 2:

[1932] Authentication process

[1933] The device sends the login information to the server.

[1934] The server checks the received login information against the registered information in the database and performs authentication.

[1935] If the authentication is successful, the server initiates a session, generates an authentication token, and returns it to the terminal.

[1936] Displays the dashboard to the user with a success message.

[1937] If the authentication fails, the server generates an error message and sends it to the terminal.

[1938] Input: Login information (user ID, password)

[1939] Output: Authentication token, success or error message

[1940] Specific behavior: The server performs authentication by referencing the database.

[1941] Step 3:

[1942] Entering device information

[1943] The user registers information about a new or existing device using the system's input form.

[1944] The information entered includes the device name, manufacturer, model, serial number, purchase date, and warranty period.

[1945] The terminal transmits the input information to the server.

[1946] Input: Device information (device name, manufacturer, model, serial number, purchase date, warranty period)

[1947] Output: Send device information to the server

[1948] What happens: A user fills in the form with details and clicks the submit button.

[1949] Step 4:

[1950] Saving and checking device information

[1951] The server analyzes the received device information and stores it in a database in an appropriate format.

[1952] Once the save is complete, the server generates a success message and sends it to the device.

[1953] The terminal notifies the user of the successful save.

[1954] Input: Device information, data to send to server

[1955] Output: Database updated, success message

[1956] What happens: The server saves the information in the corresponding field in the database.

[1957] Step 5:

[1958] Problem report

[1959] A user enters a problem or question about a specific device into a reporting form within the system and submits it.

[1960] The terminal sends the report to the server.

[1961] Input: Problem Report (user-entered text)

[1962] Output: Send problem report to server

[1963] Specific behavior: The user enters the problem description and clicks the report button.

[1964] Step 6:

[1965] Analyzing the problem

[1966] The server analyzes the report received using natural language processing (NLP) technology to identify the type of problem.

[1967] The server searches a database of past case studies for similar problems and their solutions.

[1968] Input: Problem report text

[1969] Output: Identified problem types, search results for similar problems

[1970] Specific operation: The server performs text analysis using NLP technology.

[1971] Step 7:

[1972] Providing advice

[1973] The server extracts the best solution and provides advice to the user.

[1974] The terminal displays the solution to the user.

[1975] Input: Search results, solution data

[1976] Output: Advisory message

[1977] Specific operation: The server generates an advisory message and sends it to the terminal.

[1978] Step 8:

[1979] Maintenance expiration check and notification

[1980] The server periodically scans the user's device list and detects devices that are close to expiring for maintenance.

[1981] The server generates a notification message for a device whose maintenance deadline is approaching and sends it to the terminal.

[1982] The terminal displays a notification message to the user.

[1983] Input: Equipment list data

[1984] Output: Information message

[1985] Specific operation: The server scans the device list data and checks the maintenance deadline.

[1986] Step 9:

[1987] Replacement recommendation

[1988] The server searches for new device candidates based on market information and generates appropriate recommendations.

[1989] The device displays the recommendations to the user.

[1990] Input: Market information, equipment information

[1991] Output: Recommendation message

[1992] What happens: The server searches the market database for new models.

[1993] Step 10:

[1994] Recognition of emotional states

[1995] The device collects the user's input, facial expressions and voice during operation, and sends them to the server.

[1996] The server analyzes the received data using an emotion engine to recognize the user's emotional state.

[1997] Input: facial expression data, voice data, input content

[1998] Output: Emotional state data

[1999] Specific operation: The device collects data using the camera and microphone and sends it to the server.

[2000] Step 11:

[2001] Adjusting advice based on emotions

[2002] The server tailors advice and responses based on the recognized emotion.

[2003] The terminal displays the adjusted advice to the user.

[2004] Input: Emotional state data, advice data

[2005] Output: Adjusted advisory message

[2006] Specific behavior: The server changes the tone and content of advice depending on the emotional state.

[2007] (Application example 2)

[2008] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2009] Modern brick-and-mortar stores use many electronic devices and equipment, and managing and maintaining them is a key issue for store operations. However, managing the maintenance deadlines for these devices and equipment and responding to breakdowns places a heavy burden on store staff. Furthermore, efficient problem solving can be difficult depending on the emotional state of staff. Furthermore, there is a need to visualize the usage and management status of in-store equipment to improve customer service.

[2010] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to input device information, a means for saving the input device information in a database, a means for analyzing the saved device information and providing advice for solving problems, a means for periodically checking the device information and notifying the user of maintenance deadlines and recommended replacement times, a means for analyzing the user's emotional state using an emotion recognition engine and adjusting the advice content, and a means for visualizing the management and usage status of various in-store facilities. This enables centralized management of electronic devices and facilities in the store and further enables flexible responses tailored to the emotional states of staff. Furthermore, visualization of in-store equipment usage is expected to improve operational efficiency and customer service.

[2011] Key word definitions

[2012] "Device information" refers to detailed information about electronic devices and equipment, such as the name, model, manufacturer, serial number, purchase date, and warranty period.

[2013] A "database" is an information repository designed to efficiently store, manage, and search input device information.

[2014] "Analysis" is the process of analyzing stored information to understand a particular problem or situation and derive a solution.

[2015] "Advice" refers to specific solutions or methods of dealing with problems that arise.

[2016] "Maintenance period" refers to the warranty and maintenance service period for electronic equipment and facilities.

[2017] The "recommended time for replacement" is the time when it is recommended to replace the current equipment with a new one, taking into consideration the lifespan and performance of the current equipment.

[2018] An "emotion recognition engine" is a software component that has the technology to analyze a user's emotional state from their facial expressions and voice.

[2019] The "means for adjusting the content of advice" is a function for changing the tone and content of the advice provided depending on the emotional state of the user analyzed by the emotion recognition engine.

[2020] "Various in-store equipment" refers to all electronic devices and equipment necessary for store operation, such as POS systems, security cameras, air conditioning, and lighting.

[2021] "Visualization" refers to the graphical display of the management and usage status of various equipment used within the store.

[2022] MODE FOR CARRYING OUT THE INVENTION

[2023] The following describes in detail the mode for carrying out the present invention. This system supports the management, maintenance, and troubleshooting of various electronic devices and equipment installed in brick-and-mortar stores, and also uses an emotion recognition engine to provide appropriate advice according to the emotional state of staff.

[2024] Hardware and Software Configuration

[2025] Hardware

[2026] Smartphone (iOS, Android)

[2027] Tablet (iOS, Android)

[2028] Smart glasses (e.g. Google Glass, Vuzix Blade)

[2029] Sensors (face recognition camera, voice input microphone)

[2030] software

[2031] Natural Language Processing (NLP): Google Cloud Natural Language API, IBM Watson NLP

[2032] Emotion recognition engine: Affectiva SDK, Microsoft Azure Face API

[2033] Database: Firebase, MySQL

[2034] Web Server: Node.js, Express

[2035] Specific Embodiments

[2036] Registering and managing device information

[2037] Using a device (smartphone or tablet), store staff enter information about various electronic devices (device name, model, manufacturer, serial number, purchase date, warranty period, etc.) and store staff saves the information in a database (Firebase). This information is managed via a web server (Node.js, Express), and store staff can access it as needed from a web dashboard (browser interface).

[2038] Problem Reporting and Analysis

[2039] When a user (store staff member) reports a problem, the terminal sends the information to the server. The server uses a natural language processing (NLP) engine to analyze the report and search a database for past case studies. The server finds the optimal solution, sends it back to the terminal, and displays it to the staff member. For example, if a problem occurs with the POS system, the terminal will display the advice, "Please reconnect the power cord of the POS system."

[2040] Use of emotion recognition engine

[2041] The device's built-in camera and microphone collect the staff's facial expressions and voice, which are then analyzed by an emotion recognition engine (Affectiva SDK, Microsoft Azure Face API). Based on the analysis results, the server adjusts the tone and content of the advice. For example, if the staff member is irritated, the advice will be changed to a simpler, softer tone.

[2042] Maintenance and replacement support

[2043] The server periodically scans the database for device information and generates notifications for devices that are nearing the end of their maintenance period or have reached the recommended replacement date. The terminal displays these notifications to staff and also recommends new devices based on market information. For example, if an air conditioner's maintenance period is approaching, the system will display a message saying, "We recommend the XX model of XX manufacturer for your new air conditioner."

[2044] Specific examples

[2045] Example 1: Your POS system isn't working

[2046] Staff report issues:

[2047] A staff member reports that the POS system is not responding.

[2048] Emotion engine recognizes frustration:

[2049] The system recognizes irritation from staff members' facial expressions.

[2050] Softly presented advice:

[2051] "Don't worry. Let's check the power cord first," the message read in a gentle tone.

[2052] Example 2: Air conditioner maintenance deadline notification

[2053] Maintenance Expiration Notification:

[2054] The system detects that the air conditioner's maintenance deadline is approaching and notifies you, "Your air conditioner's maintenance deadline will expire at the end of this month."

[2055] Recommendations for tired staff:

[2056] Recognizing that the staff member is tired, the system displays in a soft tone, "We recommend (manufacturer name) (model name) for your new air conditioner."

[2057] Prompt Sentence Examples

[2058] 1. If your POS system isn't working:

[2059] A user says, "The POS system is not responding." What is the best advice and how can you provide a gentle solution to the user's frustration?

[2060] 2. Maintenance Expiration Notifications and Recommendations:

[2061] When notifying a user that their air conditioner's maintenance is due at the end of this month, gently recommend a new air conditioner model to a tired user.

[2062] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2063] Program processing steps

[2064] Step 1: Register device information

[2065] 1. User Input

[2066] A user (store staff member) uses a terminal (smartphone or tablet) to input information about various electronic devices, such as the device name, model, manufacturer, serial number, purchase date, and warranty period.

[2067] 2. Data Transmission

[2068] The device sends the entered device information to the server. The entered data is packaged in JSON format and sent using a secure communication protocol (HTTPS).

[2069] 3. Database storage

[2070] The server receives the input data and saves it in the database (Firebase). The server converts the received data into the appropriate format and saves it. A success message is sent to the device.

[2071] Step 2: Report and analyze the problem

[2072] 1. User Reports

[2073] A user (staff member) uses a terminal to report a problem with a specific device. The report content is entered in natural language.

[2074] 2. Data Transmission

[2075] The device sends the report to the server, also in JSON format using a secure communication protocol.

[2076] 3. Natural Language Processing

[2077] The server passes the received report to a natural language processing (NLP) engine (Google Cloud Natural Language API, IBM Watson NLP) to analyze the type of problem. The analyzed data is stored in an internal database.

[2078] 4. Search for a solution

[2079] The server searches an internal database for past case studies to find similar problems and their solutions.

[2080] 5. Generating Advice

[2081] The server generates an optimal solution and sends it to the terminal as advice to the user, for example, "Please reconnect the power cord of the POS system."

[2082] Step 3: Emotion recognition and advice adjustment

[2083] 1. Emotional data collection

[2084] The camera and microphone installed on the device are used to collect the facial expressions and voices of users (staff) in real time.

[2085] 2. Data Transmission

[2086] The collected emotion data is sent to the server in an appropriate format (e.g., as image data or audio data).

[2087] 3. Emotion analysis

[2088] The server analyzes the received data using an emotion recognition engine (Affectiva SDK, Microsoft Azure Face API) to identify the user's emotional state.

[2089] 4. Coordination of advice

[2090] The server adjusts the tone and content of the advice it provides based on the analyzed emotional state. For example, if the user is annoyed, it changes the advice to a simpler, more gentle tone.

[2091] 5. Submitting Advice

[2092] The adjusted advice is sent to the terminal and displayed in an appropriate form to the user.

[2093] Step 4: Maintenance and replacement support

[2094] 1. Regular scans

[2095] The server periodically scans the database to check the maintenance deadlines for each device.

[2096] 2. Notification generation

[2097] The server detects devices that are nearing the end of their maintenance period and generates a notification message that includes the expiration date and the recommended replacement time.

[2098] 3. Data Transmission

[2099] A notification message is sent to the terminal and displayed to the user.

[2100] 4. Recommendation Generation

[2101] The server searches for suitable new device candidates based on market information and generates a recommendation model.

[2102] 5. Sending Recommendations

[2103] The device displays new device candidates to the user. For example, a recommendation such as "We recommend the XX model of XX manufacturer for your new air conditioner" is displayed.

[2104] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2105] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2106] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2107] [Fourth embodiment]

[2108] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2109] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2110] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2111] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2112] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2113] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2114] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2115] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2116] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2117] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2118] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2119] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2120] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2121] The following describes in detail the mode for carrying out the present invention: This system allows a user to manage information about electronic devices in the home and receive support for solving problems.

[2122] Building the user interface

[2123] 1. Access and Dashboard View

[2124] A user accesses the system using a web browser or a mobile app.

[2125] The device displays a list of the user's registered devices on a dashboard, along with visual status information for each device, such as which devices are currently in operation, which devices are nearing the end of their maintenance period, and which devices are nearing the recommended replacement date.

[2126] Input and analysis of equipment information

[2127] 1. Registering device information

[2128] Users register information about newly purchased or existing devices using an input form within the system, including details such as device name, manufacturer, model, serial number, purchase date, and warranty period.

[2129] 2. Saving to the database

[2130] The terminal transmits the input device information to the server.

[2131] The server analyzes the received information and stores it in a database in an appropriate format.

[2132] Device information analysis and problem solving

[2133] 1. Reporting and analyzing problems

[2134] Users report problems or questions about specific devices by filling out a form within the system.

[2135] The server scans the reports it receives and analyzes them according to the type of problem, using natural language processing (NLP) techniques to understand the details of the problem.

[2136] 2. Providing advice

[2137] The server searches for similar problems from past case studies in a database and extracts the best solutions.

[2138] The device will display solutions to the user, such as "If the network connection is unstable, please restart the router."

[2139] Bug debugging support

[2140] 1. Reporting a Bug

[2141] Users report in detail any issues they are experiencing with a particular feature within the system.

[2142] 2. Initial diagnosis

[2143] The server provides a basic diagnostic flow to the user, for example, by providing instructions such as "Make sure the power cord is properly connected."

[2144] 3. Perform detailed diagnostics

[2145] If the user is unable to resolve the problem through initial diagnosis, the terminal collects additional details from the user and sends them to the server.

[2146] The server then analyzes the details received and suggests more advanced diagnostic steps, such as "The internal filter may be clogged. Please clean the filter."

[2147] Expired maintenance / replacement support

[2148] 1. Check the maintenance period

[2149] The server periodically scans the user's device list and checks the maintenance deadline for each device.

[2150] 2. Generate and send notifications

[2151] The server detects devices whose maintenance period is approaching and generates a notification to the user. For example, it generates a notification saying, "The maintenance period for your washing machine will expire at the end of next month," and displays it on the device.

[2152] 3. Providing recommendations

[2153] The server searches for new device candidates suitable for the user based on market information.

[2154] The device displays a recommendation that reads, "Your washing machine's maintenance deadline is approaching, so consider purchasing a new washing machine (manufacturer name) (model name)."

[2155] Specific examples

[2156] Example 1: When a wiring problem occurs

[2157] 1. Report a problem

[2158] User: Reports "TV won't connect to the Internet."

[2159] 2. Analysis and Advice

[2160] The device will ask the user the question, "Are the lights on your router working properly?"

[2161] User: "The green light is not on."

[2162] The server will analyze the problem and provide specific advice such as "Turn the router off and on again and connect the LAN cable correctly."

[2163] Example 2: Notification of expiration of maintenance period and recommendation to purchase a new product

[2164] 1. Notification of Maintenance Expiration

[2165] The server periodically checks to detect when a particular router is nearing the end of its maintenance period.

[2166] 2. Providing notifications and recommendations

[2167] The server generates a notification that says, "Your router's maintenance will expire at the end of this month," and the device displays it to the user.

[2168] Based on market information, the server suggests new router candidates to the user, such as "(Manufacturer name) (Model name) is recommended."

[2169] The present invention can be implemented in the above manner, and can perform device management, problem solving, and maintenance management for users in a unified manner. This allows users to use their electronic devices with peace of mind and receive prompt and accurate support when a problem occurs.

[2170] The processing flow will be explained below.

[2171] Building the user interface

[2172] Step 1:

[2173] A user opens a web browser or mobile app and accesses the system.

[2174] Step 2:

[2175] The terminal sends the user's login information to the server for authentication.

[2176] Step 3:

[2177] The server verifies the user's credentials and starts a session to allow access to the dashboard.

[2178] Step 4:

[2179] The device displays a dashboard that provides a visual list of the devices the user has registered and their respective status information.

[2180] Input and analysis of equipment information

[2181] Step 1:

[2182] When a user purchases or installs new equipment, they enter the equipment information into an input form within the system.

[2183] Step 2:

[2184] The device sends the entered device information (device name, manufacturer, model, serial number, purchase date, warranty period, etc.) to the server.

[2185] Step 3:

[2186] The server analyzes the received device information and stores it in a database in an appropriate format.

[2187] Step 4:

[2188] The server checks the saved device information and notifies the user that it has been correctly stored in the database.

[2189] Device information analysis and problem solving

[2190] Step 1:

[2191] Users submit problems or questions about specific devices by filling out a report form within the system.

[2192] Step 2:

[2193] The terminal transmits the user's report to the server.

[2194] Step 3:

[2195] The server analyzes the report received using natural language processing (NLP) technology to identify the type of problem.

[2196] Step 4:

[2197] The server searches a database of past case studies for similar problems and their solutions.

[2198] Step 5:

[2199] The server will extract the optimal solution and provide advice to the user, for example, "If your Internet connection is unstable, please reboot your router."

[2200] Bug debugging support

[2201] Step 1:

[2202] The user utilizes the debug reporting feature within the system to enter details of the problem they are experiencing and report it.

[2203] Step 2:

[2204] The terminal transmits the user's report to the server.

[2205] Step 3:

[2206] The server generates a basic diagnostic flow and presents it to the user, for example, showing a procedure such as "Check if the power cord is properly connected."

[2207] Step 4:

[2208] If the user performs an initial diagnosis and the problem persists, they enter additional details into a form within the system.

[2209] Step 5:

[2210] The terminal sends the detailed information back to the server.

[2211] Step 6:

[2212] The server then performs deep analysis of the details received and generates more advanced diagnostic procedures, such as displaying advice such as "The internal filter may be clogged. Please clean the filter."

[2213] Expired maintenance / replacement support

[2214] Step 1:

[2215] The server periodically scans the user's device list and checks the maintenance expiration date for each device.

[2216] Step 2:

[2217] The server detects devices that are close to expiring for maintenance and generates a notification message.

[2218] Step 3:

[2219] The device displays a notification message to the user, for example, "Your washing machine's maintenance period expires at the end of next month."

[2220] Step 4:

[2221] The server searches for new device candidates suitable for the user based on market information.

[2222] Step 5:

[2223] The device displays a recommendation to the user saying, "Your washing machine's maintenance deadline is approaching, so consider purchasing a new washing machine (manufacturer name) (model name)."

[2224] Example 1

[2225] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2226] Management of electronic devices in the home is generally done on a per-device basis, and when a problem occurs, the response varies from device to device, which can be time-consuming and difficult for users to understand. Furthermore, managing maintenance deadlines and replacement times manually can easily lead to errors, and there is a risk that the device will not last the full lifespan of the device. A system is needed to centrally manage these issues, reduce the burden on users, and enable rapid problem resolution.

[2227] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2228] In this invention, the server includes means for a user to input device information, means for saving the input device information in a database, means for analyzing the saved device information and providing advice for solving problems, means for periodically checking device information and notifying the user of maintenance deadlines and recommended replacement times, means for a user to report problems, means for analyzing reported problems and providing appropriate solutions, means for a user to report malfunctions, means for providing initial and detailed diagnoses, and means for displaying detailed advice. This allows for centralized management of electronic devices in the home, enabling confirmation of maintenance deadlines, notifications of recommended replacements, and rapid problem resolution.

[2229] A "user" is a person who uses this system to manage information about devices in the home.

[2230] "Device information" is detailed information about electronic devices in the home, including, for example, the device name, manufacturer, model, serial number, purchase date, warranty period, and so on.

[2231] "Input means" refers to the interface through which users input device information and problem details into the system, and includes web browsers and mobile apps.

[2232] A "database" is a collection of data managed by a server, and stores registered device information and data for problem solving.

[2233] "Analysis means" is a function in which the server analyzes device information and reports from users, and performs processing to provide appropriate advice and solutions.

[2234] The "advice providing means" is a function that provides the user with the solution obtained by the analysis means, and displays specific operating methods and countermeasures.

[2235] "Maintenance expiration date" means the end of the equipment warranty or maintenance contract period.

[2236] The "notification means" is a function that notifies the user of devices whose maintenance deadlines are approaching or when replacement is recommended.

[2237] The "recommendation means" is a function that suggests suitable new device candidates to the user, for example, by displaying recommended devices based on the latest market information.

[2238] The "problem reporting means" is a function for a user to report a device problem they are experiencing to the system, and includes an input form and a feedback function.

[2239] The "initial diagnosis means" is a function for performing basic diagnosis of a problem reported by the problem reporting means.

[2240] The "detailed diagnosis means" is a function for collecting more detailed information and conducting in-depth analysis of problems that could not be resolved through initial diagnosis.

[2241] "Advice display means" is a function that displays specific advice or solutions to the user. For example, it may display on the terminal screen, "Please turn the router off and on again, and connect the LAN cable correctly."

[2242] A specific embodiment for carrying out this invention will be described in detail below. This system allows users to manage information about electronic devices in their homes and receive support for solving problems. The system is designed to allow users to register and manage device information, and report and diagnose problems using a web browser or mobile app.

[2243] Building the user interface

[2244] Access and Dashboard Viewing

[2245] A user accesses the system using a web browser or mobile app. The device sends the authentication information entered by the user to the server, which then authenticates the user based on that information. Once authentication is complete, the server obtains the user's device information and displays it on the device in dashboard format. For example, it visually provides information such as devices currently in operation, devices nearing the end of their maintenance period, and devices nearing the recommended replacement date.

[2246] Input and analysis of equipment information

[2247] Registering device information

[2248] Users register information about newly purchased or existing devices through an input form, including details such as device name, manufacturer, model, serial number, purchase date, warranty period, etc. The device sends this information to the server, which stores it in a database in the appropriate format.

[2249] Saving to a database

[2250] The server analyzes the device information received from the terminal and stores it in a database. For example, the server normalizes the device information and stores it in a database to enable efficient searches. This process uses a database management system (DBMS) or similar.

[2251] Device information analysis and problem solving

[2252] Problem reporting and analysis

[2253] Users report problems or questions about a specific device by filling out a form within the system. For example, if a user reports that their TV cannot connect to the Internet, the server analyzes the report using natural language processing (NLP) techniques, such as the Google NLP API and other generative AI models.

[2254] Providing advice

[2255] The server searches for similar problems from past case studies in a database and extracts the optimal solution. For example, it displays a question on the device, such as "Are the lights on your router lit normally?", and provides specific advice based on the user's answer. The server generates advice such as "If your network connection is unstable, please turn the router off and on again and properly connect the LAN cable," and the advice is displayed on the device.

[2256] Bug debugging support

[2257] Reporting a Bug

[2258] A user reports a problem in detail using a specific function within the system. For example, if a user reports that their washing machine has stopped working, the details of the report are collected.

[2259] Initial diagnosis

[2260] The server analyzes the problem and provides a basic diagnostic flow, such as prompting the user to "Make sure the power cord is properly connected."

[2261] Performing detailed diagnostics

[2262] If the initial diagnosis doesn't solve the problem, the device collects additional details from the user and sends them to the server, which then performs a more detailed diagnosis and displays advanced advice, such as "The filter may be clogged, so please clean it."

[2263] Expired maintenance / replacement support

[2264] Check maintenance expiration date

[2265] The server periodically scans the user's device list and checks the maintenance expiration date for each device. For example, it detects information such as "The maintenance expiration date for the washing machine will expire in one month."

[2266] Generate and send notifications

[2267] The server detects approaching maintenance deadlines and generates notifications such as, "Your washing machine's maintenance deadline will expire at the end of next month." The generated notifications are displayed on the device.

[2268] Providing recommendations

[2269] The server searches for new appliance candidates suitable for the user based on market information. For example, it displays a recommendation message on the device saying, "A new washing machine from (manufacturer name) (model name) is recommended."

[2270] Specific examples

[2271] Below are some examples of prompt sentences.

[2272] Example prompt 1:

[2273] "Please explain the programming process for a home electronics information management system."

[2274] Example prompt 2:

[2275] "Could you please explain in detail what the process is when a user reports an issue?"

[2276] Example prompt 3:

[2277] "Please give specific examples of support for replacement of electronic devices in the home when their maintenance expires."

[2278] As described above, this system realizes the unification of management, problem solving, and maintenance of electronic devices used in the home, thereby greatly improving user convenience.

[2279] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2280] Step 1:

[2281] Access and Authentication

[2282] A user accesses the system using a web browser or mobile app. The device enters the user's authentication information (username and password) and sends it to the server. The server checks the authentication information against a database and sends the authentication result back to the device.

[2283] Input: Username, Password

[2284] Data processing: Authentication verification

[2285] Output: Authentication success / failure message

[2286] Step 2:

[2287] Viewing the Dashboard

[2288] If authentication is successful, the device displays the dashboard. The server retrieves the user's device information from the database and sends it to the device. The device displays the device information received in a list on the dashboard.

[2289] Input: User ID

[2290] Data processing: Acquisition of device information

[2291] Output: Dashboard screen

[2292] Step 3:

[2293] Registering device information

[2294] The user registers information about a newly purchased or existing device using an input form. The device enters the device information (device name, manufacturer, model, serial number, purchase date, warranty period, etc.) and sends it to the server. The server saves the information in a database.

[2295] Input: Device information (device name, manufacturer, model, serial number, purchase date, warranty period, etc.)

[2296] Data processing: Saving device information

[2297] Output: Registration complete message

[2298] Step 4:

[2299] Reporting an issue

[2300] Users fill out a form about a specific device issue and send it from their device to a server, which then analyzes the report using natural language processing (NLP) techniques.

[2301] Input: Problem report

[2302] Data processing: Problem analysis using NLP

[2303] Output: Problem analysis results

[2304] Step 5:

[2305] Providing advice

[2306] The server searches for similar problems from past case studies in a database and extracts appropriate solutions. The device displays the received solutions to the user. For example, for the problem "My TV can't connect to the Internet," the device displays the advice "Power cycle my router."

[2307] Input: Problem analysis results

[2308] Data processing: Finding case studies and extracting solutions

[2309] Output: Show solution

[2310] Step 6:

[2311] Reporting a Bug

[2312] A user reports a problem using a specific function within the system, and the report is sent from the terminal to the server.

[2313] Input: Bug report

[2314] Data processing: Saving report contents

[2315] Output: Report confirmation message

[2316] Step 7:

[2317] Providing an initial diagnosis

[2318] The server provides the user with a basic diagnostic flow. The terminal displays the diagnostic procedure received from the server to the user. For example, it displays an instruction such as "Please check if the power cord is connected properly."

[2319] Input: Bug report

[2320] Data processing: Basic diagnostic flow generation

[2321] Output: Display of diagnostic steps

[2322] Step 8:

[2323] Performing detailed diagnostics

[2324] If the user is unable to resolve the issue through initial diagnosis, the device will collect additional details from the user and send them to the server, which will then perform further analysis and provide advanced diagnostic procedures, such as displaying advice such as "Please clean the internal filter."

[2325] Input: More information

[2326] Data processing: Detailed diagnostic analysis

[2327] Output: Display of advanced diagnostic procedures

[2328] Step 9:

[2329] Check maintenance expiration date

[2330] The server periodically scans the database to check the maintenance deadlines of each device, detects devices that are close to the end of their maintenance period, and generates a notification to the user.

[2331] Input: Device information

[2332] Data processing: Check maintenance period

[2333] Output: Maintenance expiration notice

[2334] Step 10:

[2335] Providing recommendations

[2336] The server searches for new appliance candidates suitable for the user based on market information. The device displays a recommendation message to the user. For example, it displays a message saying, "We recommend a new washing machine from (manufacturer name) (model name)."

[2337] Input: Maintenance expiration information

[2338] Data processing: recommendation generation

[2339] Output: Recommendation notification

[2340] Through these steps, this system can centrally manage all electronic devices in the home, and provide quick problem resolution and appropriate maintenance support.

[2341] (Application example 1)

[2342] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2343] Conventional electronic device management systems for homes and brick-and-mortar stores have the problem of taking time and effort to monitor the status of devices and manage maintenance deadlines. Furthermore, it is difficult to quickly address problems when they occur, making rapid response particularly important in brick-and-mortar stores.

[2344] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2345] In this invention, the server includes: a means for a user to input device information; a means for saving the input device information in a database; a means for analyzing the saved device information and providing advice for resolving problems; a means for periodically checking device information and notifying users of maintenance deadlines and recommended replacement times; a means for providing notifications of maintenance deadlines and recommended replacement times using a visual display device; and a means for reporting device malfunctions and providing rapid problem resolution in real time via the visual display device. This makes it possible to monitor the status of electronic devices in real time and notify users of appropriate timing for maintenance or replacement. Furthermore, rapid problem resolution enables significant efficiency improvements in electronic device management, especially in physical stores.

[2346] The "means for users to input device information" is an interface that allows users to register information about newly purchased devices and existing devices in the system.

[2347] The "means for saving input device information in a database" is a system function that saves device information entered by a user in a database in an appropriate format.

[2348] The "means for analyzing stored device information" is a function for analyzing device information in the database and understanding the device status, maintenance deadlines, etc.

[2349] "Means for providing advice for solving problems" is a system function that presents appropriate solutions to users who report equipment malfunctions or problems.

[2350] "Means for periodically checking device information" refers to a function in which the server periodically scans the device information in the database to check things like maintenance deadlines and recommended replacement times.

[2351] The "means for notifying the user of the expiration date of maintenance or the recommended time for replacement" is a function for notifying the user of devices whose maintenance deadline is approaching or whose replacement is recommended.

[2352] "Means for providing notifications of maintenance deadlines and recommended replacement times using a visual display device" refers to a function that uses a visual display device (e.g., smart glasses, head-mounted display) to visually notify the user of maintenance deadlines and recommended replacement times.

[2353] "Means for reporting equipment malfunctions and quickly resolving problems in real time through a visual display device" is a function that analyzes equipment malfunctions reported by users in real time through a visual display device and provides quick solutions.

[2354] A "server" is a central computer that manages and analyzes data for the entire system.

[2355] The following describes in detail an embodiment of the present invention. This system allows users to manage electronic device information in a store and receive support for problem-solving. The users are primarily store clerks who use smart glasses or a head-mounted display to manage the devices.

[2356] Building the user interface

[2357] 1. Access and Dashboard View

[2358] Store associates access the system using smart glasses or a head-mounted display.

[2359] The device displays a list of the user's registered devices on a dashboard, along with visual status information for each device, such as which devices are currently in operation, which devices are nearing the end of their maintenance period, and which devices are nearing the recommended replacement date.

[2360] Input and analysis of equipment information

[2361] 1. Registering device information

[2362] The store clerk registers information about newly purchased or existing devices using an input form in the system, entering details such as the device name, manufacturer, model, serial number, purchase date, and warranty period.

[2363] 2. Saving to the database

[2364] The device sends the entered device information to the server, which analyzes the received information and stores it in a database in an appropriate format.

[2365] Device information analysis and problem solving

[2366] 1. Reporting and analyzing problems

[2367] A store clerk reports a problem or question about a specific device by filling out a form in the system. The server scans the report and analyzes it based on the type of problem. It then uses natural language processing (NLP) technology to understand the details of the problem.

[2368] 2. Providing advice

[2369] The server searches for similar problems from past case studies in a database and extracts the optimal solution. The device displays the solution to the user, providing specific advice such as "If your network connection is unstable, please restart your router."

[2370] Bug debugging support

[2371] 1. Reporting a Bug

[2372] The store associate uses a specific feature within the system to report in detail the problem they are experiencing.

[2373] 2. Initial diagnosis

[2374] The server provides a basic diagnostic flow to the user, for example, by providing instructions such as "Make sure the power cord is properly connected."

[2375] 3. Perform detailed diagnostics

[2376] If the store attendant is unable to resolve the problem through initial diagnosis, the terminal collects additional details from the store attendant and transmits them to the server.

[2377] The server then analyzes the details received and suggests more advanced diagnostic steps, such as "The internal filter may be clogged. Please clean the filter."

[2378] Expired maintenance / replacement support

[2379] 1. Check the maintenance period

[2380] The server periodically scans the user's device list and checks the maintenance deadline for each device.

[2381] 2. Generate and send notifications

[2382] The server detects devices whose maintenance period is approaching and generates a notification to the user. Specifically, it generates a notification saying, "The maintenance period for your washing machine will expire at the end of next month," and displays it on the device.

[2383] 3. Providing recommendations

[2384] The server searches for suitable new appliances for the user based on market information, and the device displays a recommendation such as, "Your current washing machine's maintenance deadline is approaching, so consider (manufacturer name) (model name) as your new washing machine."

[2385] Specific examples

[2386] Example 1: When a wiring problem occurs

[2387] 1. Report a problem

[2388] Store Clerk: Report "TV won't connect to the internet."

[2389] 2. Analysis and Advice

[2390] The device will ask the user the question, "Are the lights on your router working properly?"

[2391] Store Clerk: "The green light is not on."

[2392] The server will analyze the problem and provide specific advice such as "Turn the router off and on again and connect the LAN cable correctly."

[2393] Examples of prompt statements

[2394] Shop Electronics Problem Report: The register display is not working. Please identify the cause and provide a solution.

[2395] This system ensures thorough management of electronic devices in stores, ensuring adherence to maintenance deadlines and rapid troubleshooting. Real-time status monitoring and information provided through visual display devices dramatically improves user convenience.

[2396] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2397] Step 1:

[2398] The user enters the device information

[2399] Input: The sales associate enters details of the newly purchased or existing device (device name, manufacturer, model, serial number, purchase date, warranty period) into an input form.

[2400] How it works: This information is sent to the system via smart glasses or a head-mounted display.

[2401] Output: The entered data is sent to the system.

[2402] Step 2:

[2403] Save the entered device information in the database

[2404] Input: The device details obtained in step 1.

[2405] How it works: The server parses the information it receives and stores it in a database (such as SQLite) in an appropriate format.

[2406] Output: New device information is added to the database.

[2407] Step 3:

[2408] Dashboard View

[2409] Input: Equipment information stored in the database.

[2410] How it works: The device (smart glasses or head-mounted display) displays a list of device information on a dashboard, providing visual information on the status of each device (operating, approaching maintenance deadline, recommended replacement, etc.).

[2411] Output: Store staff can check the status of the equipment.

[2412] Step 4:

[2413] Problem reporting and analysis

[2414] Input: The store clerk fills out a problem report form to describe the equipment malfunction or problem.

[2415] How it works: The server analyzes the report using natural language processing (NLP) to understand the details of the problem. It uses a generative AI model to extract the type and details of the problem.

[2416] Output: Provides advice on next steps based on the extracted problem details.

[2417] Step 5:

[2418] Providing advice

[2419] Input: Detailed information about the problem analyzed in step 4.

[2420] How it works: The server searches a database of past case studies for similar problems and extracts the best solution. The generated advice is then displayed to the salesperson on the terminal.

[2421] Output: The store clerk receives specific advice, for example, "If your network connection is unstable, please power cycle your router."

[2422] Step 6:

[2423] Check maintenance expiration date

[2424] Input: Equipment information stored in the database.

[2425] How it works: The server periodically scans device information and checks the maintenance expiration date and usage period of each device.

[2426] Output: A list of devices due for maintenance soon is generated.

[2427] Step 7:

[2428] Notification of maintenance deadlines and recommended replacement times

[2429] Input: The list of equipment due for maintenance soon, generated in step 6.

[2430] How it works: The server generates notifications for devices that are due for maintenance soon, and the device (smart glasses or head-mounted display) visually displays them to the store clerk.

[2431] Output: The store clerk sees a notification that says, "Your washing machine's maintenance deadline will expire at the end of next month."

[2432] Step 8:

[2433] Providing recommendations

[2434] Input: New device candidates generated by the server based on market information.

[2435] How it works: The server generates a recommendation that reads, "Your current washing machine's maintenance deadline is approaching, so please consider purchasing a new washing machine (manufacturer name) (model name)," and displays it to the store clerk on the terminal.

[2436] Output: The store clerk reviews the new equipment candidates.

[2437] Step 9:

[2438] Detailed diagnosis of the problem

[2439] Input: Detailed information if the store attendant was unable to resolve the issue with initial diagnosis.

[2440] How it works: The device collects additional details from the store attendant and sends them to the server, which analyzes them and suggests more advanced diagnostic steps, such as "The internal filter may be clogged. Please clean the filter."

[2441] Output: The store associate receives detailed diagnostic results and instructions.

[2442] In this way, the entire system can efficiently support a series of processes from registering device information to problem solving, maintenance management, and replacement recommendations.

[2443] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2444] The following describes in detail the mode for carryin...

Claims

1. A means for a user to input device information; A means for storing the input device information in a database; A means for analyzing the stored device information and providing advice for solving problems; A system that periodically checks equipment information and notifies users of maintenance deadlines and recommended replacement times.

2. 2. The system of claim 1, wherein the means for providing advice for solving a problem searches for and provides a solution based on past case studies.

3. 2. The system according to claim 1, which notifies the user of the expiration date of equipment maintenance or the recommended time for replacement, and recommends suitable new equipment candidates.

Images