System

Abstract

To provide a system.SOLUTION: A system includes: registration means of registering device information including a maker name and model number of electric home appliances or communication instruments a user uses and a usage situation; means of suggesting information on at least one of improvement measures against problems of the electric home appliance or communication instrument, a maintenance method of the electric home appliance or communication instrument, a purchase method of accessories of the electric home appliance or communication instrument, a replacement time of the electric home appliance or communication instrument, a successor of the electric home appliance or communication instrument, a useful usage of the electric home appliance or communication instrument, a disposal method of the electric home appliance or communication instrument, and a recycle method of the electric home appliance or communication instrument, on the basis of the registered device information by inputting a specific prompt sentence to a generative AI model; and adjustment means of adjusting a content of the information to be suggested, an expression thereof, according to an emotion engine that recognizes an emotion of the user and the emotion.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, to obtain information on troubleshooting, maintenance methods, how to purchase accessories, when to replace / replace, convenient ways to use, disposal methods, recycling methods, etc. for devices such as home appliances and communication equipment, users must individually research the instruction manuals and manufacturer websites for each device, which takes time and effort.In addition, there is no system that provides optimal information based on the user's usage situation and environment. [Means for solving the problem]

[0005] As a means for solving this problem, the system of the present invention includes: a registration means for registering device information including the manufacturer name, model number, and usage status of the home appliance or communication device used by the user; a means for inputting a specific prompt sentence into a generative AI model, based on the registered device information, suggesting at least one of the following information: a solution to a problem with the home appliance or communication device; a maintenance method for the home appliance or communication device; a method for purchasing accessories for the home appliance or communication device; a replacement time for the home appliance or communication device; a successor product for the home appliance or communication device; convenient ways to use the home appliance or communication device; a method for disposing of the home appliance or communication device; and a method for recycling the home appliance or communication device; an emotion engine that recognizes the user's emotions; and an adjustment means for adjusting the content and expression of the suggested information according to the emotion. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9]1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44.

[0019] The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0026] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0027] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0028] "Example 1"

[0029] The system of the present invention includes a means for users to register device information for home appliances, communication devices, etc. currently in use, and a means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. based on the registered device information. Specifically, users register information about the devices they are using using a dedicated application or website. The information to be registered includes the device type, manufacturer, model name, purchase date, usage status, etc.

[0030] "Example 2"

[0031] Based on the registered device information, the system retrieves information corresponding to each device from the database and provides guidance and suggestions to the user. For example, in the case of a refrigerator, it provides information such as how to clean it regularly, how to deal with error codes, how to purchase accessories, when to replace it, how to use it energy efficiently, and how to recycle it when discarding it.

[0032] "Example 3"

[0033] Furthermore, the system provides information optimized for the user's usage situation and environment. For example, if a user inquires about the temperature setting of an air conditioner, the system will suggest the optimal temperature setting taking into account the climate of the user's residence, the type of air conditioner, the user's preferences, etc.

[0034] The processing flow of each embodiment will be described below.

[0035] "Example 1"

[0036] Step 1: The user registers information about the device they are using on a dedicated application or website, including the device type, manufacturer, model name, purchase date, and usage status.

[0037] Step 2: The system retrieves the relevant device information from the database based on the registered information.

[0038] Step 3: Based on the acquired information, the system will provide the user with information and suggestions on troubleshooting, maintenance methods, how to purchase accessories, when to replace / replace the product, convenient usage, disposal methods, recycling methods, etc.

[0039] "Example 2"

[0040] Step 1: When a user makes an inquiry about a specific device, the system provides the most appropriate information based on the device information and the user's usage and environment.

[0041] Step 2: For example, if a user inquires about the temperature setting of an air conditioner, the system will suggest the optimal temperature setting taking into consideration the climate of the user's place of residence, the type of air conditioner, the user's preferences, etc.

[0042] Example 1

[0043] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0044] Modern users use a wide variety of electronic devices and communication equipment, and they must individually look up information about each device, such as troubleshooting, maintenance methods, how to purchase accessories, when to replace them, convenient ways to use them, how to dispose of them, and how to recycle them. This requires a lot of time and effort from users, making efficient device management difficult. Furthermore, if the information provided is not optimized for the user's usage situation or environment, it may not be possible to respond appropriately. Furthermore, the lack of an interactive interface makes it difficult to update or adjust the information according to user input.

[0045] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0046] In this invention, the server includes means for registering device information of electronic devices, communication devices, etc. currently in use by the user, means for storing the registered device information in a database, means for sending prompts to the generative AI model based on the registered device information, means for receiving responses from the generative AI model and providing them to the user, and means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. based on the registered device information. This allows users to centrally manage device information and quickly obtain optimized information. Furthermore, an interactive interface allows users to easily update and adjust information based on user input.

[0047] A "user" is an individual or corporation that uses the system to register information about electronic devices or communication devices and receives guidance and suggestions.

[0048] "Electronic devices" is a general term for devices that operate using electricity, such as household appliances, computers, and smartphones.

[0049] "Communication equipment" is a general term for devices used to send and receive information over the Internet or telephone lines.

[0050] "Device information" refers to detailed information about the electronic device or communication device currently being used by the user, such as the type, manufacturer, model name, purchase date, and usage status.

[0051] A "database" is an information system for efficiently storing, managing, and searching registered device information.

[0052] A "generative AI model" is an artificial intelligence algorithm that generates appropriate answers or suggestions based on user input.

[0053] A "prompt sentence" is an input sentence that instructs a generative AI model to generate specific information.

[0054] "Troubleshooting" is the process of diagnosing equipment problems or malfunctions and providing solutions.

[0055] "Maintenance method" refers to the method of regular inspection and care to keep the equipment in good condition.

[0056] "Accessory Purchasing Method" refers to procedures and information for purchasing accessories and parts related to the device.

[0057] "Replacement timing / successor product" is information on the recommended time to replace the currently used device and the new model that will be its successor.

[0058] "Helpfuls" are tips and tricks for using equipment more efficiently and effectively.

[0059] "Disposal method" means the procedure or method for properly disposing of used equipment.

[0060] A "recycling method" is a procedure or method for disposing of used equipment in a reusable form.

[0061] An "interactive interface" is an operating screen or function that allows users to input and obtain information while interacting with the system.

[0062] This invention is a system that allows users to register information about electronic devices and communication devices they are using, and based on that information, provides guidance and suggestions on troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. This system operates in cooperation with the user, terminal, and server.

[0063] Users use a dedicated application or website to enter information about the device they are using. The information to be entered includes the type of device, manufacturer, model name, purchase date, and usage status. For example, when a user registers a "television," they enter "electronic device manufacturer" in "manufacturer," "Model X" in "model name," "January 2022" in "purchase date," and "daily use" in "usage status."

[0064] The device (user's smartphone or PC) sends the entered device information to the server. At this time, the information is encrypted before being sent. The server stores the received device information in a database. The database uses a relational database such as MySQL (registered trademark) or PostgreSQL.

[0065] The server analyzes the registered device information and sends the necessary information to the generative AI model as a prompt. For example, it generates a prompt such as "Please tell me how to troubleshoot Model X." The generative AI model (e.g., OpenAI's GPT-4) receives the prompt and generates an appropriate answer. The generated answer is sent back to the server.

[0066] The server provides the answer received from the generative AI model to the user, who can then view the answer in an application or on a website. For example, if a user registers a "Model X" television, the server provides the following information:

[0067] Troubleshooting: What to do if the screen is blank

[0068] Maintenance method: How to clean the screen

[0069] How to purchase accessories: How to purchase a remote control

[0070] Replacement time / successor: Information on new models

[0071] Useful tips: How to set up voice control

[0072] Disposal Instructions: How to properly dispose of a television

[0073] How to recycle: Recycling center information

[0074] Example prompts to input to a generative AI model:

[0075] A user has registered a "Model X" TV. Please provide information on troubleshooting, maintenance, purchasing accessories, replacement timing / successor products, convenient usage, disposal, and recycling for this TV.

[0076] This system allows users to centrally manage equipment information, quickly obtain optimized information, and easily update and adjust information based on user input through an interactive interface.

[0077] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0078] Step 1:

[0079] The user enters the device information

[0080] Users access a dedicated application or website and enter information about the device they are using. The information entered includes the type of device, manufacturer, model name, purchase date, and usage status. For example, when a user registers a "television," they enter "electronic device manufacturer" in "manufacturer," "Model X" in "model name," "January 2022" in "purchase date," and "daily use" in "usage status." The entered information is saved on the device.

[0081] Step 2:

[0082] The device sends the entered information to the server.

[0083] The device (user's smartphone or PC) encrypts the entered device information and sends it to the server. The entered data includes the device type, manufacturer, model name, purchase date, usage status, etc. The server receives this data and stores it in a database.

[0084] Step 3:

[0085] The server stores the device information in a database

[0086] The server analyzes the received device information and stores it in a database using a relational database such as MySQL or PostgreSQL. The stored data includes the device type, manufacturer, model name, purchase date, usage status, etc.

[0087] Step 4:

[0088] The server sends a prompt to the generative AI model based on the device information.

[0089] The server analyzes the device information stored in the database and sends the necessary information to the generative AI model as a prompt. For example, it generates a prompt such as "Please tell me how to troubleshoot Model X." The input data is the device information, and the output data is the prompt.

[0090] Step 5:

[0091] The generative AI model generates an answer and sends it back to the server

[0092] A generative AI model (e.g., OpenAI's GPT-4) receives a prompt and generates an appropriate answer. The generated answer is sent back to the server. The input data is the prompt, and the output data is the generated answer.

[0093] Step 6:

[0094] The server provides the generated answer to the user.

[0095] The server displays the answers it receives from the generative AI model in the user's application, where the user can review the answers in the application or on a website. For example, if a user registers a "Model X" television, the server provides the following information:

[0096] Troubleshooting: What to do if the screen is blank

[0097] Maintenance method: How to clean the screen

[0098] How to purchase accessories: How to purchase a remote control

[0099] Replacement time / successor: Information on new models

[0100] Useful tips: How to set up voice control

[0101] Disposal Instructions: How to properly dispose of a television

[0102] How to recycle: Recycling center information

[0103] The input data is the answer that is generated, and the output data is the information that is presented to the user.

[0104] (Application example 1)

[0105] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0106] Conventional systems make it difficult for users to centrally manage information about the equipment they are using and provide comprehensive guidance and suggestions on troubleshooting, maintenance methods, parts purchasing methods, replacement timing, convenient usage, disposal methods, recycling methods, etc. Furthermore, particularly for robots used in factories, there was a lack of means to properly manage and efficiently provide this information. This prevented users from performing appropriate maintenance and troubleshooting, resulting in problems such as a shortened equipment lifespan and reduced efficiency.

[0107] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0108] In this invention, the server includes means for registering information about devices currently in use by users, means for providing and suggesting troubleshooting, maintenance methods, parts purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. based on the registered device information, and means for registering information about robots used in factories and providing and suggesting troubleshooting, maintenance methods, parts purchasing methods, replacement timing, convenient usage, disposal methods, and recycling methods. This allows users to centrally manage device information and perform appropriate maintenance and troubleshooting.

[0109] "Device information" is detailed data about the device currently being used by the user, such as the type, manufacturer, model name, purchase date, and usage status.

[0110] "Troubleshooting" refers to the procedures and methods for diagnosing and resolving problems or malfunctions that occur in equipment.

[0111] "Maintenance methods" are the maintenance and inspection procedures required to maintain the normal operation of equipment.

[0112] "Parts purchasing method" refers to the procedures and methods for purchasing parts required for repairing or maintaining equipment.

[0113] "Time to replace" refers to a suitable time to replace the currently used device with a new device.

[0114] A "successor product" is a next-generation model or improved version of the equipment currently in use.

[0115] "Convenient ways of using" are methods and tips for using equipment more efficiently and effectively.

[0116] "Disposal method" refers to the procedures and methods for properly disposing of used equipment.

[0117] "Recycling method" refers to the procedures and methods for disposing of used equipment in a reusable form.

[0118] "Robots used in factories" are automated machinery used on factory production lines and in workshops.

[0119] The system for implementing this invention allows users to register information about devices in use, and based on that information, provides guidance and suggestions on troubleshooting, maintenance methods, parts purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. In particular, it allows for efficient management of information about robots used in factories, enabling appropriate maintenance and troubleshooting.

[0120] System configuration

[0121] 1. Hardware and Software Use

[0122] Hardware: Smartphone (iOS or ANDROID (registered trademark))

[0123] Software: Mobile application development framework (React Native), database (Firebase), generative AI model (OpenAI GPT-4)

[0124] Program processing explanation

[0125] 1. Data registration

[0126] Users launch the app using their smartphone and enter information about the robots used in the factory (type, manufacturer, model name, purchase date, usage status, etc.).

[0127] The entered data is stored in the Firebase database.

[0128] 2. Troubleshooting and maintenance suggestions

[0129] Users input a specific problem, and a generative AI model (GPT-4) suggests ways to troubleshoot it.

[0130] Suggestions on how to maintain and purchase parts are also provided.

[0131] 3. Suggestions for replacement timing and successor products

[0132] Based on the usage status and purchase date of registered equipment, suggestions will be made on when to replace or which successor product to purchase.

[0133] 4. Information on convenient use, disposal and recycling methods

[0134] Using a generative AI model, it guides users on convenient usage, disposal, and recycling methods.

[0135] Specific examples

[0136] For example, if a user is using a robotic arm model called "XYZ-1000," the following prompt sentence can be input into the generative AI model:

[0137] Prompt Sentence Examples

[0138] Please provide information about the robotic arm "XYZ-1000" you are using. Please include the following information:

[0139] 1. Troubleshooting Methodology

[0140] 2. Maintenance method

[0141] 3. How to purchase parts

[0142] 4. Suggestions for replacement timing and successor products

[0143] 5. Convenient usage

[0144] 6. Disposal Method

[0145] 7. Recycling methods

[0146] By inputting this prompt sentence into the generative AI model, appropriate information can be provided to the user, allowing the user to centrally manage device information and perform appropriate maintenance and troubleshooting.

[0147] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0148] Step 1:

[0149] The user launches the app using a smartphone and enters information about the robots used in the factory (type, manufacturer, model name, purchase date, usage status, etc.).

[0150] Input: Information such as robot type, manufacturer, model name, purchase date, and usage status.

[0151] Output: The entered robot information is saved in the application.

[0152] What happens: When a user enters the required information into the app's input form and presses the "Save" button, the information is sent to the Firebase database.

[0153] Step 2:

[0154] The server receives the robot information stored in the Firebase database and stores it in the database.

[0155] Input: The robot information entered by the user.

[0156] Output: Robot information is saved to the Firebase database.

[0157] Specific operation: The server receives data through the Firebase API and stores it in the database.

[0158] Step 3:

[0159] The user enters a specific problem into the app and requests troubleshooting.

[0160] Input: Details of the specific problem with your robot.

[0161] Output: Suggestions on how to troubleshoot the problem.

[0162] What it does: The user enters details about the problem and presses the "Troubleshoot" button, which sends the information to the server.

[0163] Step 4:

[0164] The server inputs details of the problem into the generative AI model (GPT-4) as a prompt sentence and generates a troubleshooting method.

[0165] Input: A prompt statement containing details about the problem.

[0166] Output: Suggested troubleshooting methods.

[0167] Specific operation: The server sends a prompt to the generative AI model, and the AI ​​generates an appropriate troubleshooting method.

[0168] Step 5:

[0169] The server sends the generated troubleshooting instructions to the user's smartphone.

[0170] Input: Troubleshooting methods from generative AI models.

[0171] Output: Troubleshooting instructions displayed on the user's phone.

[0172] What it does: The server generates troubleshooting instructions and sends them to the user's smartphone, where the app displays them.

[0173] Step 6:

[0174] When a user requests information about maintenance methods or parts purchasing methods, the user inputs the information in the same manner and transmits it to the server.

[0175] Input: Requests for maintenance methods and parts purchasing methods.

[0176] Output: Suggestions on how to maintain and purchase parts.

[0177] Specific operation: The user inputs a request, and the server sends a prompt to the generative AI model to generate information and send it to the user.

[0178] Step 7:

[0179] The server will suggest replacement times and successor products based on the usage status and purchase date of the registered equipment.

[0180] Input: Equipment usage status and purchase date.

[0181] Output: Suggestions for replacement time and successor products.

[0182] Specific operation: The server retrieves usage status and purchase date from the database, sends prompts to the generative AI model to generate suggestions, and sends them to the user.

[0183] Step 8:

[0184] When the user requests convenient usage, disposal, or recycling methods, the information is entered in the same manner and transmitted to the server.

[0185] Input: Requests for convenient use, disposal and recycling methods.

[0186] Output: Suggestions for convenient use, disposal and recycling.

[0187] Specific operation: The user inputs a request, and the server sends a prompt to the generative AI model to generate information and send it to the user.

[0188] Example 2

[0189] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0190] Conventional device management systems make it difficult for users to efficiently obtain information about the home appliances and communication devices they are using and receive appropriate guidance and suggestions. Furthermore, they lack the ability to provide optimal information tailored to the user's usage situation and environment, and the ability to update and adjust information through an interactive interface is limited. This often results in inconvenience for users when it comes to device maintenance and troubleshooting, purchasing accessories, and determining when to replace devices.

[0191] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0192] In this invention, the server includes means for registering device information for home appliances, communication devices, and other devices currently in use by the user, means for retrieving relevant information from a database based on the registered device information, means for sending a prompt message to the generative AI model based on the retrieved information, means for the generative AI model to generate a guidance message, means for transmitting the generated guidance message to the user's device, and means for the user to check the guidance message on the device. This allows the user to efficiently obtain detailed information about the device and receive appropriate guidance and suggestions. Furthermore, optimal information is provided based on the user's usage status and environment, and the information can be updated and adjusted through an interactive interface.

[0193] A "user" is an individual or corporation that uses the system to register device information such as home appliances and communication equipment and receives guidance and suggestions.

[0194] "Device information" refers to detailed information about devices such as home appliances and communication equipment, such as manufacturer names, model numbers, and usage status.

[0195] The "server" is a computer system that receives device information sent by the user, queries a database, uses a generative AI model to generate guidance messages, and sends them to the user's device.

[0196] A "database" is an information management system that stores information about devices such as home appliances and communication equipment, and provides the relevant information in response to inquiries from a server.

[0197] A "generative AI model" is an artificial intelligence model that generates guidance text in natural language based on a prompt text.

[0198] A "prompt sentence" is an input sentence sent to a generative AI model to generate a guidance sentence.

[0199] "Guide text" is a text generated by a generative AI model and provided to users containing information such as how to use and maintain the device.

[0200] A "terminal" is an electronic device such as a smartphone or a personal computer that a user uses to check the guidance text.

[0201] The "interface" refers to the system's operation screen and input means for users to register device information and check guidance messages.

[0202] The present invention is a system that allows users to register device information such as home appliances and communication devices that they are using, and provides appropriate guidance and suggestions based on the registered device information. Specific embodiments of this system are described below.

[0203] Hardware and software used

[0204] server

[0205] The server receives the device information sent by the user, queries the database, generates a message using a generative AI model, and sends it to the user's device. The server is a computer system equipped with a high-performance processor and sufficient memory.

[0206] Database

[0207] The database stores information about devices such as home appliances and communication equipment, and provides the relevant information in response to queries from the server. Database management systems (DBMS) such as MySQL and PostgreSQL are used.

[0208] Generative AI Models

[0209] The generative AI model is an artificial intelligence model that generates guidance text in natural language based on a prompt text. Specifically, OpenAI's GPT-4 is used.

[0210] Terminal

[0211] The terminal is an electronic device such as a smartphone or PC that the user uses to check the information. The terminal must have an internet connection and a browser or dedicated application.

[0212] Data processing and calculation

[0213] Registering device information

[0214] Users use their own terminals to access the system's interface and register by entering information such as device type, manufacturer, and model number.

[0215] Querying the database

[0216] The server receives the device information sent by the user and queries the database for the relevant device information. For example, in the case of a refrigerator, it obtains information such as how to clean it regularly, how to deal with error codes, how to purchase accessories, when to replace it, how to use it energy efficiently, and how to recycle it when discarding it.

[0217] Sending prompts to generative AI models

[0218] The server generates a prompt based on the acquired information and sends it to the generative AI model. An example of a prompt is as follows:

[0219] A user has registered a refrigerator. Generate a message based on the following information:

[0220] Regular cleaning: It is recommended to clean the inside of the refrigerator once a month.

[0221] What to do when the error code appears: If the error code E1 appears, the cooling system needs to be inspected.

[0222] How to purchase accessories: Filters and shelves can be purchased from the official website.

[0223] Guideline for replacement: Consider replacing it after 10 years from purchase.

[0224] Energy-efficient use: Keep your refrigerator temperature set at 3-5 degrees.

[0225] How to recycle: Take to your local recycling center.

[0226] Generate and send a message

[0227] The generative AI model generates a guide message in natural language based on the received prompt. The generated guide message is sent to the user's device via the server. The user can check the guide message on their device and obtain information such as how to use and maintain the device.

[0228] Specific examples

[0229] For example, if a user registers information about a refrigerator, the server retrieves the following information from the database:

[0230] Regular cleaning: It is recommended to clean the inside of the refrigerator once a month.

[0231] What to do when the error code appears: If the error code E1 appears, the cooling system needs to be inspected.

[0232] How to purchase accessories: Filters and shelves can be purchased from the official website.

[0233] Guideline for replacement: Consider replacing it after 10 years from purchase.

[0234] Energy-efficient use: Keep your refrigerator temperature set at 3-5 degrees.

[0235] How to recycle: Take to your local recycling center.

[0236] With this information, we send the following prompt to the generative AI model:

[0237] A user has registered a refrigerator. Generate a message based on the following information:

[0238] Regular cleaning: It is recommended to clean the inside of the refrigerator once a month.

[0239] What to do when the error code appears: If the error code E1 appears, the cooling system needs to be inspected.

[0240] How to purchase accessories: Filters and shelves can be purchased from the official website.

[0241] Guideline for replacement: Consider replacing it after 10 years from purchase.

[0242] Energy-efficient use: Keep your refrigerator temperature set at 3-5 degrees.

[0243] How to recycle: Take to your local recycling center.

[0244] The generative AI model then uses this information to generate the following prompt:

[0245] Thank you for using our refrigerator. Please refer to the following information:

[0246] It is recommended that you clean the inside of your refrigerator once a month.

[0247] If error code E1 appears, the cooling system needs to be inspected.

[0248] Accessories can be purchased from the official website.

[0249] Consider replacing it after 10 years of purchase.

[0250] Keeping your refrigerator temperature between 3 and 5 degrees will make it more energy efficient.

[0251] When disposing of it, please take it to your local recycling center.

[0252] This message is sent to the user's terminal via the server, and the user can check it.

[0253] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0254] Step 1:

[0255] The user registers the device information.

[0256] Users access the system interface using their own terminals and register by entering information such as the device type, manufacturer, model number, etc. The entered device information is then sent to the server.

[0257] Input: Device type, manufacturer, model number, and other information

[0258] Output: Device information sent to the server

[0259] Step 2:

[0260] The server receives the device information and queries the database.

[0261] The server receives the device information sent by the user and queries the database for information on the corresponding device based on the received information.

[0262] Input: Device information sent by the user

[0263] Output: Information about the device retrieved from the database

[0264] Step 3:

[0265] The server retrieves information related to the device from a database.

[0266] The server retrieves information related to the device from the database, such as how to clean the refrigerator regularly, how to deal with error codes, how to purchase accessories, when to replace the refrigerator, how to use it energy-efficiently, and how to recycle it when discarded.

[0267] Input: Database query

[0268] Output: Device related information

[0269] Step 4:

[0270] Based on the information obtained by the server, a prompt sentence is sent to the generative AI model.

[0271] The server generates a prompt sentence based on the acquired information and sends the generated prompt sentence to the generative AI model.

[0272] Input: Device related information

[0273] Output: The prompt sent to the generative AI model

[0274] Step 5:

[0275] A generative AI model generates the guide text.

[0276] The generative AI model generates natural language instructions based on the received prompt, such as instructions on how to clean the refrigerator or how to troubleshoot error codes.

[0277] Input: prompt statement

[0278] Output: Generated guidance text

[0279] Step 6:

[0280] The server transmits the generated message to the user's terminal.

[0281] The server sends the guidance message received from the generative AI model to the user's device, where it is displayed.

[0282] Input: Generated guidance text

[0283] Output: The message sent to the user's device

[0284] Step 7:

[0285] The user checks the message on the device.

[0286] The user checks the guide on their device, which contains detailed instructions on how to use and maintain the device.

[0287] Input: The message sent to the user's device

[0288] Output: Guidance text confirmed by the user

[0289] (Application example 2)

[0290] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0291] In conventional systems, users had to search for maintenance and troubleshooting information for the equipment and communication devices they were using individually, making efficient maintenance difficult. Furthermore, there was a lack of a way to provide real-time maintenance information for machinery used in factories, which led to problems with reduced work efficiency.

[0292] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for registering device information of devices, communication devices, etc. currently in use by the user, means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage methods, disposal methods, recycling methods, etc. based on the registered device information, means for providing maintenance information for machines used in the factory, and means for displaying information through an application installed on a smartphone, smart glasses, or head-mounted display. This enables users to efficiently obtain equipment maintenance information and improve work efficiency in the factory.

[0293] "User" means an individual or corporation that uses the system.

[0294] "Equipment" refers to devices such as home appliances and machinery used in factories.

[0295] "Communication device" refers to a device for sending and receiving information over the Internet or a network.

[0296] "Device information" refers to specific information about a device or communication device, such as model number or usage status.

[0297] "Troubleshooting" refers to procedures and methods for diagnosing and resolving problems with equipment or communications devices.

[0298] "Maintenance method" refers to the maintenance and repair procedures for keeping equipment and communication devices operating normally.

[0299] "Accessory Purchasing Method" refers to a procedure for purchasing accessories or parts related to a device or communications equipment.

[0300] "Replacement time / successor product" refers to the expiration date of equipment or communications devices and the next new model that should be purchased.

[0301] "Convenient ways of using" refers to methods and tips for using equipment and communication devices more efficiently.

[0302] "Disposal method" refers to the procedure for properly disposing of used equipment or communications devices.

[0303] "Recycling method" refers to the procedure for reusing used appliances and communication devices.

[0304] "Machinery used in factories" refers to various types of machinery and equipment used in manufacturing and production industries.

[0305] "Maintenance information" refers to specific information regarding the maintenance and repair of equipment and communications devices.

[0306] A "smartphone" is a mobile device that has the functionality of a mobile phone and is also capable of connecting to the Internet and using applications.

[0307] "Smart glasses" refers to a wearable device in the form of glasses that has the function of displaying information.

[0308] "Head-mounted display" refers to a display device worn on the head.

[0309] "Application" refers to a software program that provides a specific function or service.

[0310] A system for implementing this invention includes means for registering device information such as equipment and communication devices currently in use by a user, means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage methods, disposal methods, recycling methods, etc. based on the registered device information, means for providing maintenance information for machines used in factories, and means for displaying information through an application installed on a smartphone, smart glasses, or head-mounted display.

[0311] The server has a database for users to register information about the devices and communication equipment they use. When a user registers device information, the server stores that information in the database. Next, the server uses a generative AI model to obtain information such as troubleshooting and maintenance methods based on the registered device information and provides it to the user.

[0312] The terminal (smartphone, smart glasses, head-mounted display) displays the information sent from the server. For example, maintenance information for machinery used in a factory can be displayed in real time. This allows users to efficiently perform equipment maintenance.

[0313] As a specific example, when a factory worker wears smart glasses and performs maintenance on an assembly robot, the following information is displayed on the smart glasses' display:

[0314] Regular cleaning method

[0315] How to deal with error codes

[0316] How to purchase accessories

[0317] Estimated replacement time

[0318] Energy-efficient use

[0319] Recycling method when disposing of

[0320] An example of a prompt is as follows:

[0321] "Please provide maintenance information for the assembly robot. The device ID is 'robot123'."

[0322] In this way, the user can efficiently obtain the necessary information and perform maintenance on the device.

[0323] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0324] Step 1:

[0325] Users register information about the devices and communication devices they are using. Using a smartphone or PC, users input device information such as the device model number and usage status. The input information is sent to the server and stored in a database. The input data is device information (e.g., model number, usage status), and the output is the device information stored in the database.

[0326] Step 2:

[0327] The server uses a generative AI model to obtain information such as troubleshooting and maintenance methods based on registered device information. The server obtains device information from the database and inputs that information into the generative AI model. The generative AI model uses prompt statements to generate appropriate maintenance information. The input data is device information (e.g., model number, usage status), and the output is the generated maintenance information.

[0328] Step 3:

[0329] The server sends the generated maintenance information to the user's device. The server sends the maintenance information obtained from the generative AI model to the user's smartphone, smart glasses, or head-mounted display. The input data is the generated maintenance information, and the output is the maintenance information sent to the user's device.

[0330] Step 4:

[0331] The terminal displays the received maintenance information. The user's smartphone, smart glasses, or head-mounted display receives the maintenance information sent from the server and displays it on the display. The input data is the maintenance information sent from the server, and the output is the maintenance information displayed on the terminal.

[0332] Step 5:

[0333] The user performs equipment maintenance based on the displayed maintenance information. The user cleans, repairs, and purchases parts for the equipment while referring to the information displayed on the terminal. The input data is the maintenance information displayed on the terminal, and the output is the maintenance work performed by the user.

[0334] Step 6:

[0335] When a user inputs a new problem or question, the terminal sends the information to the server. The user inputs a new problem or question using the terminal interface and sends it to the server. The input data is the new problem or question input by the user, and the output is the information sent to the server.

[0336] Step 7:

[0337] The server updates and adjusts the information based on new problems or questions from the user. The server receives input from the user, generates new information using a generative AI model, and sends it to the user's device. The input data is the new problem or question from the user, and the output is the updated and adjusted information.

[0338] Example 3

[0339] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0340] Conventional management systems for home appliances and communication devices have struggled to provide optimal information tailored to the user's usage situation and environment. They also lacked specific suggestions that took into account weather information and user preferences, resulting in low user convenience. Furthermore, they lacked the ability to update and adjust information in real time through an interactive interface, making it difficult to respond quickly to user needs.

[0341] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0342] In this invention, the server includes: means for registering device information for home appliances, communication devices, etc. currently in use by a user; means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. based on the registered device information; means for acquiring weather information for the user's place of residence; and means for providing information optimized according to the user's usage situation and environment using a generative AI model. This makes it possible to generate prompt text based on weather information for the user's place of residence and device information, and to provide optimal information using the generative AI model. Furthermore, information can be updated and adjusted according to user input through an interactive interface, enabling quick response to user needs.

[0343] "User" refers to any individual or legal entity that uses the System.

[0344] "Home appliances" refers to all electrical appliances used in the home.

[0345] "Communications equipment" refers to a device for sending and receiving information.

[0346] "Device information" refers to information about home appliances and communication devices currently in use by a user.

[0347] "Troubleshooting" refers to the process of diagnosing device problems and providing solutions.

[0348] "Maintenance method" refers to procedures and methods for maintaining the proper operation of a device.

[0349] "Accessory Purchase Method" refers to the method of purchasing accessories and parts related to a Device.

[0350] "Replacement / Successor" refers to information about when to replace a device or a new model.

[0351] "Helpful tips" refers to tips and tricks for using the device more effectively.

[0352] "Disposal Method" refers to the method for properly disposing of a used device.

[0353] "Recycling method" refers to a method for disposing of used devices in a reusable form.

[0354] "Climate information" refers to environmental data such as the weather and temperature in the user's place of residence.

[0355] A "generative AI model" refers to an algorithm that uses artificial intelligence to analyze data and generate optimal information.

[0356] A "prompt sentence" refers to an instruction sentence to be input into a generative AI model.

[0357] "Interactive interface" refers to an operating screen that allows a user to interact with the system.

[0358] This invention is a system that registers device information of home appliances, communication devices, etc. that a user is using, and provides information optimized according to the user's usage status and environment. Specific embodiments of this system are described below.

[0359] Hardware and software used

[0360] Hardware: Servers, user devices (PCs, smartphones, etc.)

[0361] Software: Generative AI models (e.g., GPT-4), database management systems (e.g., MySQL), weather information APIs (e.g., OpenWeatherMap)

[0362] Program processing flow

[0363] 1. Receiving user inquiries

[0364] Device: The user makes an inquiry about the temperature setting of the air conditioner. For example, the user uses a smartphone app to type, "What is the temperature setting of the air conditioner?"

[0365] Server: Receives the query sent from the terminal. The server analyzes this query and proceeds to the next processing step.

[0366] 2. Obtaining user information

[0367] Server: Using a database management system (e.g., MySQL), obtain information such as the user's place of residence, type of air conditioner, past setting history, etc. For example, obtain that the user's place of residence is "Tokyo," the type of air conditioner is "Daikin," and the past setting history is "25 degrees."

[0368] Server: Temporarily stores the acquired information in memory and proceeds to the next processing step.

[0369] 3. Obtaining weather information

[0370] Server: Uses a weather information API (e.g., OpenWeatherMap) to obtain the current weather information for the user's location. For example, obtains that the current temperature is "30 degrees."

[0371] Server: The acquired weather information is temporarily stored in memory and then the process proceeds to the next step.

[0372] 4. Optimization with generative AI models

[0373] Server: User information and weather information are input into a generative AI model (e.g., GPT-4) to create a prompt that generates the optimal temperature setting. For example, a prompt like "The user lives in Tokyo, and the current temperature is 30 degrees. The air conditioner is a Daikin product, and the user has previously preferred a setting of 25 degrees. Please suggest the optimal temperature setting."

[0374] Generative AI model: Suggests the optimal temperature setting based on a prompt. For example, it generates a result such as, "The current temperature is high, so the optimal temperature setting for the air conditioner is 24 degrees."

[0375] 5. Recommendations for optimal temperature settings

[0376] Server: Response to the user with the optimal temperature setting obtained from the generative AI model. For example, it generates a message such as, "The current temperature is high, so the optimal temperature setting for the air conditioner is 24 degrees."

[0377] On the device: The user confirms the suggested temperature setting. For example, a smartphone app displays a message saying, "The current temperature is high, so the optimal temperature setting for the air conditioner is 24 degrees."

[0378] Specific examples

[0379] User question: "What temperature setting should I set my air conditioner to?"

[0380] Example prompt: "The user lives in Tokyo, and the current temperature is 30 degrees. The air conditioner is a Daikin, and the user has previously preferred a setting of 25 degrees. Please suggest the optimal temperature setting."

[0381] By inputting this prompt into the generative AI model, the system will make specific suggestions such as, "The current temperature is high, so the optimal temperature setting for the air conditioner is 24 degrees."

[0382] The above is a specific embodiment for carrying out the present invention. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0383] Step 1:

[0384] Receiving user inquiries

[0385] Input: A user uses a device (e.g., a smartphone) to make a query such as, "What temperature setting should I set my air conditioner to?"

[0386] Specific behavior: The user enters the inquiry content into the application's input field and presses the send button.

[0387] Output: The query is sent from the terminal to the server.

[0388] Server processing: The server analyzes the received query and prepares to proceed to the next step.

[0389] Step 2:

[0390] Obtaining user information

[0391] Input: The server queries a database management system (e.g., MySQL) for information such as the user's location, type of air conditioner, and past setting history.

[0392] What happens next: The server generates an SQL query and sends it to the database.

[0393] Output: The database returns information such as the user's place of residence (e.g., Tokyo), the type of air conditioner (e.g., Daikin), and past setting history (e.g., 25 degrees).

[0394] Server processing: The acquired information is temporarily stored in memory and the next step is taken.

[0395] Step 3:

[0396] Obtaining weather information

[0397] Input: The server queries a weather information API (e.g., OpenWeatherMap) for the current weather information for the user's location.

[0398] Specific operation: The server generates an API request and sends it to the weather information API.

[0399] Output: Weather information API returns weather information such as the current temperature (e.g., 30 degrees).

[0400] Server processing: The acquired weather information is temporarily stored in memory and the next step is performed.

[0401] Step 4:

[0402] Optimization with generative AI models

[0403] Input: The server generates a prompt based on the user information and weather information.

[0404] Specific behavior: The server creates a prompt sentence that reads, "The user lives in Tokyo, and the current temperature is 30 degrees. The air conditioner is a Daikin product, and the user has previously preferred to set it to 25 degrees. Please suggest the optimal temperature setting."

[0405] Output: The generated prompt sentence is sent to a generative AI model (e.g., GPT-4).

[0406] Processing of the generative AI model: The generative AI model analyzes the prompt and suggests the optimal temperature setting (e.g., 24 degrees).

[0407] Step 5:

[0408] Proposal for optimal temperature settings

[0409] Input: Optimal temperature setting obtained from a generative AI model (e.g., 24°C)

[0410] Specific operation: The server receives the output from the generative AI model and generates a message to reply to the user.

[0411] Output: Produces the message "The current temperature is high, so the optimal temperature setting for your air conditioner is 24 degrees."

[0412] Device action: A message is displayed on the user's device, and the user confirms the suggested temperature setting.

[0413] The above is the specific flow of the program processing of this system.

[0414] (Application example 3)

[0415] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0416] Conventional systems provide limited information about the device the user is using, making it difficult to suggest optimal settings based on the user's current climate and individual preferences. Furthermore, the system lacks information optimized for the user's usage situation and environment, preventing it from fully improving user convenience. This poses the problem of users having to spend time and effort finding the optimal way to use their device.

[0417] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0418] In this invention, the server includes means for registering device information of home appliances, communication devices, etc. currently in use by the user, means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. based on the registered device information, and means for suggesting optimal settings taking into consideration the climate of the user's current location, the type of device, the user's preferences, etc. This allows the user to obtain optimal information according to their own usage situation and environment, and to easily find the optimal way to use the device.

[0419] "User" means an individual or organization that uses the system.

[0420] "Home appliances" is a general term for electrical devices used in the home.

[0421] A "communications device" is a device for sending and receiving information.

[0422] "Device information" is detailed data about the home appliances and communication devices that a user is using.

[0423] "Troubleshooting" is the process of diagnosing device problems and providing solutions.

[0424] "Maintenance methods" are procedures and methods for properly maintaining a device.

[0425] "Accessory Purchase Method" means a procedure for purchasing accessories or additional parts related to a device.

[0426] "Time to replace" is the appropriate time to replace a device with a new one.

[0427] A "successor" is a next-generation model or improved version of a device you are currently using.

[0428] "Convenient usage" is a way to use a device efficiently and effectively.

[0429] "Disposal Instructions" are procedures for properly disposing of used devices.

[0430] A "recycling method" is a procedure for converting a used device into a reusable form.

[0431] "Current location climate" refers to the weather and temperature conditions in the location where the user is currently located.

[0432] "Device type" refers to the category and model of consumer electronics or communications equipment used.

[0433] "User preferences" are the tendency of a user to prefer a particular setting or environment.

[0434] "Optimal settings" are device settings that are most suitable for the user's usage and environment.

[0435] A system for implementing this invention includes means for registering device information for home appliances, communication devices, etc. currently in use by a user, means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage methods, disposal methods, recycling methods, etc. based on the registered device information, and means for suggesting optimal settings taking into consideration the climate of the user's current location, the type of device, the user's preferences, etc.

[0436] Program processing explanation

[0437] The server has a database for registering information about devices used by users. Users enter device information using a device such as a smartphone or PC and send it to the server. The server provides information such as troubleshooting and maintenance methods based on the registered device information.

[0438] Additionally, the server uses external services such as the Weather API to obtain weather data for the user's current location. This allows the server to obtain real-time weather information for the user's current location. The server calculates and suggests optimal settings to the user based on the user's preferences and device type.

[0439] As a specific example, consider a case where a user inquires about the interior temperature setting of an autonomous vehicle. The server obtains climate data for the user's current location and suggests the optimal interior temperature, taking into account the user's past preferred temperature settings. This information is provided to the user through a smartphone application.

[0440] Hardware and software used

[0441] Hardware: Servers, smartphones, PCs

[0442] Software: Database management system, Weather API, user interface application

[0443] Prompt Sentence Examples

[0444] If the user is in Tokyo, suggest the optimal car temperature based on the current temperature. The user prefers a warm environment.

[0445] Using this prompt, the generative AI model can suggest optimal temperature settings that take into account the user's preferences and the current climate.

[0446] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0447] Step 1:

[0448] Users use devices such as smartphones and personal computers to input device information for home appliances, communication devices, etc., and send it to a server.

[0449] Input: Device information (e.g., device name, model number, start date, etc.)

[0450] Output: Device information sent to the server

[0451] Specific operation: The user opens the dedicated application, enters device information into the input form, and presses the "Submit" button.

[0452] Step 2:

[0453] The server registers the received device information in a database.

[0454] Input: Device information sent by the user

[0455] Output: Device information stored in the database

[0456] Specific operation: The server analyzes the received device information and stores it in a database in an appropriate format.

[0457] Step 3:

[0458] The user inquires about the vehicle temperature setting.

[0459] Input: User query (e.g., "What is the ideal car interior temperature?")

[0460] Output: The query sent to the server

[0461] Specific behavior: The user enters a question into an inquiry form within the application and presses the "Submit" button.

[0462] Step 4:

[0463] The server calls the Weather API to get weather data for the user's current location.

[0464] Input: User's current location (e.g., latitude, longitude)

[0465] Output: Current location climate data (e.g. temperature, humidity)

[0466] Specific operation: The server calls the Weather API based on the user's current location information and obtains real-time weather data.

[0467] Step 5:

[0468] The server calculates the optimal settings based on user preferences and device type.

[0469] Input: Climate data, user preferences, device type

[0470] Output: Optimal setting (e.g. recommended interior temperature)

[0471] What it does: The server runs an algorithm that calculates the optimal settings based on the weather data, device information, and user preferences.

[0472] Step 6:

[0473] The server notifies the user of the calculation results.

[0474] Input: Optimal settings (e.g. recommended interior temperature)

[0475] Output: Optimal configuration information displayed on the user's device

[0476] Specific behavior: The server sends the calculation result to the user's device and displays a notification within the application.

[0477] Step 7:

[0478] The user reviews the proposed settings and applies them to the device if necessary.

[0479] Input: Notification content from the server

[0480] Output: Settings applied to the device

[0481] What happens: The user sees the notification in the app and takes action to apply the suggested settings to their device.

[0482] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0483] "Example 1"

[0484] One embodiment of the present invention provides a system including an emotion engine that recognizes a user's emotions. The system includes a means for registering device information, such as a home appliance or communication device, currently used by the user, and a means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc., based on the registered device information. The emotion engine analyzes information such as the user's voice, facial expressions, and behavioral patterns to recognize emotions, and adjusts the content and expression of the guidance and suggestions according to the emotions.

[0485] "Example 2"

[0486] For example, if a user is having trouble operating an air conditioner, the system can sense confusion or dissatisfaction from the user's tone of voice and facial expression. The emotion engine analyzes this information and recognizes that the user is confused. The system then provides a more detailed and easy-to-understand explanation based on the user's emotion. Also, if the user expresses joy or satisfaction, the system can recognize that emotion and provide the user with further suggestions and information.

[0487] "Example 3"

[0488] One embodiment of the present invention provides a system including an emotion engine that recognizes a user's emotions. The system includes a means for registering device information, such as a home appliance or communication device, currently used by the user, and a means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc., based on the registered device information. The emotion engine analyzes information such as the user's voice, facial expressions, and behavioral patterns to recognize emotions, and adjusts the content and expression of the guidance and suggestions according to the emotions.

[0489] The processing flow of each embodiment will be described below.

[0490] "Example 1"

[0491] Step 1: The user registers the information of the device they are using in the system.

[0492] Step 2: Based on the registered device information, the system will provide guidance and suggestions on troubleshooting, maintenance methods, how to purchase accessories, when to replace / replace the device, convenient usage, disposal methods, recycling methods, etc.

[0493] Step 3: The emotion engine analyzes information such as the user's voice, facial expressions, and behavioral patterns to recognize emotions.

[0494] Step 4: The emotion engine adjusts the content and expression of the guidance and suggestions according to the recognized emotion.

[0495] "Example 2"

[0496] Step 1: When a user is having trouble operating the air conditioner, the system detects confusion or dissatisfaction from the user's tone of voice and facial expression.

[0497] Step 2: The emotion engine analyzes this information and recognizes that the user is confused.

[0498] Step 3: The system responds to the user's feelings by providing a more detailed and understandable explanation.

[0499] Step 4: If the user expresses happiness or satisfaction, the system recognizes the emotion and provides further suggestions or information to the user.

[0500] Example 1

[0501] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0502] Conventional device management systems allow users to register information about their electronic devices and communication devices, but they lack sufficient guidance and suggestions on troubleshooting, maintenance methods, how to purchase accessories, when to replace / replace, convenient usage, disposal methods, and recycling methods. Furthermore, since the guidance and suggestions do not take into account the user's feelings, user satisfaction is reduced. Furthermore, the lack of information provision optimized for the user's usage situation and environment, and the lack of information updating and adjustment through an interactive interface, reduces user convenience.

[0503] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0504] In this invention, the server includes means for registering device information of electronic devices, communication devices, etc. currently in use by the user, means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage methods, disposal methods, recycling methods, etc. based on the registered device information, means for recognizing emotions by analyzing information such as the user's voice, facial expressions, and behavioral patterns, and means for adjusting the content and expression of the guidance and suggestions according to the recognized emotions. This enables optimal guidance and suggestions that take the user's emotions into consideration, improving user satisfaction and convenience.

[0505] A "user" is an individual or corporation that uses the system to register information about electronic devices or communication devices and receives guidance and suggestions.

[0506] "Electronic devices" refers to all devices that users use on a daily basis, such as household appliances and communication devices.

[0507] "Communications equipment" refers to devices used to send and receive information over the Internet or telephone lines.

[0508] "Device information" refers to detailed information such as the type, manufacturer, model name, purchase date, and usage status of electronic devices or communication devices.

[0509] "Troubleshooting" refers to the process of diagnosing problems with electronic or communications devices and providing solutions.

[0510] "Maintenance method" refers to the maintenance procedures for maintaining the normal operation of electronic equipment and communication devices.

[0511] "Accessory purchasing method" refers to the procedure for purchasing accessories and parts related to electronic devices and communication devices.

[0512] "Replacement time / successor product" refers to the time when it is recommended to replace electronic devices or communication devices currently in use and the successor model.

[0513] "Convenient usage" refers to ways to use electronic devices and communication devices more effectively and efficiently.

[0514] "Disposal method" refers to the procedure for properly disposing of used electronic or communications equipment.

[0515] "Recycling method" refers to a procedure for reusing or recycling used electronic or communication devices.

[0516] "Means for recognizing emotions" refers to technology that identifies emotions by analyzing information such as the user's voice, facial expressions, and behavioral patterns.

[0517] "Means for adjusting the content and expression of guidance and suggestions" refers to technology that changes the content and expression of information provided based on the recognized user emotions.

[0518] An "interactive interface" refers to an interface that allows a user to interact with a system.

[0519] The system of the present invention allows users to register information about electronic devices and communication devices they are using, and based on that information, provides guidance and suggestions on troubleshooting, maintenance methods, how to purchase accessories, when to replace / replace, convenient ways to use, how to dispose of, how to reuse, etc. Furthermore, it includes a function to recognize the user's emotions and adjust the content and expression of the guidance and suggestions according to those emotions.

[0520] System configuration

[0521] Registering user device information

[0522] Users use dedicated applications or websites to enter information about the electronic devices and communication devices they use, such as the type of device (e.g., smartphone, refrigerator), manufacturer, model name, purchase date, and usage status (e.g., daily use, weekend use).

[0523] Receiving and storing device information

[0524] The server receives the device information sent from the terminal. The server stores the received information in a database. A relational database (e.g., MySQL, PostgreSQL) is used for the database.

[0525] Analyzing device information and generating guidance and suggestions

[0526] The server analyzes the device information stored in the database, using programming languages ​​such as Python and R and applying machine learning algorithms. Based on the analysis results, it generates guidance and suggestions on troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, and recycling methods.

[0527] Emotion recognition by emotion engine

[0528] The emotion engine recognizes emotions by analyzing information such as the user's voice, facial expressions, and behavioral patterns. It uses voice recognition software (e.g., Google® Cloud Speech-to-Text) for voice recognition, facial expression recognition software (e.g., Microsoft® Azure® Face API) for facial expression recognition, and behavioral pattern analysis software (e.g., Amazon Rekognition) for behavioral pattern analysis. This information is collected in real time while the user is using the application or website.

[0529] Emotionally tailored guidance and suggestions

[0530] The server adjusts the content and expression of the guidance and suggestions based on the user's emotions recognized by the emotion engine. For example, if the user is feeling stressed, the server can provide more concise and easy-to-understand guidance. If the user is feeling relaxed, the server can include more detailed information and additional suggestions.

[0531] Providing guidance and suggestions

[0532] The server sends the tailored guidance and suggestions to the user's device, where the user can view the provided guidance and suggestions on the application or website.

[0533] Specific examples

[0534] For example, if a user feels that their smartphone battery is draining quickly, the following steps will be taken to guide them.

[0535] 1. The user opens the application and registers their smartphone information (manufacturer: common smartphone manufacturer, model name: common model name, purchase date: January 2021, usage status: daily use).

[0536] 2. The server receives the registered information and stores it in a database.

[0537] 3. The server analyzes the smartphone information and suggests battery troubleshooting methods (e.g., limiting background apps, calibrating the battery).

[0538] 4. The emotion engine recognizes from the user's voice that they are feeling stressed.

[0539] 5. The server should be tailored to provide simple and easy-to-understand guidance.

[0540] 6. The server provides the tailored guidance to the user, who confirms it on the application.

[0541] Prompt Sentence Examples

[0542] "Please describe a system that allows users to register information about the device they are using and provides troubleshooting and maintenance guidance based on that information. Please also include a function that recognizes the user's emotions and adjusts the guidance content accordingly."

[0543] The above is a specific embodiment of the system of the present invention.

[0544] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0545] Step 1:

[0546] The user registers the device information.

[0547] Users use dedicated applications or websites to enter information about the electronic devices or communication devices they are using, including the device type, manufacturer, model name, purchase date, usage status, etc. The entered data is then sent from the device to a server.

[0548] Step 2:

[0549] The server receives the registered device information and stores it in a database.

[0550] The server receives device information sent from the terminal. The received data is stored in a database. A relational database (e.g., MySQL or PostgreSQL) is used for the database. The input data is device information, and the output is the device information stored in the database.

[0551] Step 3:

[0552] The server analyzes the device information and generates appropriate guidance and suggestions.

[0553] The server analyzes the device information stored in the database. For the analysis, it uses programming languages ​​such as Python and R and applies machine learning algorithms. Based on the analysis results, it generates guidance and suggestions on troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, and recycling methods. The input data is the device information, and the output is the generated guidance and suggestions.

[0554] Step 4:

[0555] The emotion engine recognizes the user's emotions.

[0556] The emotion engine recognizes emotions by analyzing information such as the user's voice, facial expressions, and behavioral patterns. It uses voice recognition software (e.g., Google Cloud Speech-to-Text) for voice recognition, facial expression recognition software (e.g., Microsoft Azure Face API) for facial expression recognition, and behavioral pattern analysis software (e.g., Amazon Rekognition) for behavioral pattern analysis. The input data are the user's voice, facial expressions, and behavioral patterns, and the output is the recognized emotion.

[0557] Step 5:

[0558] The server adjusts the content and expression of the guidance and suggestions according to the emotion.

[0559] The server adjusts the content and expression of the guidance and suggestions based on the user's emotions recognized by the emotion engine. For example, if the user is feeling stressed, the server will provide more concise and easy-to-understand guidance. The input data are the recognized emotions and the generated guidance and suggestions, and the output is the adjusted guidance and suggestions.

[0560] Step 6:

[0561] The server provides tailored guidance and suggestions to the user.

[0562] The server sends the tailored guidance and suggestions to the user's device, and the user can view the provided guidance and suggestions on the application or website. The input data are the tailored guidance and suggestions, and the output is the guidance and suggestions provided to the user.

[0563] (Application example 1)

[0564] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0565] Conventional maintenance systems for machinery and equipment in factories allow users to register information about devices in use and provide guidance and suggestions on troubleshooting and maintenance methods, but they lack suggestions that take user emotions into consideration or suggestions for reducing worker stress and fatigue, resulting in issues such as insufficient improvement of work efficiency and worker health management.

[0566] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0567] In this invention, the server includes: means for registering device information of home appliances, communication devices, etc. currently in use by a user; means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage methods, disposal methods, recycling methods, etc. based on the registered device information; means including an emotion engine for recognizing the user's emotions; means for adjusting the content and expression of the guidance and suggestions according to the user's emotions recognized by the emotion engine; means for providing and suggesting maintenance, troubleshooting, suggestions for part replacement timing, efficient usage methods, disposal and recycling methods, etc. for machinery and equipment in a factory; and means for recognizing the emotions of workers and making suggestions to reduce stress and fatigue. This enables optimal guidance and suggestions according to the user's emotions, thereby realizing efficient maintenance of machinery and equipment in a factory and health management of workers.

[0568] A "user" is an individual or corporation that uses the system to register device information such as home appliances and communication devices and receives guidance and suggestions.

[0569] "Device information" refers to information such as the type, manufacturer, model name, purchase date, and usage status of home appliances and communication devices currently in use by the user.

[0570] "Troubleshooting" is the process of diagnosing device malfunctions and providing solutions.

[0571] "Maintenance Method" means the maintenance and inspection procedures for maintaining the normal operation of the Device.

[0572] "Accessory purchase method" refers to the procedure for purchasing accessories and parts related to the device.

[0573] "Replacement / successor product" refers to the expiration date of the device, the time to upgrade to a new model, and its successor product.

[0574] "Convenient usage" is a way to use a device more efficiently and effectively.

[0575] "Disposal Instructions" are procedures for properly disposing of used devices.

[0576] A "recycling method" is a procedure for disposing of used devices in a reusable form.

[0577] An "emotion engine" is a system that recognizes emotions by analyzing information such as the user's voice, facial expressions, and behavioral patterns.

[0578] "Machinery and equipment in the factory" refers to production machinery and related equipment used in the factory.

[0579] "Workers" are workers who operate and manage machinery and equipment within a factory.

[0580] "Stress" is the mental or physical strain that workers experience while working.

[0581] "Fatigue" refers to the physical and mental exhaustion that workers experience while working.

[0582] As an embodiment of the present invention, a system is provided that provides guidance and suggestions for maintenance of machinery and equipment in a factory, troubleshooting, suggesting parts replacement times, efficient usage methods, disposal and recycling methods, etc. Furthermore, the system includes a function to recognize the emotions of workers and make suggestions to reduce stress and fatigue.

[0583] System Program

[0584] The server includes the following means:

[0585] 1. How to register device information

[0586] The user registers device information for the home appliances and communication devices they are using. Specifically, they enter information such as the device type, manufacturer, model name, purchase date, and usage status.

[0587] 2. Troubleshooting Methods

[0588] Based on the registered device information, the system diagnoses device failures and malfunctions and provides solutions.

[0589] 3. Maintenance method proposals

[0590] We will suggest maintenance and inspection procedures and part replacement times to maintain normal operation of the device.

[0591] 4. Means including an emotional engine

[0592] Emotions are recognized by analyzing information such as the user's voice, facial expressions, and behavioral patterns. Specifically, the user's facial expressions and voice are captured using a camera and microphone, and then analyzed using an emotion recognition engine (EmotionRecognizer).

[0593] 5. Means for tailoring guidance and suggestions according to emotions

[0594] The emotion engine adjusts the content and expression of guidance and suggestions depending on the user's emotions. For example, if the user is feeling stressed, the engine suggests taking a break.

[0595] 6. Maintenance methods for machinery and equipment in the factory

[0596] Provides guidance and suggestions on maintenance, troubleshooting, part replacement timing, efficient usage, disposal and recycling methods for machinery and equipment within the factory.

[0597] 7. Means of Recognizing Worker Emotions

[0598] The system recognizes the emotions of workers and makes suggestions to reduce stress and fatigue. Specifically, it uses cameras and microphones to capture the facial expressions and voices of workers, and analyzes them using an emotion recognition engine.

[0599] Hardware and software used

[0600] Hardware

[0601] Camera: Used to capture the facial expressions of the workers.

[0602] Microphone: Used to capture the voice of the worker.

[0603] Computer: The environment in which a program runs.

[0604] software

[0605] OpenCV: Image processing library used to acquire and display camera images.

[0606] EmotionRecognizer: An emotion recognition engine used to analyze the emotions of workers.

[0607] maintenance_suggestions: A module for generating maintenance suggestions.

[0608] Specific examples

[0609] For example, the system can register information about equipment such as conveyor belts and robotic arms in a factory and suggest regular maintenance methods and part replacement times. It can also suggest breaks if workers are feeling stressed, and encourage them to drink more water if they are feeling tired.

[0610] Prompt Sentence Examples

[0611] Develop a system that allows users to register information about the factory equipment they are using, and based on that information, provides guidance and suggestions on troubleshooting, maintenance methods, part replacement times, efficient usage methods, disposal and recycling methods, etc. Furthermore, add a function that recognizes the emotions of workers and makes suggestions to reduce stress and fatigue.

[0612] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0613] Step 1:

[0614] Register device information such as home appliances and communication devices currently used by the user.

[0615] Input: Information such as device type, manufacturer, model name, purchase date, and usage status.

[0616] Processing: The user enters device information through a dedicated application or website and sends it to the server.

[0617] Output: Device information stored on the server.

[0618] Step 2:

[0619] The server performs troubleshooting based on registered device information.

[0620] Input: Device information stored on the server.

[0621] Processing: The server parses the device information and checks it against a database of known troubleshooting solutions.

[0622] Output: Trouble diagnosis and solution.

[0623] Step 3:

[0624] The server will suggest a maintenance method.

[0625] Input: Device information stored on the server.

[0626] Processing: The server calculates the appropriate maintenance method based on device usage and the manufacturer's recommended maintenance schedule.

[0627] Output: Maintenance suggestions.

[0628] Step 4:

[0629] The server uses an emotion engine to recognize the user's emotion.

[0630] Input: User facial and voice data obtained from the camera and microphone.

[0631] Processing: The server uses EmotionRecognizer to analyze the acquired data and recognize the user's emotions.

[0632] Output: Perceived user sentiment.

[0633] Step 5:

[0634] The server adjusts the content and expression of the guidance and suggestions according to the emotion.

[0635] Input: Perceived user sentiment, troubleshooting results, and maintenance suggestions.

[0636] Processing: The server adjusts the content and expression of the guidance and suggestions according to the user's emotions. For example, if the user is feeling stressed, the server suggests taking a break.

[0637] Output: Adjusted guidance and suggestions.

[0638] Step 6:

[0639] The server provides guidance and suggestions for maintenance of machinery and equipment within the factory.

[0640] Input: Device information of machines and equipment in the factory.

[0641] Processing: The server analyzes device information and calculates maintenance methods, troubleshooting, part replacement times, efficient usage methods, disposal and recycling methods, etc.

[0642] Output: Maintenance guide and suggestions for machinery and equipment within the factory.

[0643] Step 7:

[0644] The server recognizes the worker's emotions and makes suggestions to reduce stress and fatigue.

[0645] Input: Facial expression and voice data of the worker obtained from the camera and microphone.

[0646] Processing: The server uses EmotionRecognizer to analyze the acquired data and recognize the worker's emotions. Based on the recognized emotions, it makes suggestions for reducing stress and fatigue.

[0647] Output: Suggestions based on the worker's emotions.

[0648] Example 2

[0649] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0650] Conventional support systems for home appliances and communication devices only provide uniform information for problems users face, and have the problem of being unable to respond flexibly to the user's specific situation or emotional state. Also, when a user is confused or in a particular emotional state, the system is unable to recognize that state and provide appropriate guidance or suggestions, which leads to a poor user experience.

[0651] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for registering device information of home appliances, communication devices, etc. currently used by the user, a means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage methods, disposal methods, recycling methods, etc. based on the registered device information, a means for acquiring user voice data and converting it into text data using voice recognition technology, a means for analyzing the user's tone of voice and facial expressions to recognize the user's emotional state, and a means for adjusting the information and suggestions according to the user's emotional state. This enables flexible response according to the user's specific situation and emotional state.

[0652] "Device information" is information about a specific device, such as a home appliance or communication device, that the user is using.

[0653] "Troubleshooting" refers to procedures or methods for diagnosing and resolving device problems or malfunctions.

[0654] A "maintenance method" is a procedure for regular maintenance and inspection to keep a device operating normally.

[0655] "Accessory Purchase Method" means the process or method for purchasing additional parts or accessories related to a Device.

[0656] "Replacement time / successor product" is a guideline or information for replacing a device with a new model or successor when the device is nearing the end of its useful life.

[0657] "Hints" are tips and tricks for using your device more efficiently and effectively.

[0658] "Disposal method" means the procedure or method for properly disposing of a used device.

[0659] A "recycling method" is a procedure or method for disposing of used devices in a reusable form.

[0660] "Voice data" is digital data that records the user's speech.

[0661] "Voice recognition technology" is a technology that analyzes voice data and converts it into text data.

[0662] "Text data" is character information converted using voice recognition technology.

[0663] "Tone of voice" refers to changes in pitch and intensity of a voice that indicate the emotion or intention contained in a user's speech.

[0664] "Facial expressions" refer to emotions and intentions expressed by the user's facial movements and expressions.

[0665] "Emotional state" refers to a user's current psychological state or feelings.

[0666] "Means for adjusting guidance and suggestions" refers to methods and techniques for changing the content of information and guidance provided depending on the emotional state of the user.

[0667] This invention is a system that allows users to register device information for home appliances, communication devices, etc. currently in use, and based on the registered device information, provides guidance and suggestions on troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. It also includes a function to acquire user voice data, convert it into text data using voice recognition technology, analyze the user's tone of voice and facial expressions to recognize the user's emotional state, and adjust the guidance and suggestions according to the user's emotional state.

[0668] Hardware and software used

[0669] Hardware

[0670] Device: The device used by the user (smartphone, tablet, computer, etc.)

[0671] Server: A server that handles databases and processing

[0672] Microphone: A microphone for capturing the user's voice

[0673] Camera: A camera for capturing the user's facial expressions

[0674] software

[0675] Speech recognition software: Software for converting voice data into text data (e.g., Google Speech-to-Text API)

[0676] Speech synthesis software: Software that translates text data into audio (e.g., Google Text-to-Speech API)

[0677] Emotion analysis engine: Software that analyzes a user's tone of voice and facial expressions to recognize their emotional state (e.g., OpenAI's GPT-4 API)

[0678] Database manipulation libraries: Libraries for retrieving information from databases (e.g., SQLAlchemy)

[0679] Program processing

[0680] The server registers information about the device being used by the user in a database. When the user asks a question about the device, the terminal captures the question as voice data and converts it into text data using voice recognition software. The converted text data is sent to the server, which retrieves the relevant device information from the database. The retrieved information is then sent to the terminal, which then displays the information to the user or provides audio guidance.

[0681] Furthermore, the device acquires data using a microphone and camera to analyze the user's tone of voice and facial expressions. The acquired data is sent to an emotion analysis engine, which analyzes the user's emotional state. The analysis results are sent to a server, which adjusts the guidance and suggestions according to the user's emotional state. The adjusted guidance and suggestions are sent to the device and are displayed or spoken to the user.

[0682] Specific examples

[0683] For example, if a user is having trouble operating an air conditioner, they might say, "The air conditioner isn't working." The device captures this speech and uses voice recognition software to convert it into text data saying, "The air conditioner isn't working." The server receives this text data and retrieves troubleshooting information for the air conditioner from a database. The retrieved information is sent to the device, which then displays a message saying, "Please make sure the air conditioner is turned on."

[0684] At the same time, the device analyzes the user's tone of voice and facial expression, and conveys the user's confusion to the emotion analysis engine. The emotion analysis engine analyzes this information and notifies the server that the user is confused. The server then responds by providing a more detailed explanation. For example, it might generate additional guidance such as "Please also check the batteries in the remote control" and send it to the device. The device then displays or audibly informs the user of this additional guidance.

[0685] Prompt Sentence Examples

[0686] "If a user is having trouble operating the air conditioner, how can I provide them with guidance? If I can tell from the user's tone of voice and facial expression that they are confused, how can I provide more detailed instructions?"

[0687] In this way, the system can respond flexibly to the user's specific situation and emotional state.

[0688] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0689] Step 1:

[0690] The user asks a question about the device.

[0691] Input: User's spoken question (e.g., "My air conditioner isn't working. What should I do?")

[0692] Output: Audio data

[0693] Specific operation: If the user is having trouble operating the air conditioner, they ask, "The air conditioner isn't working."

[0694] Step 2:

[0695] The terminal acquires the user's question as voice data.

[0696] Input: User's voice data

[0697] Output: Recorded audio data

[0698] Specific operation: The device uses the built-in microphone to record the user's voice.

[0699] Step 3:

[0700] The terminal converts the voice data into text data.

[0701] Input: Recorded audio data

[0702] Output: Text data (e.g., "My air conditioner isn't working. What should I do?")

[0703] What it does: The device uses voice recognition software (e.g., Google Speech-to-Text API) to convert voice data into text data.

[0704] Step 4:

[0705] The terminal transmits the text data to the server.

[0706] Input: Text data

[0707] Output: Text data as an HTTP request

[0708] Specific operation: The terminal sends the converted text data to the server as an HTTP request.

[0709] Step 5:

[0710] The server retrieves the relevant device information from the database.

[0711] Input: Text data (e.g., "The air conditioner isn't working")

[0712] Output: Device information (e.g. "Please check if the air conditioner is turned on")

[0713] Specific operation: The server uses SQLAlchemy to retrieve air conditioner troubleshooting information from the database.

[0714] Step 6:

[0715] The server transmits the acquired information to the terminal.

[0716] Input:Device information

[0717] Output: Device information in JSON format

[0718] Specific operation: The server sends the acquired information to the terminal in JSON format.

[0719] Step 7:

[0720] The terminal displays or audibly guides the user through the information.

[0721] Input: Device information in JSON format

[0722] Output: Displayed information or audio prompts

[0723] Specific operation: The device displays the acquired information on the screen or uses voice synthesis software (e.g., Google Text-to-Speech API) to provide voice guidance.

[0724] Step 8:

[0725] The device analyzes the user's tone of voice and facial expressions.

[0726] Input: User's voice tone and facial expression data

[0727] Output: Parsed emotion data

[0728] What it does: The device uses the built-in camera and microphone to record the user's facial expressions and tone of voice.

[0729] Step 9:

[0730] The device sends the analysis results to the emotion engine.

[0731] Input: Recorded facial expressions and tone of voice data

[0732] Output: Sending data to the emotion engine

[0733] Specific operation: The device sends the recorded data to the emotion engine.

[0734] Step 10:

[0735] An emotion engine analyzes the user's emotional state.

[0736] Input: facial expression and tone of voice data

[0737] Output: Parsed emotional state (e.g., "User is confused")

[0738] How it works: The emotion engine uses OpenAI's GPT-4 API to analyze the user's emotional state.

[0739] Step 11:

[0740] The emotion engine sends the analysis results to the server.

[0741] Input: Parsed emotional state

[0742] Output: Send analysis results to the server

[0743] Specific operation: The emotion engine sends the analysis results in JSON format to the server.

[0744] Step 12:

[0745] The server generates additional guidance and suggestions according to the user's emotional state.

[0746] Input: Parsed emotional state (e.g., "User is confused")

[0747] Output: Additional instructions and suggestions (e.g., "Please also check the batteries in the remote control.")

[0748] Specific behavior: The server generates a more detailed explanation if the user is confused.

[0749] Step 13:

[0750] The server sends additional guidance and suggestions to the terminal.

[0751] Input: Additional guidance and suggestions

[0752] Output: Additional instructions and suggestions in JSON format

[0753] Specific operation: The server sends the generated additional guidance and suggestions to the terminal in JSON format.

[0754] Step 14:

[0755] The terminal displays or audibly provides additional guidance and suggestions to the user.

[0756] Input: Additional instructions and suggestions in JSON format

[0757] Output: Displayed additional information or audio information

[0758] Specific actions: The device will display additional guidance and suggestions on the screen or provide audible guidance using speech synthesis software.

[0759] (Application example 2)

[0760] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0761] Conventional guidance and recommendation systems for home appliances and communication devices provide uniform information without considering the user's emotions, which can lead to confusion and dissatisfaction. Furthermore, the inability to respond appropriately to the user's emotions has led to a lack of improvement in the user experience. Furthermore, in physical stores, there is a lack of support for store clerks to provide appropriate guidance and suggestions to customers.

[0762] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0763] In this invention, the server includes a means for registering device information of home appliances, communication devices, etc. currently used by the user, a means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. based on the registered device information, and a means for analyzing emotions from the user's tone of voice and facial expressions and providing guidance and suggestions according to the emotions. This enables appropriate guidance and suggestions according to the user's emotions, which is expected to improve the user experience. It also enables support for store clerks in physical stores to provide appropriate guidance and suggestions to customers.

[0764] "Device information" is information about appliances such as home appliances and communication devices currently in use by the user.

[0765] "Troubleshooting" is the process of diagnosing device problems or malfunctions and providing solutions.

[0766] A "maintenance method" is a procedure for maintaining and inspecting a device to ensure that it operates normally.

[0767] "Accessory Purchasing Method" refers to the procedure for purchasing accessories and parts related to a device.

[0768] "Replacement / Successor" is information about the expiration date of a device or when it's time to upgrade to a new model.

[0769] "Convenient usage" is a way to use a device efficiently and effectively.

[0770] "Disposal Instructions" are procedures for properly disposing of used devices.

[0771] A "recycling method" is a procedure for disposing of used devices in a reusable form.

[0772] "Voice tone" refers to voice characteristics that express the pitch, strength, and emotion of a user's voice.

[0773] "Facial expressions" refer to emotions and reactions shown by the user's facial movements and expressions.

[0774] "Emotion analysis" is the process of identifying and understanding a user's emotions from their tone of voice and facial expressions.

[0775] "Guide and Suggestion" is information and advice provided to the user.

[0776] An "interactive interface" is an operation screen or input means that allows a user to interact with a system in a two-way manner.

[0777] A system for implementing this invention includes means for registering device information such as home appliances and communication equipment currently in use by a user, means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage methods, disposal methods, recycling methods, etc. based on the registered device information, and means for analyzing the user's emotions from their tone of voice and facial expression and providing guidance and suggestions according to their emotions.

[0778] System Program

[0779] The system program uses the following hardware and software:

[0780] Hardware: Smart glasses (with built-in camera and microphone)

[0781] software:

[0782] EmotionRecognizer: A library for recognizing emotions from facial images

[0783] VoiceRecognizer: A library for recognizing emotions from speech

[0784] ProductDatabase: Database for managing product information

[0785] Program processing

[0786] The server uses EmotionRecognizer to analyze the customer's facial image captured by the smart glasses' camera and recognize the customer's emotions. Next, it uses VoiceRecognizer to analyze the audio recorded by the smart glasses' microphone and recognize the tone of the customer's voice. This allows it to determine the customer's emotions, such as whether they are confused or satisfied.

[0787] The server then retrieves information corresponding to the specified product ID from the product database. Depending on the customer's emotions, it generates appropriate guidance and suggestions and displays them on the smart glasses' display. For example, if the customer is confused, it provides a more detailed explanation, and if satisfied, it makes additional suggestions.

[0788] Specific examples

[0789] As a concrete example, the following prompt sentence is input into the generative AI model:

[0790] A customer has a question about product ID 12345. Analyze the customer's facial image and voice to generate guidance and suggestions based on their emotions.

[0791] By inputting this prompt sentence into a generative AI model, it is possible to automatically generate appropriate guidance and suggestions based on the customer's emotions. This is expected to improve the user experience. It can also be used in physical stores to assist store clerks in providing appropriate guidance and suggestions to customers.

[0792] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0793] Step 1:

[0794] The user wears the smart glasses and captures the customer's facial image and voice.

[0795] Input: Customer's face image taken by the smart glasses camera, customer's voice recorded by the smart glasses microphone

[0796] Output: Customer's face image data, customer's voice data

[0797] How it works: The camera in the smart glasses captures a picture of the customer's face, and the microphone records the customer's questions and comments.

[0798] Step 2:

[0799] The server analyzes the facial image data using EmotionRecognizer to recognize the customer's emotions.

[0800] Input: Customer's facial image data

[0801] Output: Customer sentiment data (e.g., confused, satisfied, etc.)

[0802] Specific operation: The server inputs facial image data into EmotionRecognizer and uses an emotion recognition algorithm to identify the customer's emotions.

[0803] Step 3:

[0804] The server analyzes the voice data using VoiceRecognizer and recognizes the tone of the customer's voice.

[0805] Input: Customer voice data

[0806] Output: Customer's voice tone data (e.g. high, low, strong, weak, etc.)

[0807] What it does: The server inputs the audio data into VoiceRecognizer, which uses a speech analysis algorithm to identify the tone of the customer's voice.

[0808] Step 4:

[0809] The server retrieves information corresponding to the specified product ID from the product database.

[0810] Input: Product ID

[0811] Output: Product information data (e.g., troubleshooting, maintenance methods, etc.)

[0812] Specific operation: The server searches the product database using the product ID and obtains detailed information about the corresponding product.

[0813] Step 5:

[0814] The server generates appropriate guidance and suggestions based on the customer's emotional data and tone of voice data.

[0815] Input: Customer emotion data, tone of voice data, product information data

[0816] Output: Text data of guidance and suggestions

[0817] Specific operation: The server analyzes the emotion data and tone of voice data, and refers to the product information data to generate guidance and suggestions that correspond to the customer's emotions.

[0818] Step 6:

[0819] The server generates guidance and suggestions that are displayed on the smart glasses' display.

[0820] Input: Text data for guidance and suggestions

[0821] Output: Guidance and suggestions displayed on the smart glasses display

[0822] Specific operation: The server generates text data of guidance and suggestions, which are sent to the smart glasses and displayed on the screen.

[0823] Step 7:

[0824] The user checks the guidance and suggestions displayed on the smart glasses display and provides appropriate explanations to the customer.

[0825] Input: Guidance and suggestions displayed on the smart glasses display

[0826] Output: Customer instructions

[0827] Specific operation: The user reads the information displayed on the smart glasses display and verbally explains it to the customer.

[0828] Example 3

[0829] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0830] Conventional systems provide information about electronic devices in use to the user in a uniform manner, making it difficult to provide optimal information tailored to the user's individual usage situation, environment, and emotions. Furthermore, guidance and suggestions do not take the user's emotions into consideration, which results in a problem of not sufficiently improving the user experience.

[0831] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0832] In this invention, the server includes means for registering information about the electronic device being used by the user, means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage methods, disposal methods, recycling methods, etc. based on the registered information about the electronic device, means for acquiring weather information about the user's place of residence, means for recognizing emotions by analyzing information such as the user's voice, facial expressions, and behavioral patterns, and means for adjusting the content and expression of the guidance and suggestions according to the recognized emotions. This makes it possible to provide optimal information according to the user's individual usage situation, environment, and emotions.

[0833] "User" refers to any individual or entity that uses the System.

[0834] "Electronic devices" refer to devices that operate using electricity, such as home appliances and communication equipment.

[0835] "Means for registering information" refers to the functionality that allows users to input and save details of the electronic device they are using into the system.

[0836] "Troubleshooting" refers to the process of diagnosing problems or malfunctions in electronic equipment and providing solutions.

[0837] "Maintenance method" refers to the maintenance and management procedures for ensuring the normal operation of electronic equipment.

[0838] "Accessory Purchasing Method" refers to the procedure for purchasing accessories and parts related to electronic devices.

[0839] "Replacement time / successor product" refers to the expiration date of electronic devices or when it's time to update to a new model.

[0840] "Convenient usage" refers to ways to use electronic devices more efficiently and effectively.

[0841] "Disposal Method" refers to the procedure for properly disposing of used electronic equipment.

[0842] "Recycling method" refers to a procedure for processing used electronic equipment in a reusable form.

[0843] "Means for acquiring weather information" refers to a function for collecting weather data such as temperature, humidity, and season of the user's place of residence.

[0844] "Means for recognizing emotions" refers to a function for determining emotions by analyzing the user's voice, facial expressions, behavioral patterns, etc.

[0845] "Means for adjusting the content and expression of guidance and suggestions" refers to a function for changing the content and expression of the information provided according to the recognized user's emotions.

[0846] MODE FOR CARRYING OUT THE INVENTION

[0847] This invention is a system that registers information about the electronic devices a user is using and provides optimal guidance and suggestions based on that information.The system acquires weather information about the user's location, analyzes the user's emotions, and provides optimal information.

[0848] Hardware and software used

[0849] Hardware:

[0850] server

[0851] User devices (smartphones, tablets, PCs, etc.)

[0852] software:

[0853] Emotion Engine

[0854] Device Information Registration System

[0855] Troubleshooting System

[0856] Program processing explanation

[0857] The server provides a means for users to register information about the electronic devices they are using. Users enter detailed information about the electronic devices they are using (e.g., manufacturer, model, age, etc.), which allows the system to understand the specific devices they are using.

[0858] Next, the server obtains weather information for the user's location. Specifically, it uses weather databases and APIs to collect information such as the current temperature, humidity, and season. This provides the basic data needed to provide information appropriate for the user's environment.

[0859] Furthermore, the server uses an emotion engine to analyze the user's voice, facial expressions, and behavioral patterns. For example, if a user says, "It's hot today," the server recognizes through voice analysis that the user is feeling the heat. Also, if facial expression analysis reveals that the user looks uncomfortable, the server takes that emotion into consideration.

[0860] The server generates optimal information based on the acquired device information, environmental information, and user emotion information. For example, when it comes to air conditioner temperature settings, the server considers that the user prefers a cool environment and suggests setting the air conditioner to 26 degrees if the current temperature is 30 degrees.

[0861] Finally, the server provides the generated optimal information to the user. Specifically, it sends a notification to the user's device (smartphone, tablet, PC, etc.). For example, it displays a message such as, "The current temperature is 30 degrees. We recommend setting the air conditioner to 26 degrees."

[0862] Examples of specific examples and prompts

[0863] Examples:

[0864] When a user asks, "What temperature should I set the air conditioner to?", the server performs the following process.

[0865] 1. The user registers information about the air conditioner they are using (manufacturer, model, age, etc.).

[0866] 2. The server obtains weather information for the user's location (e.g., Tokyo).

[0867] 3. The emotion engine analyzes the user's voice and facial expressions to determine that they prefer a cool environment.

[0868] 4. Based on this information, the server suggests, "Since the average summer temperature in Tokyo is around 30 degrees, the optimal setting for the air conditioner is 26 degrees."

[0869] Example prompt sentence:

[0870] "Please tell me about the temperature settings for the air conditioner. I live in Tokyo and I like a cool environment."

[0871] In this way, the server realizes a system that provides optimal information according to the user's environment and preferences. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0872] Step 1:

[0873] The user registers the device information.

[0874] The user inputs details of the electronic device they are using (e.g., manufacturer, model, age, etc.) into the terminal. The terminal then sends the input device information to the server. The server stores the received device information in a database. This allows the system to know the specific device the user is using.

[0875] Input: Device information entered by the user on the device

[0876] Output: Device information stored on the server

[0877] Step 2:

[0878] The server obtains the environment information.

[0879] The server sends a request to a weather database or API to obtain weather information for the user's location. The weather database or API returns information such as the current temperature, humidity, and season to the server. The server then stores the obtained weather information in a database. This provides the basic data needed to provide information appropriate to the user's environment.

[0880] Input: User's location information

[0881] Output: Climate information stored on the server

[0882] Step 3:

[0883] The server analyzes the user's emotions.

[0884] The server uses an emotion engine to analyze the user's voice, facial expressions, behavioral patterns, etc. For example, if a user says, "It's hot today," the server recognizes through voice analysis that the user is feeling the heat. Also, if facial expression analysis reveals that the user looks uncomfortable, the server takes that emotion into consideration. The analysis results are stored on the server.

[0885] Input: User's voice, facial expressions, and behavioral patterns

[0886] Output: Sentiment analysis results stored on the server

[0887] Step 4:

[0888] The server generates the best information.

[0889] The server generates optimal information based on the acquired device information, environmental information, and user emotion information. For example, when it comes to air conditioner temperature settings, the server considers that the user prefers a cool environment and suggests setting the air conditioner to 26 degrees if the current temperature is 30 degrees. The generated information is stored on the server.

[0890] Input: Device information, weather information, emotion analysis results

[0891] Output: The best information generated

[0892] Step 5:

[0893] The server provides the information to the user.

[0894] The server sends the generated optimal information to the user's device. The device then notifies the user of the received information. For example, it displays a message such as, "The current temperature is 30 degrees. We recommend setting the air conditioner to 26 degrees."

[0895] Input: Generated optimal information

[0896] Output: Notification displayed on the user's device

[0897] In this way, the server realizes a system that provides optimal information according to the user's environment and preferences.

[0898] (Application example 3)

[0899] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0900] Conventional systems have difficulty providing optimal information according to the user's usage situation and environment, and in particular have not provided guidance or suggestions that take the user's emotions into consideration.In addition, there has been a lack of suggestions and support to optimize work efficiency in factories, and there has been a need for a system that can recognize the emotions of workers and provide appropriate support.

[0901] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0902] In this invention, the server includes: means for registering device information for home appliances, communication devices, etc. currently in use by the user; means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage methods, disposal methods, recycling methods, etc. based on the registered device information; means including an emotion engine for recognizing the user's emotions; means for the emotion engine to analyze information such as the user's voice, facial expressions, and behavioral patterns to recognize emotions and adjust the content and expression of the guidance and suggestions in accordance with the emotions; and means for recognizing the emotions of workers and making suggestions and support to optimize work efficiency. This makes it possible to provide optimal information taking user emotions into consideration and provide appropriate support to improve work efficiency in factories.

[0903] "User" means an individual or corporation that uses the system.

[0904] "Home appliances" is a general term for electrical devices used in the home.

[0905] A "communications device" is a device for sending and receiving information.

[0906] "Device information" is data related to home appliances and communication devices currently in use by the user.

[0907] "Troubleshooting" means the process of diagnosing and resolving device problems.

[0908] A "maintenance method" is a maintenance procedure for keeping a device operating normally.

[0909] "Accessory Purchasing Method" refers to the procedure for purchasing accessories and parts related to a device.

[0910] "Replacement / successor" refers to when a device needs to be replaced and its successor product.

[0911] "Convenient usage" is a way to use a device efficiently and effectively.

[0912] "Disposal Instructions" are procedures for properly disposing of used devices.

[0913] A "recycling method" is a procedure for reusing a used device.

[0914] The "emotion engine" is an analysis system for recognizing the user's emotions.

[0915] "Voice" refers to words and sounds uttered by the user.

[0916] "Facial expression" refers to the movements and expressions of the user's face.

[0917] "Behavioral patterns" refer to the user's movements and behavioral tendencies.

[0918] "Workers" are people who perform work within a factory.

[0919] "Work efficiency" refers to the efficiency with which work is carried out.

[0920] "Suggestions" are advice or recommendations provided to users or workers.

[0921] "Support" refers to assistance and help for users and workers.

[0922] The system for implementing the present invention uses the following hardware and software.

[0923] Hardware used

[0924] Camera (for facial expression recognition)

[0925] Microphone (for voice recognition)

[0926] Sensors (for behavioral pattern recognition)

[0927] Factory robots (for proposals and support execution)

[0928] Software used

[0929] Emotion Recognition Engine

[0930] Data Analysis Software

[0931] Troubleshooting Software

[0932] Proposal Generation AI Model

[0933] Explanation of data processing and data calculation

[0934] Data collection

[0935] The server collects data in real time from cameras, microphones, and sensors, capturing the voices, facial expressions, and behavioral patterns of users and workers.

[0936] emotion recognition

[0937] The server uses an emotion recognition engine to analyze the collected data and recognize the emotions of users and workers, for example, identifying when a user is tired or stressed.

[0938] Data analysis

[0939] The server uses data analysis software to integrate emotion data and work situation data, and evaluates the current work efficiency and potential for trouble, making it possible to make optimal recommendations based on the status of the user and worker.

[0940] Proposal generation

[0941] The server uses a proposal generation AI model to generate suggestions and troubleshooting methods to optimize work efficiency based on the analysis results. The generated suggestions are sent to the factory robots and executed.

[0942] Proposal and support execution

[0943] The factory robot then carries out the suggestions and support received from the server, for example, suggesting that workers take a break or prompting them to recheck their work procedures.

[0944] Specific examples

[0945] For example, if the emotion recognition engine determines that a worker is tired, it will make the following suggestions:

[0946] "Work is progressing well, but I suggest you take a short break."

[0947] This task is a little complicated, so let's review the steps.

[0948] Prompt Sentence Examples

[0949] An example of a prompt sentence to input to the generative AI model is as follows:

[0950] Worker Emotion Data: {Emotion Data}

[0951] Work status data: {Work status data}

[0952] Generate a proposal.

[0953] Based on this prompt, the AI ​​model generates optimal suggestions based on emotional data and work situation data.

[0954] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0955] Step 1:

[0956] The server collects data in real time from cameras, microphones and sensors.

[0957] Input: Real-time data from cameras, microphones, and sensors

[0958] Output: Collected voice, facial expression, and behavioral pattern data

[0959] Specific operations: The camera captures the user's facial expressions, the microphone records audio, and the sensors detect behavioral patterns.

[0960] Step 2:

[0961] The server analyzes the collected data using an emotion recognition engine and recognizes the emotions of users and workers.

[0962] Input: Collected voice, facial expression, and behavioral pattern data

[0963] Output: Recognized emotion data

[0964] Specific behavior: The emotion recognition engine analyzes voice tone, facial expression changes, and behavioral patterns to identify the user's emotional state (e.g., fatigue, stress, joy).

[0965] Step 3:

[0966] The server uses data analysis software to integrate emotional data and work situation data to evaluate current work efficiency and the possibility of trouble.

[0967] Input: Emotion data, work situation data

[0968] Output: Work efficiency evaluation data, trouble possibility evaluation data

[0969] Specific operation: Data analysis software compares emotional data with work situation data to evaluate signs of declines in work efficiency or trouble.

[0970] Step 4:

[0971] The server uses a proposal generation AI model to generate proposals and troubleshooting methods to optimize work efficiency based on the analysis results.

[0972] Input: Work efficiency evaluation data, trouble possibility evaluation data

[0973] Output: Proposal content data

[0974] Specific actions: The proposal generation AI model generates specific suggestions (e.g., suggesting breaks, reconfirming work procedures) based on the evaluation data.

[0975] Step 5:

[0976] The server transmits the generated proposal to the factory robot and has it execute it.

[0977] Input: Proposal content data

[0978] Output: Suggestions implemented

[0979] Specific operation: The factory robot receives the suggestions and implements them to the worker (e.g., suggesting a break by voice, displaying work procedures on a screen).

[0980] Step 6:

[0981] Users can accept suggestions and support from factory robots to improve work efficiency.

[0982] Input: Proposal from factory robot

[0983] Output: Improved work efficiency

[0984] Specific behavior: By taking a break or reconfirming work procedures as suggested, the user's work efficiency improves.

[0985] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0986] The data generation model 58 is a so-called generative AI (Artificial Intelligence).

[0987] An example of the data generation model 58 is ChatGPT (registered trademark)<URL: https: / / openai.com / blog / chatgpt> ) and other generative AIs.

[0988] The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0989] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[0990] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0991] [Second embodiment]

[0992] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0993] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0994] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0995] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0996] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0997] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0998] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0999] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1000] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1001] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1002] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1003] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1004] "Example 1"

[1005] The system of the present invention includes a means for users to register device information for home appliances, communication devices, etc. currently in use, and a means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. based on the registered device information. Specifically, users register information about the devices they are using using a dedicated application or website. The information to be registered includes the device type, manufacturer, model name, purchase date, usage status, etc.

[1006] "Example 2"

[1007] Based on the registered device information, the system retrieves information corresponding to each device from the database and provides guidance and suggestions to the user. For example, in the case of a refrigerator, it provides information such as how to clean it regularly, how to deal with error codes, how to purchase accessories, when to replace it, how to use it energy efficiently, and how to recycle it when discarding it.

[1008] "Example 3"

[1009] Furthermore, the system provides information optimized for the user's usage situation and environment. For example, if a user inquires about the temperature setting of an air conditioner, the system will suggest the optimal temperature setting taking into account the climate of the user's residence, the type of air conditioner, the user's preferences, etc.

[1010] The processing flow of each embodiment will be described below.

[1011] "Example 1"

[1012] Step 1: The user registers information about the device they are using on a dedicated application or website, including the device type, manufacturer, model name, purchase date, and usage status.

[1013] Step 2: The system retrieves the relevant device information from the database based on the registered information.

[1014] Step 3: Based on the acquired information, the system will provide the user with information and suggestions on troubleshooting, maintenance methods, how to purchase accessories, when to replace / replace the product, convenient usage, disposal methods, recycling methods, etc.

[1015] "Example 2"

[1016] Step 1: When a user makes an inquiry about a specific device, the system provides the most appropriate information based on the device information and the user's usage and environment.

[1017] Step 2: For example, if a user inquires about the temperature setting of an air conditioner, the system will suggest the optimal temperature setting taking into consideration the climate of the user's place of residence, the type of air conditioner, the user's preferences, etc.

[1018] Example 1

[1019] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1020] Modern users use a wide variety of electronic devices and communication equipment, and they must individually look up information about each device, such as troubleshooting, maintenance methods, how to purchase accessories, when to replace them, convenient ways to use them, how to dispose of them, and how to recycle them. This requires a lot of time and effort from users, making efficient device management difficult. Furthermore, if the information provided is not optimized for the user's usage situation or environment, it may not be possible to respond appropriately. Furthermore, the lack of an interactive interface makes it difficult to update or adjust the information according to user input.

[1021] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1022] In this invention, the server includes means for registering device information of electronic devices, communication devices, etc. currently in use by the user, means for storing the registered device information in a database, means for sending prompts to the generative AI model based on the registered device information, means for receiving responses from the generative AI model and providing them to the user, and means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. based on the registered device information. This allows users to centrally manage device information and quickly obtain optimized information. Furthermore, an interactive interface allows users to easily update and adjust information based on user input.

[1023] A "user" is an individual or corporation that uses the system to register information about electronic devices or communication devices and receives guidance and suggestions.

[1024] "Electronic devices" is a general term for devices that operate using electricity, such as household appliances, computers, and smartphones.

[1025] "Communication equipment" is a general term for devices used to send and receive information over the Internet or telephone lines.

[1026] "Device information" refers to detailed information about the electronic device or communication device currently being used by the user, such as the type, manufacturer, model name, purchase date, and usage status.

[1027] A "database" is an information system for efficiently storing, managing, and searching registered device information.

[1028] A "generative AI model" is an artificial intelligence algorithm that generates appropriate answers or suggestions based on user input.

[1029] A "prompt sentence" is an input sentence that instructs a generative AI model to generate specific information.

[1030] "Troubleshooting" is the process of diagnosing equipment problems or malfunctions and providing solutions.

[1031] "Maintenance method" refers to the method of regular inspection and care to keep the equipment in good condition.

[1032] "Accessory Purchasing Method" refers to procedures and information for purchasing accessories and parts related to the device.

[1033] "Replacement timing / successor product" is information on the recommended time to replace the currently used device and the new model that will be its successor.

[1034] "Helpfuls" are tips and tricks for using equipment more efficiently and effectively.

[1035] "Disposal method" means the procedure or method for properly disposing of used equipment.

[1036] A "recycling method" is a procedure or method for disposing of used equipment in a reusable form.

[1037] An "interactive interface" is an operating screen or function that allows users to input and obtain information while interacting with the system.

[1038] This invention is a system that allows users to register information about electronic devices and communication devices they are using, and based on that information, provides guidance and suggestions on troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. This system operates in cooperation with the user, terminal, and server.

[1039] Users use a dedicated application or website to enter information about the device they are using. The information to be entered includes the type of device, manufacturer, model name, purchase date, and usage status. For example, when a user registers a "television," they enter "electronic device manufacturer" in "manufacturer," "Model X" in "model name," "January 2022" in "purchase date," and "daily use" in "usage status."

[1040] The device (user's smartphone or PC) sends the entered device information to the server. The information is encrypted before being sent. The server stores the received device information in a database. The database uses a relational database such as MySQL or PostgreSQL.

[1041] The server analyzes the registered device information and sends the necessary information to the generative AI model as a prompt. For example, it generates a prompt such as "Please tell me how to troubleshoot Model X." The generative AI model (e.g., OpenAI's GPT-4) receives the prompt and generates an appropriate answer. The generated answer is sent back to the server.

[1042] The server provides the answer received from the generative AI model to the user, who can then view the answer in an application or on a website. For example, if a user registers a "Model X" television, the server provides the following information:

[1043] Troubleshooting: What to do if the screen is blank

[1044] Maintenance method: How to clean the screen

[1045] How to purchase accessories: How to purchase a remote control

[1046] Replacement time / successor: Information on new models

[1047] Useful tips: How to set up voice control

[1048] Disposal Instructions: How to properly dispose of a television

[1049] How to recycle: Recycling center information

[1050] Example prompts to input to a generative AI model:

[1051] A user has registered a "Model X" TV. Please provide information on troubleshooting, maintenance, purchasing accessories, replacement timing / successor products, convenient usage, disposal, and recycling for this TV.

[1052] This system allows users to centrally manage equipment information, quickly obtain optimized information, and easily update and adjust information based on user input through an interactive interface.

[1053] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1054] Step 1:

[1055] The user enters the device information

[1056] Users access a dedicated application or website and enter information about the device they are using. The information entered includes the type of device, manufacturer, model name, purchase date, and usage status. For example, when a user registers a "television," they enter "electronic device manufacturer" in "manufacturer," "Model X" in "model name," "January 2022" in "purchase date," and "daily use" in "usage status." The entered information is saved on the device.

[1057] Step 2:

[1058] The device sends the entered information to the server.

[1059] The device (user's smartphone or PC) encrypts the entered device information and sends it to the server. The entered data includes the device type, manufacturer, model name, purchase date, usage status, etc. The server receives this data and stores it in a database.

[1060] Step 3:

[1061] The server stores the device information in a database

[1062] The server analyzes the received device information and stores it in a database using a relational database such as MySQL or PostgreSQL. The stored data includes the device type, manufacturer, model name, purchase date, usage status, etc.

[1063] Step 4:

[1064] The server sends a prompt to the generative AI model based on the device information.

[1065] The server analyzes the device information stored in the database and sends the necessary information to the generative AI model as a prompt. For example, it generates a prompt such as "Please tell me how to troubleshoot Model X." The input data is the device information, and the output data is the prompt.

[1066] Step 5:

[1067] The generative AI model generates an answer and sends it back to the server

[1068] A generative AI model (e.g., OpenAI's GPT-4) receives a prompt and generates an appropriate answer. The generated answer is sent back to the server. The input data is the prompt, and the output data is the generated answer.

[1069] Step 6:

[1070] The server provides the generated answer to the user.

[1071] The server displays the answers it receives from the generative AI model in the user's application, where the user can review the answers in the application or on a website. For example, if a user registers a "Model X" television, the server provides the following information:

[1072] Troubleshooting: What to do if the screen is blank

[1073] Maintenance method: How to clean the screen

[1074] How to purchase accessories: How to purchase a remote control

[1075] Replacement time / successor: Information on new models

[1076] Useful tips: How to set up voice control

[1077] Disposal Instructions: How to properly dispose of a television

[1078] How to recycle: Recycling center information

[1079] The input data is the answer that is generated, and the output data is the information that is presented to the user.

[1080] (Application example 1)

[1081] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1082] Conventional systems make it difficult for users to centrally manage information about the equipment they are using and provide comprehensive guidance and suggestions on troubleshooting, maintenance methods, parts purchasing methods, replacement timing, convenient usage, disposal methods, recycling methods, etc. Furthermore, particularly for robots used in factories, there was a lack of means to properly manage and efficiently provide this information. This prevented users from performing appropriate maintenance and troubleshooting, resulting in problems such as a shortened equipment lifespan and reduced efficiency.

[1083] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1084] In this invention, the server includes means for registering information about devices currently in use by users, means for providing and suggesting troubleshooting, maintenance methods, parts purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. based on the registered device information, and means for registering information about robots used in factories and providing and suggesting troubleshooting, maintenance methods, parts purchasing methods, replacement timing, convenient usage, disposal methods, and recycling methods. This allows users to centrally manage device information and perform appropriate maintenance and troubleshooting.

[1085] "Device information" is detailed data about the device currently being used by the user, such as the type, manufacturer, model name, purchase date, and usage status.

[1086] "Troubleshooting" refers to the procedures and methods for diagnosing and resolving problems or malfunctions that occur in equipment.

[1087] "Maintenance methods" are the maintenance and inspection procedures required to maintain the normal operation of equipment.

[1088] "Parts purchasing method" refers to the procedures and methods for purchasing parts required for repairing or maintaining equipment.

[1089] "Time to replace" refers to a suitable time to replace the currently used device with a new device.

[1090] A "successor product" is a next-generation model or improved version of the equipment currently in use.

[1091] "Convenient ways of using" are methods and tips for using equipment more efficiently and effectively.

[1092] "Disposal method" refers to the procedures and methods for properly disposing of used equipment.

[1093] "Recycling method" refers to the procedures and methods for disposing of used equipment in a reusable form.

[1094] "Robots used in factories" are automated machinery used on factory production lines and in workshops.

[1095] The system for implementing this invention allows users to register information about devices in use, and based on that information, provides guidance and suggestions on troubleshooting, maintenance methods, parts purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. In particular, it allows for efficient management of information about robots used in factories, enabling appropriate maintenance and troubleshooting.

[1096] System configuration

[1097] 1. Hardware and Software Use

[1098] Hardware: Smartphone (iOS or Android)

[1099] Software: Mobile application development framework (React Native), database (Firebase), generative AI model (OpenAI GPT-4)

[1100] Program processing explanation

[1101] 1. Data registration

[1102] Users launch the app using their smartphone and enter information about the robots used in the factory (type, manufacturer, model name, purchase date, usage status, etc.).

[1103] The entered data is stored in the Firebase database.

[1104] 2. Troubleshooting and maintenance suggestions

[1105] Users input a specific problem, and a generative AI model (GPT-4) suggests ways to troubleshoot it.

[1106] Suggestions on how to maintain and purchase parts are also provided.

[1107] 3. Suggestions for replacement timing and successor products

[1108] Based on the usage status and purchase date of registered equipment, suggestions will be made on when to replace or which successor product to purchase.

[1109] 4. Information on convenient use, disposal and recycling methods

[1110] Using a generative AI model, it guides users on convenient usage, disposal, and recycling methods.

[1111] Specific examples

[1112] For example, if a user is using a robotic arm model called "XYZ-1000," the following prompt sentence can be input into the generative AI model:

[1113] Prompt Sentence Examples

[1114] Please provide information about the robotic arm "XYZ-1000" you are using. Please include the following information:

[1115] 1. Troubleshooting Methodology

[1116] 2. Maintenance method

[1117] 3. How to purchase parts

[1118] 4. Suggestions for replacement timing and successor products

[1119] 5. Convenient usage

[1120] 6. Disposal Method

[1121] 7. Recycling methods

[1122] By inputting this prompt sentence into the generative AI model, appropriate information can be provided to the user, allowing the user to centrally manage device information and perform appropriate maintenance and troubleshooting.

[1123] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1124] Step 1:

[1125] The user launches the app using a smartphone and enters information about the robots used in the factory (type, manufacturer, model name, purchase date, usage status, etc.).

[1126] Input: Information such as robot type, manufacturer, model name, purchase date, and usage status.

[1127] Output: The entered robot information is saved in the application.

[1128] What happens: When a user enters the required information into the app's input form and presses the "Save" button, the information is sent to the Firebase database.

[1129] Step 2:

[1130] The server receives the robot information stored in the Firebase database and stores it in the database.

[1131] Input: The robot information entered by the user.

[1132] Output: Robot information is saved to the Firebase database.

[1133] Specific operation: The server receives data through the Firebase API and stores it in the database.

[1134] Step 3:

[1135] The user enters a specific problem into the app and requests troubleshooting.

[1136] Input: Details of the specific problem with your robot.

[1137] Output: Suggestions on how to troubleshoot the problem.

[1138] What it does: The user enters details about the problem and presses the "Troubleshoot" button, which sends the information to the server.

[1139] Step 4:

[1140] The server inputs details of the problem into the generative AI model (GPT-4) as a prompt sentence and generates a troubleshooting method.

[1141] Input: A prompt statement containing details about the problem.

[1142] Output: Suggested troubleshooting methods.

[1143] Specific operation: The server sends a prompt to the generative AI model, and the AI ​​generates an appropriate troubleshooting method.

[1144] Step 5:

[1145] The server sends the generated troubleshooting instructions to the user's smartphone.

[1146] Input: Troubleshooting methods from generative AI models.

[1147] Output: Troubleshooting instructions displayed on the user's phone.

[1148] What it does: The server generates troubleshooting instructions and sends them to the user's smartphone, where the app displays them.

[1149] Step 6:

[1150] When a user requests information about maintenance methods or parts purchasing methods, the user inputs the information in the same manner and transmits it to the server.

[1151] Input: Requests for maintenance methods and parts purchasing methods.

[1152] Output: Suggestions on how to maintain and purchase parts.

[1153] Specific operation: The user inputs a request, and the server sends a prompt to the generative AI model to generate information and send it to the user.

[1154] Step 7:

[1155] The server will suggest replacement times and successor products based on the usage status and purchase date of the registered equipment.

[1156] Input: Equipment usage status and purchase date.

[1157] Output: Suggestions for replacement time and successor products.

[1158] Specific operation: The server retrieves usage status and purchase date from the database, sends prompts to the generative AI model to generate suggestions, and sends them to the user.

[1159] Step 8:

[1160] When the user requests convenient usage, disposal, or recycling methods, the information is entered in the same manner and transmitted to the server.

[1161] Input: Requests for convenient use, disposal and recycling methods.

[1162] Output: Suggestions for convenient use, disposal and recycling.

[1163] Specific operation: The user inputs a request, and the server sends a prompt to the generative AI model to generate information and send it to the user.

[1164] Example 2

[1165] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1166] Conventional device management systems make it difficult for users to efficiently obtain information about the home appliances and communication devices they are using and receive appropriate guidance and suggestions. Furthermore, they lack the ability to provide optimal information tailored to the user's usage situation and environment, and the ability to update and adjust information through an interactive interface is limited. This often results in inconvenience for users when it comes to device maintenance and troubleshooting, purchasing accessories, and determining when to replace devices.

[1167] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1168] In this invention, the server includes means for registering device information for home appliances, communication devices, and other devices currently in use by the user, means for retrieving relevant information from a database based on the registered device information, means for sending a prompt message to the generative AI model based on the retrieved information, means for the generative AI model to generate a guidance message, means for transmitting the generated guidance message to the user's device, and means for the user to check the guidance message on the device. This allows the user to efficiently obtain detailed information about the device and receive appropriate guidance and suggestions. Furthermore, optimal information is provided based on the user's usage status and environment, and the information can be updated and adjusted through an interactive interface.

[1169] A "user" is an individual or corporation that uses the system to register device information such as home appliances and communication equipment and receives guidance and suggestions.

[1170] "Device information" refers to detailed information about devices such as home appliances and communication equipment, such as manufacturer names, model numbers, and usage status.

[1171] The "server" is a computer system that receives device information sent by the user, queries a database, uses a generative AI model to generate guidance messages, and sends them to the user's device.

[1172] A "database" is an information management system that stores information about devices such as home appliances and communication equipment, and provides the relevant information in response to inquiries from a server.

[1173] A "generative AI model" is an artificial intelligence model that generates guidance text in natural language based on a prompt text.

[1174] A "prompt sentence" is an input sentence sent to a generative AI model to generate a guidance sentence.

[1175] "Guide text" is a text generated by a generative AI model and provided to users containing information such as how to use and maintain the device.

[1176] A "terminal" is an electronic device such as a smartphone or a personal computer that a user uses to check the guidance text.

[1177] The "interface" refers to the system's operation screen and input means for users to register device information and check guidance messages.

[1178] The present invention is a system that allows users to register device information such as home appliances and communication devices that they are using, and provides appropriate guidance and suggestions based on the registered device information. Specific embodiments of this system are described below.

[1179] Hardware and software used

[1180] server

[1181] The server receives the device information sent by the user, queries the database, generates a message using a generative AI model, and sends it to the user's device. The server is a computer system equipped with a high-performance processor and sufficient memory.

[1182] Database

[1183] The database stores information about devices such as home appliances and communication equipment, and provides the relevant information in response to queries from the server. Database management systems (DBMS) such as MySQL and PostgreSQL are used.

[1184] Generative AI Models

[1185] The generative AI model is an artificial intelligence model that generates guidance text in natural language based on a prompt text. Specifically, OpenAI's GPT-4 is used.

[1186] Terminal

[1187] The terminal is an electronic device such as a smartphone or PC that the user uses to check the information. The terminal must have an internet connection and a browser or dedicated application.

[1188] Data processing and calculation

[1189] Registering device information

[1190] Users use their own terminals to access the system's interface and register by entering information such as device type, manufacturer, and model number.

[1191] Querying the database

[1192] The server receives the device information sent by the user and queries the database for the relevant device information. For example, in the case of a refrigerator, it obtains information such as how to clean it regularly, how to deal with error codes, how to purchase accessories, when to replace it, how to use it energy efficiently, and how to recycle it when discarding it.

[1193] Sending prompts to generative AI models

[1194] The server generates a prompt based on the acquired information and sends it to the generative AI model. An example of a prompt is as follows:

[1195] A user has registered a refrigerator. Generate a message based on the following information:

[1196] Regular cleaning: It is recommended to clean the inside of the refrigerator once a month.

[1197] What to do when the error code appears: If the error code E1 appears, the cooling system needs to be inspected.

[1198] How to purchase accessories: Filters and shelves can be purchased from the official website.

[1199] Guideline for replacement: Consider replacing it after 10 years from purchase.

[1200] Energy-efficient use: Keep your refrigerator temperature set at 3-5 degrees.

[1201] How to recycle: Take to your local recycling center.

[1202] Generate and send a message

[1203] The generative AI model generates a guide message in natural language based on the received prompt. The generated guide message is sent to the user's device via the server. The user can check the guide message on their device and obtain information such as how to use and maintain the device.

[1204] Specific examples

[1205] For example, if a user registers information about a refrigerator, the server retrieves the following information from the database:

[1206] Regular cleaning: It is recommended to clean the inside of the refrigerator once a month.

[1207] What to do when the error code appears: If the error code E1 appears, the cooling system needs to be inspected.

[1208] How to purchase accessories: Filters and shelves can be purchased from the official website.

[1209] Guideline for replacement: Consider replacing it after 10 years from purchase.

[1210] Energy-efficient use: Keep your refrigerator temperature set at 3-5 degrees.

[1211] How to recycle: Take to your local recycling center.

[1212] With this information, we send the following prompt to the generative AI model:

[1213] A user has registered a refrigerator. Generate a message based on the following information:

[1214] Regular cleaning: It is recommended to clean the inside of the refrigerator once a month.

[1215] What to do when the error code appears: If the error code E1 appears, the cooling system needs to be inspected.

[1216] How to purchase accessories: Filters and shelves can be purchased from the official website.

[1217] Guideline for replacement: Consider replacing it after 10 years from purchase.

[1218] Energy-efficient use: Keep your refrigerator temperature set at 3-5 degrees.

[1219] How to recycle: Take to your local recycling center.

[1220] The generative AI model then uses this information to generate the following prompt:

[1221] Thank you for using our refrigerator. Please refer to the following information:

[1222] It is recommended that you clean the inside of your refrigerator once a month.

[1223] If error code E1 appears, the cooling system needs to be inspected.

[1224] Accessories can be purchased from the official website.

[1225] Consider replacing it after 10 years of purchase.

[1226] Keeping your refrigerator temperature between 3 and 5 degrees will make it more energy efficient.

[1227] When disposing of it, please take it to your local recycling center.

[1228] This message is sent to the user's terminal via the server, and the user can check it.

[1229] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1230] Step 1:

[1231] The user registers the device information.

[1232] Users access the system interface using their own terminals and register by entering information such as the device type, manufacturer, model number, etc. The entered device information is then sent to the server.

[1233] Input: Device type, manufacturer, model number, and other information

[1234] Output: Device information sent to the server

[1235] Step 2:

[1236] The server receives the device information and queries the database.

[1237] The server receives the device information sent by the user and queries the database for information on the corresponding device based on the received information.

[1238] Input: Device information sent by the user

[1239] Output: Information about the device retrieved from the database

[1240] Step 3:

[1241] The server retrieves information related to the device from a database.

[1242] The server retrieves information related to the device from the database, such as how to clean the refrigerator regularly, how to deal with error codes, how to purchase accessories, when to replace the refrigerator, how to use it energy-efficiently, and how to recycle it when discarded.

[1243] Input: Database query

[1244] Output: Device related information

[1245] Step 4:

[1246] Based on the information obtained by the server, a prompt sentence is sent to the generative AI model.

[1247] The server generates a prompt sentence based on the acquired information and sends the generated prompt sentence to the generative AI model.

[1248] Input: Device related information

[1249] Output: The prompt sent to the generative AI model

[1250] Step 5:

[1251] A generative AI model generates the guide text.

[1252] The generative AI model generates natural language instructions based on the received prompt, such as instructions on how to clean the refrigerator or how to troubleshoot error codes.

[1253] Input: prompt statement

[1254] Output: Generated guidance text

[1255] Step 6:

[1256] The server transmits the generated message to the user's terminal.

[1257] The server sends the guidance message received from the generative AI model to the user's device, where it is displayed.

[1258] Input: Generated guidance text

[1259] Output: The message sent to the user's device

[1260] Step 7:

[1261] The user checks the message on the device.

[1262] The user checks the guide on their device, which contains detailed instructions on how to use and maintain the device.

[1263] Input: The message sent to the user's device

[1264] Output: Guidance text confirmed by the user

[1265] (Application example 2)

[1266] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1267] In conventional systems, users had to search for maintenance and troubleshooting information for the equipment and communication devices they were using individually, making efficient maintenance difficult. Furthermore, there was a lack of a way to provide real-time maintenance information for machinery used in factories, which led to problems with reduced work efficiency.

[1268] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for registering device information of devices, communication devices, etc. currently in use by the user, means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage methods, disposal methods, recycling methods, etc. based on the registered device information, means for providing maintenance information for machines used in the factory, and means for displaying information through an application installed on a smartphone, smart glasses, or head-mounted display. This enables users to efficiently obtain equipment maintenance information and improve work efficiency in the factory.

[1269] "User" means an individual or corporation that uses the system.

[1270] "Equipment" refers to devices such as home appliances and machinery used in factories.

[1271] "Communication device" refers to a device for sending and receiving information over the Internet or a network.

[1272] "Device information" refers to specific information about a device or communication device, such as model number or usage status.

[1273] "Troubleshooting" refers to procedures and methods for diagnosing and resolving problems with equipment or communications devices.

[1274] "Maintenance method" refers to the maintenance and repair procedures for keeping equipment and communication devices operating normally.

[1275] "Accessory Purchasing Method" refers to a procedure for purchasing accessories or parts related to a device or communications equipment.

[1276] "Replacement time / successor product" refers to the expiration date of equipment or communications devices and the next new model that should be purchased.

[1277] "Convenient ways of using" refers to methods and tips for using equipment and communication devices more efficiently.

[1278] "Disposal method" refers to the procedure for properly disposing of used equipment or communications devices.

[1279] "Recycling method" refers to the procedure for reusing used appliances and communication devices.

[1280] "Machinery used in factories" refers to various types of machinery and equipment used in manufacturing and production industries.

[1281] "Maintenance information" refers to specific information regarding the maintenance and repair of equipment and communications devices.

[1282] A "smartphone" is a mobile device that has the functionality of a mobile phone and is also capable of connecting to the Internet and using applications.

[1283] "Smart glasses" refers to a wearable device in the form of glasses that has the function of displaying information.

[1284] "Head-mounted display" refers to a display device worn on the head.

[1285] "Application" refers to a software program that provides a specific function or service.

[1286] A system for implementing this invention includes means for registering device information such as equipment and communication devices currently in use by a user, means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage methods, disposal methods, recycling methods, etc. based on the registered device information, means for providing maintenance information for machines used in factories, and means for displaying information through an application installed on a smartphone, smart glasses, or head-mounted display.

[1287] The server has a database for users to register information about the devices and communication equipment they use. When a user registers device information, the server stores that information in the database. Next, the server uses a generative AI model to obtain information such as troubleshooting and maintenance methods based on the registered device information and provides it to the user.

[1288] The terminal (smartphone, smart glasses, head-mounted display) displays the information sent from the server. For example, maintenance information for machinery used in a factory can be displayed in real time. This allows users to efficiently perform equipment maintenance.

[1289] As a specific example, when a factory worker wears smart glasses and performs maintenance on an assembly robot, the following information is displayed on the smart glasses' display:

[1290] Regular cleaning method

[1291] How to deal with error codes

[1292] How to purchase accessories

[1293] Estimated replacement time

[1294] Energy-efficient use

[1295] Recycling method when disposing of

[1296] An example of a prompt is as follows:

[1297] "Please provide maintenance information for the assembly robot. The device ID is 'robot123'."

[1298] In this way, the user can efficiently obtain the necessary information and perform maintenance on the device.

[1299] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1300] Step 1:

[1301] Users register information about the devices and communication devices they are using. Using a smartphone or PC, users input device information such as the device model number and usage status. The input information is sent to the server and stored in a database. The input data is device information (e.g., model number, usage status), and the output is the device information stored in the database.

[1302] Step 2:

[1303] The server uses a generative AI model to obtain information such as troubleshooting and maintenance methods based on registered device information. The server obtains device information from the database and inputs that information into the generative AI model. The generative AI model uses prompt statements to generate appropriate maintenance information. The input data is device information (e.g., model number, usage status), and the output is the generated maintenance information.

[1304] Step 3:

[1305] The server sends the generated maintenance information to the user's device. The server sends the maintenance information obtained from the generative AI model to the user's smartphone, smart glasses, or head-mounted display. The input data is the generated maintenance information, and the output is the maintenance information sent to the user's device.

[1306] Step 4:

[1307] The terminal displays the received maintenance information. The user's smartphone, smart glasses, or head-mounted display receives the maintenance information sent from the server and displays it on the display. The input data is the maintenance information sent from the server, and the output is the maintenance information displayed on the terminal.

[1308] Step 5:

[1309] The user performs equipment maintenance based on the displayed maintenance information. The user cleans, repairs, and purchases parts for the equipment while referring to the information displayed on the terminal. The input data is the maintenance information displayed on the terminal, and the output is the maintenance work performed by the user.

[1310] Step 6:

[1311] When a user inputs a new problem or question, the terminal sends the information to the server. The user inputs a new problem or question using the terminal interface and sends it to the server. The input data is the new problem or question input by the user, and the output is the information sent to the server.

[1312] Step 7:

[1313] The server updates and adjusts the information based on new problems or questions from the user. The server receives input from the user, generates new information using a generative AI model, and sends it to the user's device. The input data is the new problem or question from the user, and the output is the updated and adjusted information.

[1314] Example 3

[1315] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1316] Conventional management systems for home appliances and communication devices have struggled to provide optimal information tailored to the user's usage situation and environment. They also lacked specific suggestions that took into account weather information and user preferences, resulting in low user convenience. Furthermore, they lacked the ability to update and adjust information in real time through an interactive interface, making it difficult to respond quickly to user needs.

[1317] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1318] In this invention, the server includes: means for registering device information for home appliances, communication devices, etc. currently in use by a user; means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. based on the registered device information; means for acquiring weather information for the user's place of residence; and means for providing information optimized according to the user's usage situation and environment using a generative AI model. This makes it possible to generate prompt text based on weather information for the user's place of residence and device information, and to provide optimal information using the generative AI model. Furthermore, information can be updated and adjusted according to user input through an interactive interface, enabling quick response to user needs.

[1319] "User" refers to any individual or legal entity that uses the System.

[1320] "Home appliances" refers to all electrical appliances used in the home.

[1321] "Communications equipment" refers to a device for sending and receiving information.

[1322] "Device information" refers to information about home appliances and communication devices currently in use by a user.

[1323] "Troubleshooting" refers to the process of diagnosing device problems and providing solutions.

[1324] "Maintenance method" refers to procedures and methods for maintaining the proper operation of a device.

[1325] "Accessory Purchase Method" refers to the method of purchasing accessories and parts related to a Device.

[1326] "Replacement / Successor" refers to information about when to replace a device or a new model.

[1327] "Helpful tips" refers to tips and tricks for using the device more effectively.

[1328] "Disposal Method" refers to the method for properly disposing of a used device.

[1329] "Recycling method" refers to a method for disposing of used devices in a reusable form.

[1330] "Climate information" refers to environmental data such as the weather and temperature in the user's place of residence.

[1331] A "generative AI model" refers to an algorithm that uses artificial intelligence to analyze data and generate optimal information.

[1332] A "prompt sentence" refers to an instruction sentence to be input into a generative AI model.

[1333] "Interactive interface" refers to an operating screen that allows a user to interact with the system.

[1334] This invention is a system that registers device information of home appliances, communication devices, etc. that a user is using, and provides information optimized according to the user's usage status and environment. Specific embodiments of this system are described below.

[1335] Hardware and software used

[1336] Hardware: Servers, user devices (PCs, smartphones, etc.)

[1337] Software: Generative AI models (e.g., GPT-4), database management systems (e.g., MySQL), weather information APIs (e.g., OpenWeatherMap)

[1338] Program processing flow

[1339] 1. Receiving user inquiries

[1340] Device: The user makes an inquiry about the temperature setting of the air conditioner. For example, the user uses a smartphone app to type, "What is the temperature setting of the air conditioner?"

[1341] Server: Receives the query sent from the terminal. The server analyzes this query and proceeds to the next processing step.

[1342] 2. Obtaining user information

[1343] Server: Using a database management system (e.g., MySQL), obtain information such as the user's place of residence, type of air conditioner, past setting history, etc. For example, obtain that the user's place of residence is "Tokyo," the type of air conditioner is "Daikin," and the past setting history is "25 degrees."

[1344] Server: Temporarily stores the acquired information in memory and proceeds to the next processing step.

[1345] 3. Obtaining weather information

[1346] Server: Uses a weather information API (e.g., OpenWeatherMap) to obtain the current weather information for the user's location. For example, obtains that the current temperature is "30 degrees."

[1347] Server: The acquired weather information is temporarily stored in memory and then the process proceeds to the next step.

[1348] 4. Optimization with generative AI models

[1349] Server: User information and weather information are input into a generative AI model (e.g., GPT-4) to create a prompt that generates the optimal temperature setting. For example, a prompt like "The user lives in Tokyo, and the current temperature is 30 degrees. The air conditioner is a Daikin product, and the user has previously preferred a setting of 25 degrees. Please suggest the optimal temperature setting."

[1350] Generative AI model: Suggests the optimal temperature setting based on a prompt. For example, it generates a result such as, "The current temperature is high, so the optimal temperature setting for the air conditioner is 24 degrees."

[1351] 5. Recommendations for optimal temperature settings

[1352] Server: Response to the user with the optimal temperature setting obtained from the generative AI model. For example, it generates a message such as, "The current temperature is high, so the optimal temperature setting for the air conditioner is 24 degrees."

[1353] On the device: The user confirms the suggested temperature setting. For example, a smartphone app displays a message saying, "The current temperature is high, so the optimal temperature setting for the air conditioner is 24 degrees."

[1354] Specific examples

[1355] User question: "What temperature setting should I set my air conditioner to?"

[1356] Example prompt: "The user lives in Tokyo, and the current temperature is 30 degrees. The air conditioner is a Daikin, and the user has previously preferred a setting of 25 degrees. Please suggest the optimal temperature setting."

[1357] By inputting this prompt into the generative AI model, the system will make specific suggestions such as, "The current temperature is high, so the optimal temperature setting for the air conditioner is 24 degrees."

[1358] The above is a specific embodiment for carrying out the present invention. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1359] Step 1:

[1360] Receiving user inquiries

[1361] Input: A user uses a device (e.g., a smartphone) to make a query such as, "What temperature setting should I set my air conditioner to?"

[1362] Specific behavior: The user enters the inquiry content into the application's input field and presses the send button.

[1363] Output: The query is sent from the terminal to the server.

[1364] Server processing: The server analyzes the received query and prepares to proceed to the next step.

[1365] Step 2:

[1366] Obtaining user information

[1367] Input: The server queries a database management system (e.g., MySQL) for information such as the user's location, type of air conditioner, and past setting history.

[1368] What happens next: The server generates an SQL query and sends it to the database.

[1369] Output: The database returns information such as the user's place of residence (e.g., Tokyo), the type of air conditioner (e.g., Daikin), and past setting history (e.g., 25 degrees).

[1370] Server processing: The acquired information is temporarily stored in memory and the next step is taken.

[1371] Step 3:

[1372] Obtaining weather information

[1373] Input: The server queries a weather information API (e.g., OpenWeatherMap) for the current weather information for the user's location.

[1374] Specific operation: The server generates an API request and sends it to the weather information API.

[1375] Output: Weather information API returns weather information such as the current temperature (e.g., 30 degrees).

[1376] Server processing: The acquired weather information is temporarily stored in memory and the next step is performed.

[1377] Step 4:

[1378] Optimization with generative AI models

[1379] Input: The server generates a prompt based on the user information and weather information.

[1380] Specific behavior: The server creates a prompt sentence that reads, "The user lives in Tokyo, and the current temperature is 30 degrees. The air conditioner is a Daikin product, and the user has previously preferred to set it to 25 degrees. Please suggest the optimal temperature setting."

[1381] Output: The generated prompt sentence is sent to a generative AI model (e.g., GPT-4).

[1382] Processing of the generative AI model: The generative AI model analyzes the prompt and suggests the optimal temperature setting (e.g., 24 degrees).

[1383] Step 5:

[1384] Proposal for optimal temperature settings

[1385] Input: Optimal temperature setting obtained from a generative AI model (e.g., 24°C)

[1386] Specific operation: The server receives the output from the generative AI model and generates a message to reply to the user.

[1387] Output: Produces the message "The current temperature is high, so the optimal temperature setting for your air conditioner is 24 degrees."

[1388] Device action: A message is displayed on the user's device, and the user confirms the suggested temperature setting.

[1389] The above is the specific flow of the program processing of this system.

[1390] (Application example 3)

[1391] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1392] Conventional systems provide limited information about the device the user is using, making it difficult to suggest optimal settings based on the user's current climate and individual preferences. Furthermore, the system lacks information optimized for the user's usage situation and environment, preventing it from fully improving user convenience. This poses the problem of users having to spend time and effort finding the optimal way to use their device.

[1393] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1394] In this invention, the server includes means for registering device information of home appliances, communication devices, etc. currently in use by the user, means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. based on the registered device information, and means for suggesting optimal settings taking into consideration the climate of the user's current location, the type of device, the user's preferences, etc. This allows the user to obtain optimal information according to their own usage situation and environment, and to easily find the optimal way to use the device.

[1395] "User" means an individual or organization that uses the system.

[1396] "Home appliances" is a general term for electrical devices used in the home.

[1397] A "communications device" is a device for sending and receiving information.

[1398] "Device information" is detailed data about the home appliances and communication devices that a user is using.

[1399] "Troubleshooting" is the process of diagnosing device problems and providing solutions.

[1400] "Maintenance methods" are procedures and methods for properly maintaining a device.

[1401] "Accessory Purchase Method" means a procedure for purchasing accessories or additional parts related to a device.

[1402] "Time to replace" is the appropriate time to replace a device with a new one.

[1403] A "successor" is a next-generation model or improved version of a device you are currently using.

[1404] "Convenient usage" is a way to use a device efficiently and effectively.

[1405] "Disposal Instructions" are procedures for properly disposing of used devices.

[1406] A "recycling method" is a procedure for converting a used device into a reusable form.

[1407] "Current location climate" refers to the weather and temperature conditions in the location where the user is currently located.

[1408] "Device type" refers to the category and model of consumer electronics or communications equipment used.

[1409] "User preferences" are the tendency of a user to prefer a particular setting or environment.

[1410] "Optimal settings" are device settings that are most suitable for the user's usage and environment.

[1411] A system for implementing this invention includes means for registering device information for home appliances, communication devices, etc. currently in use by a user, means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage methods, disposal methods, recycling methods, etc. based on the registered device information, and means for suggesting optimal settings taking into consideration the climate of the user's current location, the type of device, the user's preferences, etc.

[1412] Program processing explanation

[1413] The server has a database for registering information about devices used by users. Users enter device information using a device such as a smartphone or PC and send it to the server. The server provides information such as troubleshooting and maintenance methods based on the registered device information.

[1414] Additionally, the server uses external services such as the Weather API to obtain weather data for the user's current location. This allows the server to obtain real-time weather information for the user's current location. The server calculates and suggests optimal settings to the user based on the user's preferences and device type.

[1415] As a specific example, consider a case where a user inquires about the interior temperature setting of an autonomous vehicle. The server obtains climate data for the user's current location and suggests the optimal interior temperature, taking into account the user's past preferred temperature settings. This information is provided to the user through a smartphone application.

[1416] Hardware and software used

[1417] Hardware: Servers, smartphones, PCs

[1418] Software: Database management system, Weather API, user interface application

[1419] Prompt Sentence Examples

[1420] If the user is in Tokyo, suggest the optimal car temperature based on the current temperature. The user prefers a warm environment.

[1421] Using this prompt, the generative AI model can suggest optimal temperature settings that take into account the user's preferences and the current climate.

[1422] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1423] Step 1:

[1424] Users use devices such as smartphones and personal computers to input device information for home appliances, communication devices, etc., and send it to a server.

[1425] Input: Device information (e.g., device name, model number, start date, etc.)

[1426] Output: Device information sent to the server

[1427] Specific operation: The user opens the dedicated application, enters device information into the input form, and presses the "Submit" button.

[1428] Step 2:

[1429] The server registers the received device information in a database.

[1430] Input: Device information sent by the user

[1431] Output: Device information stored in the database

[1432] Specific operation: The server analyzes the received device information and stores it in a database in an appropriate format.

[1433] Step 3:

[1434] The user inquires about the vehicle temperature setting.

[1435] Input: User query (e.g., "What is the ideal car interior temperature?")

[1436] Output: The query sent to the server

[1437] Specific behavior: The user enters a question into an inquiry form within the application and presses the "Submit" button.

[1438] Step 4:

[1439] The server calls the Weather API to get weather data for the user's current location.

[1440] Input: User's current location (e.g., latitude, longitude)

[1441] Output: Current location climate data (e.g. temperature, humidity)

[1442] Specific operation: The server calls the Weather API based on the user's current location information and obtains real-time weather data.

[1443] Step 5:

[1444] The server calculates the optimal settings based on user preferences and device type.

[1445] Input: Climate data, user preferences, device type

[1446] Output: Optimal setting (e.g. recommended interior temperature)

[1447] What it does: The server runs an algorithm that calculates the optimal settings based on the weather data, device information, and user preferences.

[1448] Step 6:

[1449] The server notifies the user of the calculation results.

[1450] Input: Optimal settings (e.g. recommended interior temperature)

[1451] Output: Optimal configuration information displayed on the user's device

[1452] Specific behavior: The server sends the calculation result to the user's device and displays a notification within the application.

[1453] Step 7:

[1454] The user reviews the proposed settings and applies them to the device if necessary.

[1455] Input: Notification content from the server

[1456] Output: Settings applied to the device

[1457] What happens: The user sees the notification in the app and takes action to apply the suggested settings to their device.

[1458] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1459] "Example 1"

[1460] One embodiment of the present invention provides a system including an emotion engine that recognizes a user's emotions. The system includes a means for registering device information, such as a home appliance or communication device, currently used by the user, and a means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc., based on the registered device information. The emotion engine analyzes information such as the user's voice, facial expressions, and behavioral patterns to recognize emotions, and adjusts the content and expression of the guidance and suggestions according to the emotions.

[1461] "Example 2"

[1462] For example, if a user is having trouble operating an air conditioner, the system can sense confusion or dissatisfaction from the user's tone of voice and facial expression. The emotion engine analyzes this information and recognizes that the user is confused. The system then provides a more detailed and easy-to-understand explanation based on the user's emotion. Also, if the user expresses joy or satisfaction, the system can recognize that emotion and provide the user with further suggestions and information.

[1463] "Example 3"

[1464] One embodiment of the present invention provides a system including an emotion engine that recognizes a user's emotions. The system includes a means for registering device information, such as a home appliance or communication device, currently used by the user, and a means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc., based on the registered device information. The emotion engine analyzes information such as the user's voice, facial expressions, and behavioral patterns to recognize emotions, and adjusts the content and expression of the guidance and suggestions according to the emotions.

[1465] The processing flow of each embodiment will be described below.

[1466] "Example 1"

[1467] Step 1: The user registers the information of the device they are using in the system.

[1468] Step 2: Based on the registered device information, the system will provide guidance and suggestions on troubleshooting, maintenance methods, how to purchase accessories, when to replace / replace the device, convenient usage, disposal methods, recycling methods, etc.

[1469] Step 3: The emotion engine analyzes information such as the user's voice, facial expressions, and behavioral patterns to recognize emotions.

[1470] Step 4: The emotion engine adjusts the content and expression of the guidance and suggestions according to the recognized emotion.

[1471] "Example 2"

[1472] Step 1: When a user is having trouble operating the air conditioner, the system detects confusion or dissatisfaction from the user's tone of voice and facial expression.

[1473] Step 2: The emotion engine analyzes this information and recognizes that the user is confused.

[1474] Step 3: The system responds to the user's feelings by providing a more detailed and understandable explanation.

[1475] Step 4: If the user expresses happiness or satisfaction, the system recognizes the emotion and provides further suggestions or information to the user.

[1476] Example 1

[1477] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1478] Conventional device management systems allow users to register information about their electronic devices and communication devices, but they lack sufficient guidance and suggestions on troubleshooting, maintenance methods, how to purchase accessories, when to replace / replace, convenient usage, disposal methods, and recycling methods. Furthermore, since the guidance and suggestions do not take into account the user's feelings, user satisfaction is reduced. Furthermore, the lack of information provision optimized for the user's usage situation and environment, and the lack of information updating and adjustment through an interactive interface, reduces user convenience.

[1479] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1480] In this invention, the server includes means for registering device information of electronic devices, communication devices, etc. currently in use by the user, means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage methods, disposal methods, recycling methods, etc. based on the registered device information, means for recognizing emotions by analyzing information such as the user's voice, facial expressions, and behavioral patterns, and means for adjusting the content and expression of the guidance and suggestions according to the recognized emotions. This enables optimal guidance and suggestions that take the user's emotions into consideration, improving user satisfaction and convenience.

[1481] A "user" is an individual or corporation that uses the system to register information about electronic devices or communication devices and receives guidance and suggestions.

[1482] "Electronic devices" refers to all devices that users use on a daily basis, such as household appliances and communication devices.

[1483] "Communications equipment" refers to devices used to send and receive information over the Internet or telephone lines.

[1484] "Device information" refers to detailed information such as the type, manufacturer, model name, purchase date, and usage status of electronic devices or communication devices.

[1485] "Troubleshooting" refers to the process of diagnosing problems with electronic or communications devices and providing solutions.

[1486] "Maintenance method" refers to the maintenance procedures for maintaining the normal operation of electronic equipment and communication devices.

[1487] "Accessory purchasing method" refers to the procedure for purchasing accessories and parts related to electronic devices and communication devices.

[1488] "Replacement time / successor product" refers to the time when it is recommended to replace electronic devices or communication devices currently in use and the successor model.

[1489] "Convenient usage" refers to ways to use electronic devices and communication devices more effectively and efficiently.

[1490] "Disposal method" refers to the procedure for properly disposing of used electronic or communications equipment.

[1491] "Recycling method" refers to a procedure for reusing or recycling used electronic or communication devices.

[1492] "Means for recognizing emotions" refers to technology that identifies emotions by analyzing information such as the user's voice, facial expressions, and behavioral patterns.

[1493] "Means for adjusting the content and expression of guidance and suggestions" refers to technology that changes the content and expression of information provided based on the recognized user emotions.

[1494] An "interactive interface" refers to an interface that allows a user to interact with a system.

[1495] The system of the present invention allows users to register information about electronic devices and communication devices they are using, and based on that information, provides guidance and suggestions on troubleshooting, maintenance methods, how to purchase accessories, when to replace them / what to buy next, convenient ways to use them, how to dispose of them, how to reuse them, etc. Furthermore, it includes a function to recognize the user's emotions and adjust the content and expression of the guidance and suggestions according to those emotions.

[1496] System configuration

[1497] Registering user device information

[1498] Users use dedicated applications or websites to enter information about the electronic devices and communication devices they use, such as the type of device (e.g., smartphone, refrigerator), manufacturer, model name, purchase date, and usage status (e.g., daily use, weekend use).

[1499] Receiving and storing device information

[1500] The server receives the device information sent from the terminal. The server stores the received information in a database. A relational database (e.g., MySQL, PostgreSQL) is used for the database.

[1501] Analyzing device information and generating guidance and suggestions

[1502] The server analyzes the device information stored in the database, using programming languages ​​such as Python and R and applying machine learning algorithms. Based on the analysis results, it generates guidance and suggestions on troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, and recycling methods.

[1503] Emotion recognition by emotion engine

[1504] The emotion engine recognizes emotions by analyzing information such as the user's voice, facial expressions, and behavioral patterns. It uses voice recognition software (e.g., Google Cloud Speech-to-Text) for voice recognition, facial expression recognition software (e.g., Microsoft Azure Face API) for facial expression recognition, and behavioral pattern analysis software (e.g., Amazon Rekognition) for behavioral pattern analysis. This information is collected in real time while the user is using the application or website.

[1505] Emotionally tailored guidance and suggestions

[1506] The server adjusts the content and expression of the guidance and suggestions based on the user's emotions recognized by the emotion engine. For example, if the user is feeling stressed, the server can provide more concise and easy-to-understand guidance. If the user is feeling relaxed, the server can include more detailed information and additional suggestions.

[1507] Providing guidance and suggestions

[1508] The server sends the tailored guidance and suggestions to the user's device, where the user can view the provided guidance and suggestions on the application or website.

[1509] Specific examples

[1510] For example, if a user feels that their smartphone battery is draining quickly, the following steps will be taken to guide them.

[1511] 1. The user opens the application and registers their smartphone information (manufacturer: common smartphone manufacturer, model name: common model name, purchase date: January 2021, usage status: daily use).

[1512] 2. The server receives the registered information and stores it in a database.

[1513] 3. The server analyzes the smartphone information and suggests battery troubleshooting methods (e.g., limiting background apps, calibrating the battery).

[1514] 4. The emotion engine recognizes from the user's voice that they are feeling stressed.

[1515] 5. The server should be tailored to provide simple and easy-to-understand guidance.

[1516] 6. The server provides the tailored guidance to the user, who confirms it on the application.

[1517] Prompt Sentence Examples

[1518] "Please describe a system that allows users to register information about the device they are using and provides troubleshooting and maintenance guidance based on that information. Please also include a function that recognizes the user's emotions and adjusts the guidance content accordingly."

[1519] The above is a specific embodiment of the system of the present invention.

[1520] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1521] Step 1:

[1522] The user registers the device information.

[1523] Users use dedicated applications or websites to enter information about the electronic devices or communication devices they are using, including the device type, manufacturer, model name, purchase date, usage status, etc. The entered data is then sent from the device to a server.

[1524] Step 2:

[1525] The server receives the registered device information and stores it in a database.

[1526] The server receives device information sent from the terminal. The received data is stored in a database. A relational database (e.g., MySQL or PostgreSQL) is used for the database. The input data is device information, and the output is the device information stored in the database.

[1527] Step 3:

[1528] The server analyzes the device information and generates appropriate guidance and suggestions.

[1529] The server analyzes the device information stored in the database. For the analysis, it uses programming languages ​​such as Python and R and applies machine learning algorithms. Based on the analysis results, it generates guidance and suggestions on troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, and recycling methods. The input data is the device information, and the output is the generated guidance and suggestions.

[1530] Step 4:

[1531] The emotion engine recognizes the user's emotions.

[1532] The emotion engine recognizes emotions by analyzing information such as the user's voice, facial expressions, and behavioral patterns. It uses voice recognition software (e.g., Google Cloud Speech-to-Text) for voice recognition, facial expression recognition software (e.g., Microsoft Azure Face API) for facial expression recognition, and behavioral pattern analysis software (e.g., Amazon Rekognition) for behavioral pattern analysis. The input data are the user's voice, facial expressions, and behavioral patterns, and the output is the recognized emotion.

[1533] Step 5:

[1534] The server adjusts the content and expression of the guidance and suggestions according to the emotion.

[1535] The server adjusts the content and expression of the guidance and suggestions based on the user's emotions recognized by the emotion engine. For example, if the user is feeling stressed, the server will provide more concise and easy-to-understand guidance. The input data are the recognized emotions and the generated guidance and suggestions, and the output is the adjusted guidance and suggestions.

[1536] Step 6:

[1537] The server provides tailored guidance and suggestions to the user.

[1538] The server sends the tailored guidance and suggestions to the user's device, and the user can view the provided guidance and suggestions on the application or website. The input data are the tailored guidance and suggestions, and the output is the guidance and suggestions provided to the user.

[1539] (Application example 1)

[1540] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1541] Conventional maintenance systems for machinery and equipment in factories allow users to register information about devices in use and provide guidance and suggestions on troubleshooting and maintenance methods, but they lack suggestions that take user emotions into consideration or suggestions for reducing worker stress and fatigue, resulting in issues such as insufficient improvement of work efficiency and worker health management.

[1542] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1543] In this invention, the server includes: means for registering device information of home appliances, communication devices, etc. currently in use by a user; means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage methods, disposal methods, recycling methods, etc. based on the registered device information; means including an emotion engine for recognizing the user's emotions; means for adjusting the content and expression of the guidance and suggestions according to the user's emotions recognized by the emotion engine; means for providing and suggesting maintenance, troubleshooting, suggestions for part replacement timing, efficient usage methods, disposal and recycling methods, etc. for machinery and equipment in a factory; and means for recognizing the emotions of workers and making suggestions to reduce stress and fatigue. This enables optimal guidance and suggestions according to the user's emotions, thereby realizing efficient maintenance of machinery and equipment in a factory and health management of workers.

[1544] A "user" is an individual or corporation that uses the system to register device information such as home appliances and communication devices and receives guidance and suggestions.

[1545] "Device information" refers to information such as the type, manufacturer, model name, purchase date, and usage status of home appliances and communication devices currently in use by the user.

[1546] "Troubleshooting" is the process of diagnosing device malfunctions and providing solutions.

[1547] "Maintenance Method" means the maintenance and inspection procedures for maintaining the normal operation of the Device.

[1548] "Accessory purchase method" refers to the procedure for purchasing accessories and parts related to the device.

[1549] "Replacement / successor product" refers to the expiration date of the device, the time to upgrade to a new model, and its successor product.

[1550] "Convenient usage" is a way to use a device more efficiently and effectively.

[1551] "Disposal Instructions" are procedures for properly disposing of used devices.

[1552] A "recycling method" is a procedure for disposing of used devices in a reusable form.

[1553] An "emotion engine" is a system that recognizes emotions by analyzing information such as the user's voice, facial expressions, and behavioral patterns.

[1554] "Machinery and equipment in the factory" refers to production machinery and related equipment used in the factory.

[1555] "Workers" are workers who operate and manage machinery and equipment within a factory.

[1556] "Stress" is the mental or physical strain that workers experience while working.

[1557] "Fatigue" refers to the physical and mental exhaustion that workers experience while working.

[1558] As an embodiment of the present invention, a system is provided that provides guidance and suggestions for maintenance of machinery and equipment in a factory, troubleshooting, suggesting parts replacement times, efficient usage methods, disposal and recycling methods, etc. Furthermore, the system includes a function to recognize the emotions of workers and make suggestions to reduce stress and fatigue.

[1559] System Program

[1560] The server includes the following means:

[1561] 1. How to register device information

[1562] The user registers device information for the home appliances and communication devices they are using. Specifically, they enter information such as the device type, manufacturer, model name, purchase date, and usage status.

[1563] 2. Troubleshooting Methods

[1564] Based on the registered device information, the system diagnoses device failures and malfunctions and provides solutions.

[1565] 3. Maintenance method proposals

[1566] We will suggest maintenance and inspection procedures and part replacement times to maintain normal operation of the device.

[1567] 4. Means including an emotional engine

[1568] Emotions are recognized by analyzing information such as the user's voice, facial expressions, and behavioral patterns. Specifically, the user's facial expressions and voice are captured using a camera and microphone, and then analyzed using an emotion recognition engine (EmotionRecognizer).

[1569] 5. Means for tailoring guidance and suggestions according to emotions

[1570] The emotion engine adjusts the content and expression of guidance and suggestions depending on the user's emotions. For example, if the user is feeling stressed, the engine suggests taking a break.

[1571] 6. Maintenance methods for machinery and equipment in the factory

[1572] Provides guidance and suggestions on maintenance, troubleshooting, part replacement timing, efficient usage, disposal and recycling methods for machinery and equipment within the factory.

[1573] 7. Means of Recognizing Worker Emotions

[1574] The system recognizes the emotions of workers and makes suggestions to reduce stress and fatigue. Specifically, it uses cameras and microphones to capture the facial expressions and voices of workers, and analyzes them using an emotion recognition engine.

[1575] Hardware and software used

[1576] Hardware

[1577] Camera: Used to capture the facial expressions of the workers.

[1578] Microphone: Used to capture the voice of the worker.

[1579] Computer: The environment in which a program runs.

[1580] software

[1581] OpenCV: Image processing library used to acquire and display camera images.

[1582] EmotionRecognizer: An emotion recognition engine used to analyze the emotions of workers.

[1583] maintenance_suggestions: A module for generating maintenance suggestions.

[1584] Specific examples

[1585] For example, the system can register information about equipment such as conveyor belts and robotic arms in a factory and suggest regular maintenance methods and part replacement times. It can also suggest breaks if workers are feeling stressed, and encourage them to drink more water if they are feeling tired.

[1586] Prompt Sentence Examples

[1587] Develop a system that allows users to register information about the factory equipment they are using, and based on that information, provides guidance and suggestions on troubleshooting, maintenance methods, part replacement times, efficient usage methods, disposal and recycling methods, etc. Furthermore, add a function that recognizes the emotions of workers and makes suggestions to reduce stress and fatigue.

[1588] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1589] Step 1:

[1590] Register device information such as home appliances and communication devices currently used by the user.

[1591] Input: Information such as device type, manufacturer, model name, purchase date, and usage status.

[1592] Processing: The user enters device information through a dedicated application or website and sends it to the server.

[1593] Output: Device information stored on the server.

[1594] Step 2:

[1595] The server performs troubleshooting based on registered device information.

[1596] Input: Device information stored on the server.

[1597] Processing: The server parses the device information and checks it against a database of known troubleshooting solutions.

[1598] Output: Trouble diagnosis and solution.

[1599] Step 3:

[1600] The server will suggest a maintenance method.

[1601] Input: Device information stored on the server.

[1602] Processing: The server calculates the appropriate maintenance method based on device usage and the manufacturer's recommended maintenance schedule.

[1603] Output: Maintenance suggestions.

[1604] Step 4:

[1605] The server uses an emotion engine to recognize the user's emotion.

[1606] Input: User facial and voice data obtained from the camera and microphone.

[1607] Processing: The server uses EmotionRecognizer to analyze the acquired data and recognize the user's emotions.

[1608] Output: Perceived user sentiment.

[1609] Step 5:

[1610] The server adjusts the content and expression of the guidance and suggestions according to the emotion.

[1611] Input: Perceived user sentiment, troubleshooting results, and maintenance suggestions.

[1612] Processing: The server adjusts the content and expression of the guidance and suggestions according to the user's emotions. For example, if the user is feeling stressed, the server suggests taking a break.

[1613] Output: Adjusted guidance and suggestions.

[1614] Step 6:

[1615] The server provides guidance and suggestions for maintenance of machinery and equipment within the factory.

[1616] Input: Device information of machines and equipment in the factory.

[1617] Processing: The server analyzes device information and calculates maintenance methods, troubleshooting, part replacement times, efficient usage methods, disposal and recycling methods, etc.

[1618] Output: Maintenance guide and suggestions for machinery and equipment within the factory.

[1619] Step 7:

[1620] The server recognizes the worker's emotions and makes suggestions to reduce stress and fatigue.

[1621] Input: Facial expression and voice data of the worker obtained from the camera and microphone.

[1622] Processing: The server uses EmotionRecognizer to analyze the acquired data and recognize the worker's emotions. Based on the recognized emotions, it makes suggestions for reducing stress and fatigue.

[1623] Output: Suggestions based on the worker's emotions.

[1624] Example 2

[1625] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1626] Conventional support systems for home appliances and communication devices only provide uniform information for problems users face, and have the problem of being unable to respond flexibly to the user's specific situation or emotional state. Also, when a user is confused or in a particular emotional state, the system is unable to recognize that state and provide appropriate guidance or suggestions, which leads to a poor user experience.

[1627] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for registering device information of home appliances, communication devices, etc. currently used by the user, a means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage methods, disposal methods, recycling methods, etc. based on the registered device information, a means for acquiring user voice data and converting it into text data using voice recognition technology, a means for analyzing the user's tone of voice and facial expressions to recognize the user's emotional state, and a means for adjusting the information and suggestions according to the user's emotional state. This enables flexible response according to the user's specific situation and emotional state.

[1628] "Device information" is information about a specific device, such as a home appliance or communication device, that the user is using.

[1629] "Troubleshooting" refers to procedures or methods for diagnosing and resolving device problems or malfunctions.

[1630] A "maintenance method" is a procedure for regular maintenance and inspection to keep a device operating normally.

[1631] "Accessory Purchase Method" means the process or method for purchasing additional parts or accessories related to a Device.

[1632] "Replacement time / successor product" is a guideline or information for replacing a device with a new model or successor when the device is nearing the end of its useful life.

[1633] "Hints" are tips and tricks for using your device more efficiently and effectively.

[1634] "Disposal method" means the procedure or method for properly disposing of a used device.

[1635] A "recycling method" is a procedure or method for disposing of used devices in a reusable form.

[1636] "Voice data" is digital data that records the user's speech.

[1637] "Voice recognition technology" is a technology that analyzes voice data and converts it into text data.

[1638] "Text data" is character information converted using voice recognition technology.

[1639] "Tone of voice" refers to changes in pitch and intensity of a voice that indicate the emotion or intention contained in a user's speech.

[1640] "Facial expressions" refer to emotions and intentions expressed by the user's facial movements and expressions.

[1641] "Emotional state" refers to a user's current psychological state or feelings.

[1642] "Means for adjusting guidance and suggestions" refers to methods and techniques for changing the content of information and guidance provided depending on the emotional state of the user.

[1643] This invention is a system that allows users to register device information for home appliances, communication devices, etc. currently in use, and based on the registered device information, provides guidance and suggestions on troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. It also includes a function to acquire user voice data, convert it into text data using voice recognition technology, recognize the user's emotional state by analyzing the user's tone of voice and facial expressions, and adjust the guidance and suggestions according to the user's emotional state.

[1644] Hardware and software used

[1645] Hardware

[1646] Device: The device used by the user (smartphone, tablet, computer, etc.)

[1647] Server: A server that handles databases and processing

[1648] Microphone: A microphone for capturing the user's voice

[1649] Camera: A camera for capturing the user's facial expressions

[1650] software

[1651] Speech recognition software: Software for converting voice data into text data (e.g., Google Speech-to-Text API)

[1652] Speech synthesis software: Software that translates text data into audio (e.g., Google Text-to-Speech API)

[1653] Emotion analysis engine: Software that analyzes a user's tone of voice and facial expressions to recognize their emotional state (e.g., OpenAI's GPT-4 API)

[1654] Database manipulation libraries: Libraries for retrieving information from databases (e.g., SQLAlchemy)

[1655] Program processing

[1656] The server registers information about the device being used by the user in a database. When the user asks a question about the device, the terminal captures the question as voice data and converts it into text data using voice recognition software. The converted text data is sent to the server, which retrieves the relevant device information from the database. The retrieved information is then sent to the terminal, which then displays the information to the user or provides audio guidance.

[1657] Furthermore, the device acquires data using a microphone and camera to analyze the user's tone of voice and facial expressions. The acquired data is sent to an emotion analysis engine, which analyzes the user's emotional state. The analysis results are sent to a server, which adjusts the guidance and suggestions according to the user's emotional state. The adjusted guidance and suggestions are sent to the device and are displayed or spoken to the user.

[1658] Specific examples

[1659] For example, if a user is having trouble operating an air conditioner, they might say, "The air conditioner isn't working." The device captures this speech and uses voice recognition software to convert it into text data saying, "The air conditioner isn't working." The server receives this text data and retrieves troubleshooting information for the air conditioner from a database. The retrieved information is sent to the device, which then displays a message saying, "Please make sure the air conditioner is turned on."

[1660] At the same time, the device analyzes the user's tone of voice and facial expression, and conveys the user's confusion to the emotion analysis engine. The emotion analysis engine analyzes this information and notifies the server that the user is confused. The server then responds by providing a more detailed explanation. For example, it might generate additional guidance such as "Please also check the batteries in the remote control" and send it to the device. The device then displays or audibly informs the user of this additional guidance.

[1661] Prompt Sentence Examples

[1662] "If a user is having trouble operating the air conditioner, how can I provide them with guidance? If I can tell from the user's tone of voice and facial expression that they are confused, how can I provide more detailed instructions?"

[1663] In this way, the system can respond flexibly to the user's specific situation and emotional state.

[1664] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1665] Step 1:

[1666] The user asks a question about the device.

[1667] Input: User's spoken question (e.g., "My air conditioner isn't working. What should I do?")

[1668] Output: Audio data

[1669] Specific operation: If the user is having trouble operating the air conditioner, they ask, "The air conditioner isn't working."

[1670] Step 2:

[1671] The terminal acquires the user's question as voice data.

[1672] Input: User's voice data

[1673] Output: Recorded audio data

[1674] Specific operation: The device uses the built-in microphone to record the user's voice.

[1675] Step 3:

[1676] The terminal converts the voice data into text data.

[1677] Input: Recorded audio data

[1678] Output: Text data (e.g., "My air conditioner isn't working. What should I do?")

[1679] What it does: The device uses voice recognition software (e.g., Google Speech-to-Text API) to convert voice data into text data.

[1680] Step 4:

[1681] The terminal transmits the text data to the server.

[1682] Input: Text data

[1683] Output: Text data as an HTTP request

[1684] Specific operation: The terminal sends the converted text data to the server as an HTTP request.

[1685] Step 5:

[1686] The server retrieves the relevant device information from the database.

[1687] Input: Text data (e.g., "The air conditioner isn't working")

[1688] Output: Device information (e.g. "Please check if the air conditioner is turned on")

[1689] Specific operation: The server uses SQLAlchemy to retrieve air conditioner troubleshooting information from the database.

[1690] Step 6:

[1691] The server transmits the acquired information to the terminal.

[1692] Input:Device information

[1693] Output: Device information in JSON format

[1694] Specific operation: The server sends the acquired information to the terminal in JSON format.

[1695] Step 7:

[1696] The terminal displays or audibly guides the user through the information.

[1697] Input: Device information in JSON format

[1698] Output: Displayed information or audio prompts

[1699] Specific operation: The device displays the acquired information on the screen or uses voice synthesis software (e.g., Google Text-to-Speech API) to provide voice guidance.

[1700] Step 8:

[1701] The device analyzes the user's tone of voice and facial expressions.

[1702] Input: User's voice tone and facial expression data

[1703] Output: Parsed emotion data

[1704] What it does: The device uses the built-in camera and microphone to record the user's facial expressions and tone of voice.

[1705] Step 9:

[1706] The device sends the analysis results to the emotion engine.

[1707] Input: Recorded facial expressions and tone of voice data

[1708] Output: Sending data to the emotion engine

[1709] Specific operation: The device sends the recorded data to the emotion engine.

[1710] Step 10:

[1711] An emotion engine analyzes the user's emotional state.

[1712] Input: facial expression and tone of voice data

[1713] Output: Parsed emotional state (e.g., "User is confused")

[1714] How it works: The emotion engine uses OpenAI's GPT-4 API to analyze the user's emotional state.

[1715] Step 11:

[1716] The emotion engine sends the analysis results to the server.

[1717] Input: Parsed emotional state

[1718] Output: Send analysis results to the server

[1719] Specific operation: The emotion engine sends the analysis results in JSON format to the server.

[1720] Step 12:

[1721] The server generates additional guidance and suggestions according to the user's emotional state.

[1722] Input: Parsed emotional state (e.g., "User is confused")

[1723] Output: Additional instructions and suggestions (e.g., "Please also check the batteries in the remote control.")

[1724] Specific behavior: The server generates a more detailed explanation if the user is confused.

[1725] Step 13:

[1726] The server sends additional guidance and suggestions to the terminal.

[1727] Input: Additional guidance and suggestions

[1728] Output: Additional instructions and suggestions in JSON format

[1729] Specific operation: The server sends the generated additional guidance and suggestions to the terminal in JSON format.

[1730] Step 14:

[1731] The terminal displays or audibly provides additional guidance and suggestions to the user.

[1732] Input: Additional instructions and suggestions in JSON format

[1733] Output: Displayed additional information or audio information

[1734] Specific actions: The device will display additional guidance and suggestions on the screen or provide audible guidance using speech synthesis software.

[1735] (Application example 2)

[1736] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1737] Conventional guidance and recommendation systems for home appliances and communication devices provide uniform information without considering the user's emotions, which can lead to confusion and dissatisfaction. Furthermore, the inability to respond appropriately to the user's emotions has led to a lack of improvement in the user experience. Furthermore, in physical stores, there is a lack of support for store clerks to provide appropriate guidance and suggestions to customers.

[1738] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1739] In this invention, the server includes a means for registering device information of home appliances, communication devices, etc. currently used by the user, a means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. based on the registered device information, and a means for analyzing emotions from the user's tone of voice and facial expressions and providing guidance and suggestions according to the emotions. This enables appropriate guidance and suggestions according to the user's emotions, which is expected to improve the user experience. It also enables support for store clerks in physical stores to provide appropriate guidance and suggestions to customers.

[1740] "Device information" is information about appliances such as home appliances and communication devices currently in use by the user.

[1741] "Troubleshooting" is the process of diagnosing device problems or malfunctions and providing solutions.

[1742] A "maintenance method" is a procedure for maintaining and inspecting a device to ensure that it operates normally.

[1743] "Accessory Purchasing Method" refers to the procedure for purchasing accessories and parts related to a device.

[1744] "Replacement / Successor" is information about the expiration date of a device or when it's time to upgrade to a new model.

[1745] "Convenient usage" is a way to use a device efficiently and effectively.

[1746] "Disposal Instructions" are procedures for properly disposing of used devices.

[1747] A "recycling method" is a procedure for disposing of used devices in a reusable form.

[1748] "Voice tone" refers to voice characteristics that express the pitch, strength, and emotion of a user's voice.

[1749] "Facial expressions" refer to emotions and reactions shown by the user's facial movements and expressions.

[1750] "Emotion analysis" is the process of identifying and understanding a user's emotions from their tone of voice and facial expressions.

[1751] "Guide and Suggestion" is information and advice provided to the user.

[1752] An "interactive interface" is an operation screen or input means that allows a user to interact with a system in a two-way manner.

[1753] A system for implementing this invention includes means for registering device information such as home appliances and communication equipment currently in use by a user, means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage methods, disposal methods, recycling methods, etc. based on the registered device information, and means for analyzing the user's emotions from their tone of voice and facial expression and providing guidance and suggestions according to their emotions.

[1754] System Program

[1755] The system program uses the following hardware and software:

[1756] Hardware: Smart glasses (with built-in camera and microphone)

[1757] software:

[1758] EmotionRecognizer: A library for recognizing emotions from facial images

[1759] VoiceRecognizer: A library for recognizing emotions from speech

[1760] ProductDatabase: Database for managing product information

[1761] Program processing

[1762] The server uses EmotionRecognizer to analyze the customer's facial image captured by the smart glasses' camera and recognize the customer's emotions. Next, it uses VoiceRecognizer to analyze the audio recorded by the smart glasses' microphone and recognize the tone of the customer's voice. This allows it to determine the customer's emotions, such as whether they are confused or satisfied.

[1763] The server then retrieves information corresponding to the specified product ID from the product database. Depending on the customer's emotions, it generates appropriate guidance and suggestions and displays them on the smart glasses' display. For example, if the customer is confused, it provides a more detailed explanation, and if satisfied, it makes additional suggestions.

[1764] Specific examples

[1765] As a concrete example, the following prompt sentence is input into the generative AI model:

[1766] A customer has a question about product ID 12345. Analyze the customer's facial image and voice to generate guidance and suggestions based on their emotions.

[1767] By inputting this prompt sentence into a generative AI model, it is possible to automatically generate appropriate guidance and suggestions based on the customer's emotions. This is expected to improve the user experience. It can also be used in physical stores to assist store clerks in providing appropriate guidance and suggestions to customers.

[1768] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1769] Step 1:

[1770] The user wears the smart glasses and captures the customer's facial image and voice.

[1771] Input: Customer's face image taken by the smart glasses camera, customer's voice recorded by the smart glasses microphone

[1772] Output: Customer's face image data, customer's voice data

[1773] How it works: The camera in the smart glasses captures a picture of the customer's face, and the microphone records the customer's questions and comments.

[1774] Step 2:

[1775] The server analyzes the facial image data using EmotionRecognizer to recognize the customer's emotions.

[1776] Input: Customer's facial image data

[1777] Output: Customer sentiment data (e.g., confused, satisfied, etc.)

[1778] Specific operation: The server inputs facial image data into EmotionRecognizer and uses an emotion recognition algorithm to identify the customer's emotions.

[1779] Step 3:

[1780] The server analyzes the voice data using VoiceRecognizer and recognizes the tone of the customer's voice.

[1781] Input: Customer voice data

[1782] Output: Customer's voice tone data (e.g. high, low, strong, weak, etc.)

[1783] What it does: The server inputs the audio data into VoiceRecognizer, which uses a speech analysis algorithm to identify the tone of the customer's voice.

[1784] Step 4:

[1785] The server retrieves information corresponding to the specified product ID from the product database.

[1786] Input: Product ID

[1787] Output: Product information data (e.g., troubleshooting, maintenance methods, etc.)

[1788] Specific operation: The server searches the product database using the product ID and obtains detailed information about the corresponding product.

[1789] Step 5:

[1790] The server generates appropriate guidance and suggestions based on the customer's emotional data and tone of voice data.

[1791] Input: Customer emotion data, tone of voice data, product information data

[1792] Output: Text data of guidance and suggestions

[1793] Specific operation: The server analyzes the emotion data and tone of voice data, and refers to the product information data to generate guidance and suggestions that correspond to the customer's emotions.

[1794] Step 6:

[1795] The server generates guidance and suggestions that are displayed on the smart glasses' display.

[1796] Input: Text data for guidance and suggestions

[1797] Output: Guidance and suggestions displayed on the smart glasses display

[1798] Specific operation: The server generates text data of guidance and suggestions, which are sent to the smart glasses and displayed on the screen.

[1799] Step 7:

[1800] The user checks the guidance and suggestions displayed on the smart glasses display and provides appropriate explanations to the customer.

[1801] Input: Guidance and suggestions displayed on the smart glasses display

[1802] Output: Customer instructions

[1803] Specific operation: The user reads the information displayed on the smart glasses display and verbally explains it to the customer.

[1804] Example 3

[1805] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1806] Conventional systems provide information about electronic devices in use to the user in a uniform manner, making it difficult to provide optimal information tailored to the user's individual usage situation, environment, and emotions. Furthermore, guidance and suggestions do not take the user's emotions into consideration, which results in a problem of not sufficiently improving the user experience.

[1807] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1808] In this invention, the server includes means for registering information about the electronic device being used by the user, means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage methods, disposal methods, recycling methods, etc. based on the registered information about the electronic device, means for acquiring weather information about the user's place of residence, means for recognizing emotions by analyzing information such as the user's voice, facial expressions, and behavioral patterns, and means for adjusting the content and expression of the guidance and suggestions according to the recognized emotions. This makes it possible to provide optimal information according to the user's individual usage situation, environment, and emotions.

[1809] "User" refers to any individual or entity that uses the System.

[1810] "Electronic devices" refer to devices that operate using electricity, such as home appliances and communication equipment.

[1811] "Means for registering information" refers to the functionality that allows users to input and save details of the electronic device they are using into the system.

[1812] "Troubleshooting" refers to the process of diagnosing problems or malfunctions in electronic equipment and providing solutions.

[1813] "Maintenance method" refers to the maintenance and management procedures for ensuring the normal operation of electronic equipment.

[1814] "Accessory Purchasing Method" refers to the procedure for purchasing accessories and parts related to electronic devices.

[1815] "Replacement time / successor product" refers to the expiration date of electronic devices or when it's time to update to a new model.

[1816] "Convenient usage" refers to ways to use electronic devices more efficiently and effectively.

[1817] "Disposal Method" refers to the procedure for properly disposing of used electronic equipment.

[1818] "Recycling method" refers to a procedure for processing used electronic equipment in a reusable form.

[1819] "Means for acquiring weather information" refers to a function for collecting weather data such as temperature, humidity, and season of the user's place of residence.

[1820] "Means for recognizing emotions" refers to a function for determining emotions by analyzing the user's voice, facial expressions, behavioral patterns, etc.

[1821] "Means for adjusting the content and expression of guidance and suggestions" refers to a function for changing the content and expression of the information provided according to the recognized user's emotions.

[1822] MODE FOR CARRYING OUT THE INVENTION

[1823] This invention is a system that registers information about the electronic devices a user is using and provides optimal guidance and suggestions based on that information.The system acquires weather information about the user's location, analyzes the user's emotions, and provides optimal information.

[1824] Hardware and software used

[1825] Hardware:

[1826] server

[1827] User devices (smartphones, tablets, PCs, etc.)

[1828] software:

[1829] Emotion Engine

[1830] Device Information Registration System

[1831] Troubleshooting System

[1832] Program processing explanation

[1833] The server provides a means for users to register information about the electronic devices they are using. Users enter detailed information about the electronic devices they are using (e.g., manufacturer, model, age, etc.), which allows the system to understand the specific devices they are using.

[1834] Next, the server obtains weather information for the user's location. Specifically, it uses weather databases and APIs to collect information such as the current temperature, humidity, and season. This provides the basic data needed to provide information appropriate for the user's environment.

[1835] Furthermore, the server uses an emotion engine to analyze the user's voice, facial expressions, and behavioral patterns. For example, if a user says, "It's hot today," the server recognizes through voice analysis that the user is feeling the heat. Also, if facial expression analysis reveals that the user looks uncomfortable, the server takes that emotion into consideration.

[1836] The server generates optimal information based on the acquired device information, environmental information, and user emotion information. For example, when it comes to air conditioner temperature settings, the server considers that the user prefers a cool environment and suggests setting the air conditioner to 26 degrees if the current temperature is 30 degrees.

[1837] Finally, the server provides the generated optimal information to the user. Specifically, it sends a notification to the user's device (smartphone, tablet, PC, etc.). For example, it displays a message such as, "The current temperature is 30 degrees. We recommend setting the air conditioner to 26 degrees."

[1838] Examples of specific examples and prompts

[1839] Examples:

[1840] When a user asks, "What temperature should I set the air conditioner to?", the server performs the following process.

[1841] 1. The user registers information about the air conditioner they are using (manufacturer, model, age, etc.).

[1842] 2. The server obtains weather information for the user's location (e.g., Tokyo).

[1843] 3. The emotion engine analyzes the user's voice and facial expressions to determine that they prefer a cool environment.

[1844] 4. Based on this information, the server suggests, "Since the average summer temperature in Tokyo is around 30 degrees, the optimal setting for the air conditioner is 26 degrees."

[1845] Example prompt sentence:

[1846] "Please tell me about the temperature settings for the air conditioner. I live in Tokyo and I like a cool environment."

[1847] In this way, the server realizes a system that provides optimal information according to the user's environment and preferences. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1848] Step 1:

[1849] The user registers the device information.

[1850] The user inputs details of the electronic device they are using (e.g., manufacturer, model, age, etc.) into the terminal. The terminal then sends the input device information to the server. The server stores the received device information in a database, allowing the system to understand the specific device the user is using.

[1851] Input: Device information entered by the user on the device

[1852] Output: Device information stored on the server

[1853] Step 2:

[1854] The server obtains the environment information.

[1855] The server sends a request to a weather database or API to obtain weather information for the user's location. The weather database or API returns information such as the current temperature, humidity, and season to the server. The server then stores the obtained weather information in a database. This provides the basic data needed to provide information appropriate to the user's environment.

[1856] Input: User's location information

[1857] Output: Climate information stored on the server

[1858] Step 3:

[1859] The server analyzes the user's emotions.

[1860] The server uses an emotion engine to analyze the user's voice, facial expressions, behavioral patterns, etc. For example, if a user says, "It's hot today," the server recognizes through voice analysis that the user is feeling the heat. Also, if facial expression analysis reveals that the user looks uncomfortable, the server takes that emotion into consideration. The analysis results are stored on the server.

[1861] Input: User's voice, facial expressions, and behavioral patterns

[1862] Output: Sentiment analysis results stored on the server

[1863] Step 4:

[1864] The server generates the best information.

[1865] The server generates optimal information based on the acquired device information, environmental information, and user emotion information. For example, when it comes to air conditioner temperature settings, the server considers that the user prefers a cool environment and suggests setting the air conditioner to 26 degrees if the current temperature is 30 degrees. The generated information is stored on the server.

[1866] Input: Device information, weather information, emotion analysis results

[1867] Output: The best information generated

[1868] Step 5:

[1869] The server provides the information to the user.

[1870] The server sends the generated optimal information to the user's device. The device then notifies the user of the received information. For example, it displays a message such as, "The current temperature is 30 degrees. We recommend setting the air conditioner to 26 degrees."

[1871] Input: Generated optimal information

[1872] Output: Notification displayed on the user's device

[1873] In this way, the server realizes a system that provides optimal information according to the user's environment and preferences.

[1874] (Application example 3)

[1875] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1876] Conventional systems have difficulty providing optimal information according to the user's usage situation and environment, and in particular have not provided guidance or suggestions that take the user's emotions into consideration.In addition, there has been a lack of suggestions and support to optimize work efficiency in factories, and there has been a need for a system that can recognize the emotions of workers and provide appropriate support.

[1877] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1878] In this invention, the server includes: means for registering device information for home appliances, communication devices, etc. currently in use by users; means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage methods, disposal methods, recycling methods, etc. based on the registered device information; means including an emotion engine for recognizing the user's emotions; means for the emotion engine to analyze information such as the user's voice, facial expressions, and behavioral patterns to recognize emotions and adjust the content and expression of the guidance and suggestions in accordance with the emotions; and means for recognizing the emotions of workers and making suggestions and support to optimize work efficiency. This makes it possible to provide optimal information taking user emotions into consideration and provide appropriate support to improve work efficiency in factories.

[1879] "User" means an individual or corporation that uses the system.

[1880] "Home appliances" is a general term for electrical devices used in the home.

[1881] A "communications device" is a device for sending and receiving information.

[1882] "Device information" is data related to home appliances and communication devices currently in use by the user.

[1883] "Troubleshooting" means the process of diagnosing and resolving device problems.

[1884] A "maintenance method" is a maintenance procedure for keeping a device operating normally.

[1885] "Accessory Purchasing Method" refers to the procedure for purchasing accessories and parts related to a device.

[1886] "Replacement / successor" refers to when a device needs to be replaced and its successor product.

[1887] "Convenient usage" is a way to use a device efficiently and effectively.

[1888] "Disposal Instructions" are procedures for properly disposing of used devices.

[1889] A "recycling method" is a procedure for reusing a used device.

[1890] The "emotion engine" is an analysis system for recognizing the user's emotions.

[1891] "Voice" refers to words and sounds uttered by the user.

[1892] "Facial expression" refers to the movements and expressions of the user's face.

[1893] "Behavioral patterns" refer to the user's movements and behavioral tendencies.

[1894] "Workers" are people who perform work within a factory.

[1895] "Work efficiency" refers to the efficiency with which work is carried out.

[1896] "Suggestions" are advice or recommendations provided to users or workers.

[1897] "Support" refers to assistance and help for users and workers.

[1898] The system for implementing the present invention uses the following hardware and software.

[1899] Hardware used

[1900] Camera (for facial expression recognition)

[1901] Microphone (for voice recognition)

[1902] Sensors (for behavioral pattern recognition)

[1903] Factory robots (for proposals and support execution)

[1904] Software used

[1905] Emotion Recognition Engine

[1906] Data Analysis Software

[1907] Troubleshooting Software

[1908] Proposal Generation AI Model

[1909] Explanation of data processing and data calculation

[1910] Data collection

[1911] The server collects data in real time from cameras, microphones, and sensors, capturing the voices, facial expressions, and behavioral patterns of users and workers.

[1912] emotion recognition

[1913] The server analyzes the collected data using an emotion recognition engine to recognize the emotions of users and workers, for example, identifying when a user is tired or stressed.

[1914] Data analysis

[1915] The server uses data analysis software to integrate emotion data and work situation data, and evaluates the current work efficiency and potential for trouble, making it possible to make optimal recommendations based on the status of the user and worker.

[1916] Proposal generation

[1917] The server uses a proposal generation AI model to generate suggestions and troubleshooting methods to optimize work efficiency based on the analysis results. The generated suggestions are sent to the factory robots and executed.

[1918] Proposal and support execution

[1919] The factory robot then carries out the suggestions and support received from the server, for example, suggesting that workers take a break or prompting them to recheck their work procedures.

[1920] Specific examples

[1921] For example, if the emotion recognition engine determines that a worker is tired, it will make the following suggestions:

[1922] "Work is progressing well, but I suggest you take a short break."

[1923] This task is a little complicated, so let's review the steps.

[1924] Prompt Sentence Examples

[1925] An example of a prompt sentence to input to the generative AI model is as follows:

[1926] Worker Emotion Data: {Emotion Data}

[1927] Work status data: {Work status data}

[1928] Generate a proposal.

[1929] Based on this prompt, the AI ​​model generates optimal suggestions based on emotional data and work situation data.

[1930] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1931] Step 1:

[1932] The server collects data in real time from cameras, microphones and sensors.

[1933] Input: Real-time data from cameras, microphones, and sensors

[1934] Output: Collected voice, facial expression, and behavioral pattern data

[1935] Specific operations: The camera captures the user's facial expressions, the microphone records audio, and the sensors detect behavioral patterns.

[1936] Step 2:

[1937] The server analyzes the collected data using an emotion recognition engine and recognizes the emotions of users and workers.

[1938] Input: Collected voice, facial expression, and behavioral pattern data

[1939] Output: Recognized emotion data

[1940] Specific behavior: The emotion recognition engine analyzes voice tone, facial expression changes, and behavioral patterns to identify the user's emotional state (e.g., fatigue, stress, joy).

[1941] Step 3:

[1942] The server uses data analysis software to integrate emotional data and work situation data to evaluate current work efficiency and the possibility of trouble.

[1943] Input: Emotion data, work situation data

[1944] Output: Work efficiency evaluation data, trouble possibility evaluation data

[1945] Specific operation: Data analysis software compares emotional data with work situation data to evaluate signs of declines in work efficiency or trouble.

[1946] Step 4:

[1947] The server uses a proposal generation AI model to generate proposals and troubleshooting methods to optimize work efficiency based on the analysis results.

[1948] Input: Work efficiency evaluation data, trouble possibility evaluation data

[1949] Output: Proposal content data

[1950] Specific actions: The proposal generation AI model generates specific suggestions (e.g., suggesting breaks, reconfirming work procedures) based on the evaluation data.

[1951] Step 5:

[1952] The server transmits the generated proposal to the factory robot and has it execute it.

[1953] Input: Proposal content data

[1954] Output: Suggestions implemented

[1955] Specific operation: The factory robot receives the suggestions and implements them to the worker (e.g., suggesting a break by voice, displaying work procedures on a screen).

[1956] Step 6:

[1957] Users can accept suggestions and support from factory robots to improve work efficiency.

[1958] Input: Proposal from factory robot

[1959] Output: Improved work efficiency

[1960] Specific behavior: By taking a break or reconfirming work procedures as suggested, the user's work efficiency improves.

[1961] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1962] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in the form of data such as voice data and text data.

[1963] Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1964] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[1965] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1966] [Third embodiment]

[1967] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1968] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1969] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1970] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1971] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1972] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1973] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1974] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1975] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1976] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1977] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1978] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1979] "Example 1"

[1980] The system of the present invention includes a means for users to register device information for home appliances, communication devices, etc. currently in use, and a means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. based on the registered device information. Specifically, users register information about the devices they are using using a dedicated application or website. The information to be registered includes the device type, manufacturer, model name, purchase date, usage status, etc.

[1981] "Example 2"

[1982] Based on the registered device information, the system retrieves information corresponding to each device from the database and provides guidance and suggestions to the user. For example, in the case of a refrigerator, it provides information such as how to clean it regularly, how to deal with error codes, how to purchase accessories, when to replace it, how to use it energy efficiently, and how to recycle it when discarding it.

[1983] "Example 3"

[1984] Furthermore, the system provides information optimized for the user's usage situation and environment. For example, if a user inquires about the temperature setting of an air conditioner, the system will suggest the optimal temperature setting taking into account the climate of the user's residence, the type of air conditioner, the user's preferences, etc.

[1985] The processing flow of each embodiment will be described below.

[1986] "Example 1"

[1987] Step 1: The user registers information about the device they are using on a dedicated application or website, including the device type, manufacturer, model name, purchase date, and usage status.

[1988] Step 2: The system retrieves the relevant device information from the database based on the registered information.

[1989] Step 3: Based on the acquired information, the system will provide the user with information and suggestions on troubleshooting, maintenance methods, how to purchase accessories, when to replace / replace the product, convenient usage, disposal methods, recycling methods, etc.

[1990] "Example 2"

[1991] Step 1: When a user makes a query about a specific device, the system

[1992] Provide optimal information by taking into account device information, user usage, and environment.

[1993] Step 2: For example, if a user inquires about the temperature setting of an air conditioner, the system will suggest the optimal temperature setting taking into consideration the climate of the user's place of residence, the type of air conditioner, the user's preferences, etc.

[1994] Example 1

[1995] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1996] Modern users use a wide variety of electronic devices and communication equipment, and they must individually look up information about each device, such as troubleshooting, maintenance methods, how to purchase accessories, when to replace them, convenient ways to use them, how to dispose of them, and how to recycle them. This requires a lot of time and effort from users, making efficient device management difficult. Furthermore, if the information provided is not optimized for the user's usage situation or environment, it may not be possible to respond appropriately. Furthermore, the lack of an interactive interface makes it difficult to update or adjust the information according to user input.

[1997] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1998] In this invention, the server includes means for registering device information of electronic devices, communication devices, etc. currently in use by the user, means for storing the registered device information in a database, means for sending prompts to the generative AI model based on the registered device information, means for receiving responses from the generative AI model and providing them to the user, and means for providing and suggesting troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. based on the registered device information. This allows users to centrally manage device information and quickly obtain optimized information. Furthermore, an interactive interface allows users to easily update and adjust information based on user input.

[1999] A "user" is an individual or corporation that uses the system to register information about electronic devices or communication devices and receives guidance and suggestions.

[2000] "Electronic devices" is a general term for devices that operate using electricity, such as household appliances, computers, and smartphones.

[2001] "Communication equipment" is a general term for devices used to send and receive information over the Internet or telephone lines.

[2002] "Device information" refers to detailed information about the electronic device or communication device currently being used by the user, such as the type, manufacturer, model name, purchase date, and usage status.

[2003] A "database" is an information system for efficiently storing, managing, and searching registered device information.

[2004] A "generative AI model" is an artificial intelligence algorithm that generates appropriate answers or suggestions based on user input.

[2005] A "prompt sentence" is an input sentence that instructs a generative AI model to generate specific information.

[2006] "Troubleshooting" is the process of diagnosing equipment problems or malfunctions and providing solutions.

[2007] "Maintenance method" refers to the method of regular inspection and care to keep the equipment in good condition.

[2008] "Accessory Purchasing Method" refers to procedures and information for purchasing accessories and parts related to the device.

[2009] "Replacement timing / successor product" is information on the recommended time to replace the currently used device and the new model that will be its successor.

[2010] "Helpfuls" are tips and tricks for using equipment more efficiently and effectively.

[2011] "Disposal method" means the procedure or method for properly disposing of used equipment.

[2012] A "recycling method" is a procedure or method for disposing of used equipment in a reusable form.

[2013] An "interactive interface" is an operating screen or function that allows users to input and obtain information while interacting with the system.

[2014] This invention is a system that allows users to register information about electronic devices and communication devices they are using, and based on that information, provides guidance and suggestions on troubleshooting, maintenance methods, accessory purchasing methods, replacement timing / successor products, convenient usage, disposal methods, recycling methods, etc. This system operates in cooperation with the user, terminal, and server.

[2015] Users use a dedicated application or website to enter information about the device they are using. The information to be entered includes the type of device, manufacturer, model name, purchase date, and usage status. For example, when a user registers a "television," they enter "electronic device manufacturer" in "manufacturer," "Model X" in "model name," "January 2022" in "purchase date," and "daily use" in "usage status."

[2016] The device (user's smartphone or PC) sends the entered device information to the server. The information is encrypted before being sent. The server stores the received device information in a database. The database uses a relational database such as MySQL or PostgreSQL.

[2017] The server analyzes the registered device information and sends the necessary information to the generative AI model as a prompt. For example, it generates a prompt such as "Please tell me how to troubleshoot Model X." The generative AI model (e.g., OpenAI's GPT-4) receives the prompt and generates an appropriate answer. The generated answer is sent back to the server.

[2018] The server provides the answer received from the generative AI model to the user, who can then view the answer in an application or on a website. For example, if a user registers a "Model X" television, the server provides the following information:

[2019] Troubleshooting: What to do if the screen is blank

[2020] Maintenance method: How to clean the screen

[2021] How to purchase accessories: How to purchase a remote control

[2022] Replacement time / successor: Information on new models

[2023] Useful tips: How to set up voice control

[2024] Disposal Instructions: How to properly dispose of a television

[2025] How to recycle: Recycling center information

[2026] Example prompts to input to a generative AI model:

[2027] A user has registered a "Model X" TV. Please provide information on troubleshooting, maintenance, purchasing accessories, replacement timing / successor products, convenient usage, disposal, and recycling for this TV.

[2028] This system allows users to centrally manage equipment information, quickly obtain optimized information, and easily update and adjust information based on user input through an interactive interface.

[2029] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2030] Step 1:

[2031] The user enters the device information

[2032] Users access a dedicated application or website and enter information about the device they are using. The information entered includes the type of device, manufacturer, model name, purchase date, and usage status. For example, when a user registers a "television," they enter "electronic device manufacturer" in "manufacturer," "Model X" in "model name," "January 2022" in "purchase date," and "daily use" in "usage status." The entered information is saved on the device.

[2033] Step 2:

[2034] The device sends the entered information to the server.

[2035] The device (user's smartphone or PC) encrypts the entered device information and sends it to the server. The entered data includes the device type, manufacturer, model name, purchase date, usage status, etc. The server receives this data and stores it in a database.

[2036] Step 3:

[2037] The server stores the device information in a database

[2038] The server analyzes the received device information and stores it in a database using a relational database such as MySQL or PostgreSQL. The stored data includes the device type, manufacturer, model name, purchase date, usage status, etc.

[2039] Step 4:

[2040] The server sends a prompt to the generative AI model based on the device information.

[2041] The server analyzes the device information stored in the database and sends the necessary information to the generative AI model as a prompt. For example, it generates a prompt such as "Please tell me how to troubleshoot Model X." The input data is the device information, and the output data is the prompt.

[2042] Step 5:

[2043] The generative AI model generates an answer and sends it back to the server

[2044] A generative AI model (e.g., OpenAI's GPT-4) receives a prompt and generates an appropriate answer. The generated answer is sent back to the server. The input data is the prompt, and the output data is the generated answer.

[2045] Step 6:

[2046] The server provides the generated answer to the user.

[2047] The server displays the answers it receives from the generative AI model in the user's application, where the user can review the answers in the application or on a website. For example, if a user registers a "Model X" television, the server provides the following information:

[2048] Troubleshooting: What to do if the screen is blank

[2049] Maintenance method: How to clean the screen

[2050] How to purchase accessories: How to purchase a remote control

[2051] Replacement time / successor: Information on new models

[2052] Useful tips: How to set up voice control

[2053] Disposal Instructions: How to properly dispose of a television

[2054] How to recycle: Recycling center information

[2055] The input data is the answer that is generated, and the output data is the information that is presented to the user.

[2056] (Application example 1)

[2057] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2058] Conventional systems make it difficult for users to centrally manage information about the equipment they are using and provide comprehensive guidance and suggestions on troubleshooting, maintenance methods, parts purchasing methods, replacement timing, convenient usage, disposal methods, recycling methods, etc. Furthermore, particularly for robots used in factories, there was a lack of means to properly manage and efficiently provide this information. This prevented users from performing appropriate maintenance and troubleshooting, resulting in problems such as a shortened equipment lifespan and reduced efficiency.

[2059] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2060] In this invention, the server includes means for registering information about devices currently in use by users, means for providing and suggesting troubleshooting, maintenance methods, parts pu...

Claims

1. A registration means for registering device information including a manufacturer name, a model number, and a usage status of a home appliance or a communication device used by a user; A means for suggesting at least one of information regarding a solution to a problem of the home appliance or the communication device, a maintenance method for the home appliance or the communication device, a method for purchasing accessories for the home appliance or the communication device, a time to replace the home appliance or the communication device, a successor product for the home appliance or the communication device, a convenient way to use the home appliance or the communication device, a method for disposing of the home appliance or the communication device, and a method for recycling the home appliance or the communication device, based on the registered device information, by inputting a specific prompt sentence into a generative AI model; an emotion engine that recognizes the emotion of the user; an adjustment means for adjusting the content and expression of the proposed information in response to the emotion; A system including:

2. The system according to claim 1 , wherein the adjustment means makes a suggestion for reducing the stress or fatigue when the user feels stress or fatigue due to the emotion.

3. The system of claim 1 , further comprising: means for displaying the suggested information on a smartphone, smart glasses, or a head mounted display.

Images