system

Abstract

We provide the system. [Solution] An input method for receiving questions and requests from users, An analysis tool for analyzing received questions and requests, A search method for searching for household chore-related information based on the analysis results, A response generation means that provides the searched information to the user, A notification method that sends push notifications to encourage household chores to be performed, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In daily life, there are many people who are not good at housework, and they even find it troublesome to look up how to do housework. Also, many people feel resistance to using housework proxy services due to the impression that they are expensive and privacy issues. Therefore, especially with the increase in the number of people staying at home, there is a need to provide an efficient and reliable housework support system.

Means for Solving the Problems

[0005] This invention first includes an input means for receiving questions and requests from users. This input means allows users to easily input questions about their household chores. Next, an analysis means is provided to analyze the received content, and important keywords are extracted from the questions using natural language processing technology. Based on the analysis results, a search means is provided to search a database for relevant household chore information. Furthermore, a response generation means is provided to generate a message including links to videos and instruction manuals based on the acquired information, and this message is provided to the user. Finally, a notification means is provided to send push notifications to encourage the execution of household chores. This series of means makes it possible to build a system that effectively solves the challenges users face with household chores.

[0006] A "user" refers to an individual person who uses the system, or the account used for that purpose.

[0007] "Questions and requests" refer to any questions or requests regarding household chores that users enter into the system.

[0008] "Input method" refers to the interface or device used by a user to input questions or requests into the system.

[0009] "Analysis means" refers to a function that analyzes received questions and requests and processes them to extract important information.

[0010] "Natural language processing technology" refers to the technology that enables computers to understand and process information expressed in human language.

[0011] "Search method" refers to the function of searching for relevant household information from a database based on the analyzed information.

[0012] "Response generation means" refers to a function that generates a message to be provided to the user based on the retrieved information.

[0013] "Notification method" refers to a function that sends push notifications to users to encourage them to perform household chores.

[0014] "Videos and instruction manuals" refer to resources that visually and in writing show users how to do household chores and the procedures involved. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Embodiment for Carrying out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention provides a system to support users with household chore-related challenges. First, the user operates a dedicated application using a device such as a smartphone or PC. The application is designed to allow easy input of questions and requests regarding household chores through its user interface. Once the user completes the input, the information is immediately sent to a server for analysis.

[0037] Next, the server analyzes the information submitted by the user using natural language processing techniques to extract important keywords. The results obtained from this analysis form the basis for the subsequent search process. Based on the extracted keywords, the server searches the database for relevant household information, such as cleaning procedures and instructional videos. This search process is performed quickly and accurately to select the most relevant information for the user.

[0038] The server then uses the retrieved information to generate a response for the user. This response includes links to resources and concise advice, designed to help the user perform household chores effectively. The generated response is sent to the terminal and displayed to the user.

[0039] Furthermore, to ensure users don't forget to do household chores, the device is equipped with a push notification function. Users can set notification schedules within the application, for example, to receive cleaning reminders on specific days. These notifications are managed through coordination with a server.

[0040] As a concrete example, consider a scenario where a user wants to know how to clean their living room. The user inputs "living room cleaning procedure," and the server, after analysis, generates a response including relevant videos and instructions, which are then displayed on the device. This process enables the user to efficiently perform household chores and receive support for a more comfortable life.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] The user accesses the application on their device and enters questions or requests related to household chores. The entered information is sent to the server when the submit button is pressed.

[0044] Step 2:

[0045] The terminal converts the data entered by the user into JSON format and sends an HTTP request to the server's API endpoint.

[0046] Step 3:

[0047] The server receives data from the terminal for analysis. The received data is first validated to confirm that it is in the correct format.

[0048] Step 4:

[0049] The server uses a natural language processing (NLP) module to analyze the content of incoming questions and requests. It then extracts relevant keywords and phrases.

[0050] Step 5:

[0051] The server uses the extracted keywords to search a database containing information about household chores. The search results include resources such as instruction manuals and explanatory videos.

[0052] Step 6:

[0053] After retrieving the appropriate information, the server generates a response to provide to the user. The generated response includes the URL and description of the searched resource.

[0054] Step 7:

[0055] The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0056] Step 8:

[0057] The terminal analyzes the received response data and displays it on the user interface. This allows the user to access suggested household chore methods and resources.

[0058] Step 9:

[0059] Users can configure push notification settings through their device. Once the settings are complete, that information is sent back to the server.

[0060] Step 10:

[0061] The server registers the user's notification settings as a schedule and sends push notifications to the device at the specified time to remind the user to perform household chores.

[0062] (Example 1)

[0063] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0064] In today's busy lifestyle, efficiently managing household tasks and providing users with necessary information quickly and accurately is challenging. In particular, there is a need to significantly reduce the time and effort required in the process of acquiring useful data from multiple sources, appropriately analyzing it, and communicating beneficial results to users.

[0065] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0066] In this invention, the server includes terminal means for receiving information input from users, data processing means for analyzing the received information, and information extraction means for searching for life management-related information based on the analysis results. This enables users to efficiently and easily obtain the necessary information and appropriately manage their daily tasks.

[0067] "Terminal means" refers to a device used by users to input information, such as a smartphone or personal computer.

[0068] "Data processing means" refers to a function that analyzes input information and identifies important attributes.

[0069] An "information extraction method" is a process that has the function of searching for necessary information from relevant databases based on the results of the analysis.

[0070] "Result display means" refers to an output function that conveys searched information to the user in an easy-to-understand manner, and includes systems that display text and links.

[0071] "Communication means" refers to a method of providing a communication function to send notifications to users regarding daily life activities.

[0072] Generative modeling is a machine learning technique used to extract specific patterns or attributes from data.

[0073] An "information link" is connection information that indicates the location of information in a way that users can access.

[0074] This invention is a system for efficiently acquiring information necessary for daily life and managing daily living tasks. First, the user accesses the system using a terminal. This terminal consists of information input devices such as smartphones and personal computers. Through the terminal, the user can operate a dedicated application and input questions and requests related to daily living tasks.

[0075] When a device transmits information, the server receives it. The server utilizes natural language processing technology, specifically generative modeling. For example, it uses generative models such as BERT or GPT to identify important attributes and keywords from the input information. This allows for an accurate understanding of the user's intent.

[0076] Subsequently, the server uses information extraction means to search relevant databases based on the identified keywords. These databases contain procedures and explanatory materials related to the target daily living activity, and the server quickly selects the most relevant information. The results obtained through this process are presented to the user via a results display means.

[0077] The server generates results that include text information and resource links, which can be viewed on the user's device. Finally, using communication methods, push notifications are sent to the device based on the user's set time for daily activities. This helps users remember to perform necessary activities.

[0078] For example, when a user enters a prompt such as, "Please tell me the steps for cleaning the living room," the system provides relevant cleaning instructions and links to explanatory videos. This entire process minimizes the time users spend searching for information, allowing them to efficiently carry out household tasks.

[0079] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0080] Step 1:

[0081] The user launches a dedicated application on their device and enters information. The device receives a natural language prompt, such as, "Please tell me the steps for cleaning the living room." It then prepares to send this input information to the system.

[0082] Step 2:

[0083] The terminal sends the entered prompt text to the server. The server receives the data and verifies the input information as received data. This input data becomes the basis for proceeding to the next analysis step.

[0084] Step 3:

[0085] The server analyzes the input prompt sentence using natural language processing techniques. This analysis utilizes generative AI models such as BERT and GPT to extract important keywords from the sentence, such as "cleaning" and "procedure." The extracted keywords are then output and used in the next search process.

[0086] Step 4:

[0087] The server searches the database based on the keywords obtained in the analysis step. It quickly finds relevant lifestyle information, such as cleaning procedures and links to explanatory videos. The output of this search is information selected as the most suitable resource.

[0088] Step 5:

[0089] The server uses the search results to generate a response message to present to the user. This message includes text information and links and is ready to be sent to the terminal. The generated message is output and ready for display.

[0090] Step 6:

[0091] The terminal displays the response message received from the server in an easy-to-understand format for the user. The user can easily access the information they need. This output allows the user to review the steps for cleaning the living room.

[0092] Step 7:

[0093] When a user sets a schedule for daily task notifications, the server sends push notifications based on specific dates and times. These notifications help users remember to complete the tasks they have set.

[0094] (Application Example 1)

[0095] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0096] A lack of efficient work management within physical stores can lead to improperly performed tasks such as cleaning and inventory replenishment. This can result in decreased operational efficiency and potentially damage customer satisfaction. Therefore, a system is needed to support and streamline operations within physical stores.

[0097] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0098] In this invention, the server includes input means for receiving tasks and services from users, analysis device means for analyzing the received tasks and services, and in-facility work support device means for managing work plans. This enables effective and rapid work management within the store.

[0099] "Users" refer to individuals who actually use the system to manage household tasks or facility-based work.

[0100] "Issues and services" refer to specific requests or questions related to household tasks or facility work that users wish to resolve or perform.

[0101] "Input method" refers to the interface through which users input tasks or services into the system.

[0102] An "analysis device" refers to a device that understands the input tasks or services and extracts important information.

[0103] A "detection device" refers to a device used to find the most relevant household business data based on the analyzed information.

[0104] A "response generation device" refers to a device that configures detected information to be provided to the user.

[0105] A "reporting device" refers to a device that sends push notifications to users when they are performing household tasks.

[0106] "In-facility work support equipment" refers to equipment that manages work plans within a physical store and supports the effective execution of tasks.

[0107] To realize this invention, a program is needed to construct the entire system. The processing flow of that program is described below in natural language.

[0108] The server is implemented using Node.js and the Express framework. This server receives tasks and services entered by users using their smartphones. The data is sent from a user interface on the device developed using React Native.

[0109] The server then uses the Google® Cloud Natural Language API to analyze the tasks and services and identify key concepts. This analysis extracts information related to home-based tasks and in-store operations.

[0110] The server searches the data stored in MongoDB based on the extracted information and finds the relevant information. Then, the response generator constructs the information to be provided to the user as video links or instruction manual links.

[0111] Furthermore, the reporting device sends push notifications to the terminal, prompting users to perform household chores.

[0112] For example, if a store staff member types in, "Please tell me the procedure for wiping dust off shelves. I would also like to know what tools are needed," the server will quickly and accurately support the task by providing appropriate procedure videos and a list of tools.

[0113] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0114] Step 1:

[0115] The user uses their smartphone to input the tasks or services they wish to perform. A React Native app installed on the device receives this input and sends it to the server. The input data includes the details of the task and the user's requests. The output is a data request sent to the server.

[0116] Step 2:

[0117] The server uses Node.js and Express to receive data from the terminal. The received data is parsed using the Google Cloud Natural Language API, and important concepts and keywords are extracted. The parsed input data generates a list of concepts as output.

[0118] Step 3:

[0119] The server searches the MongoDB database based on the concepts obtained from the analysis. The database contains information related to household tasks and facility-based tasks, and relevant resources are selected. A list of concepts is used as input, and the relevant information is retrieved as output.

[0120] Step 4:

[0121] Based on the acquired information, the server generates a response. The response includes links to instructional videos and specific instruction manuals. The information is constructed using prompts generated by the generation AI model and provided to the user as a response.

[0122] Step 5:

[0123] The server uses the response information to send a push notification to the terminal. This notification prompts the user to perform their task at the appropriate time. The output is a notification that is displayed on the user's terminal.

[0124] Step 6:

[0125] Users efficiently perform household chores and facility-based tasks based on information displayed on their devices. Based on user input, appropriate resources are used to complete the tasks.

[0126] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0127] This invention relates to a system that recognizes the user's emotional state and provides household support. Users can send questions and requests regarding household chores to the system from a device such as a smartphone or PC. These requests can be made through a dedicated application with a natural interface.

[0128] When the terminal receives user input, that information is immediately sent to the server. The server then processes the received data. A particularly noteworthy aspect here is the use of an emotion engine to recognize the user's emotional state. The emotion engine analyzes emotional nuances from the input language data to identify the user's current emotions. This allows for determinations such as whether the user is stressed or relaxed.

[0129] The server then uses natural language processing technology to analyze the question and retrieve relevant household information from its database. The search results include appropriate instructions and explanatory videos. The retrieved information is then used to generate a response that is tailored to the user's emotional state. For example, if the user is confused, a more detailed explanation is provided; if they are calm, only concise information is provided.

[0130] The generated response is sent from the server to the terminal and provided to the user. The terminal updates its interface based on this information, and it is designed to be visually user-friendly. In addition, a push notification function is included, allowing the user to be prompted to do household chores at the appropriate time. The content and timing of the notifications are also adjusted according to the user's emotional state, so appropriate support is provided without causing the user excessive stress.

[0131] For example, suppose a user asks, "How do I clean?" and the server's emotion engine recognizes that the user is tired. In that case, the system selects a simple and effective cleaning method, generates a response that includes a link to a relaxing audio guide, and presents it to the user. In this way, it enables household support that takes the user's emotions into consideration.

[0132] The following describes the processing flow.

[0133] Step 1:

[0134] The user opens the application on their device and enters questions or requests related to household chores. After completing the input, they press the submit button to send the information to the server.

[0135] Step 2:

[0136] The terminal formats the user's input data and sends an HTTP request to the server. During this process, the data is converted to JSON format.

[0137] Step 3:

[0138] The server receives data sent from the terminal and uses an emotion engine to analyze the user's emotional state from their input. For example, it detects emotions from contexts such as "tired" or "stressed."

[0139] Step 4:

[0140] The server uses natural language processing technology to analyze input questions and requests and extract relevant keywords. This allows the type of information the user is specifically seeking to be identified.

[0141] Step 5:

[0142] Based on the analysis results, the server searches the database for household-related information. Appropriate resources such as instruction manuals and explanatory videos are selected.

[0143] Step 6:

[0144] The server considers the emotional state provided by the emotion engine and generates a response based on the retrieved information. For example, if the user is feeling stressed, the response will include encouraging messages and information that can help with relaxation.

[0145] Step 7:

[0146] The server sends the generated response to the terminal in JSON format.

[0147] Step 8:

[0148] The terminal receives a response from the server and displays it on the user interface. The user then reviews the information provided and uses it to help with household chores.

[0149] Step 9:

[0150] Users can schedule appropriate push notifications through their devices. This information is then sent back to the server.

[0151] Step 10:

[0152] The server is configured to send push notifications at specific times based on the user's notification settings. The timing and content of notifications are optimized according to the user's emotional state.

[0153] (Example 2)

[0154] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0155] In recent years, there has been a growing demand for efficient performance of various tasks in daily life. However, conventional systems that provide support while considering the emotional state of the user are not yet fully established. When a user is stressed or in a particular emotional state, the information provided may not always be appropriate. Therefore, there is a need for a system that can facilitate daily life tasks by providing information tailored to the user's emotions.

[0156] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0157] In this invention, the server includes an input device means for receiving inquiries and requests from users, an emotion analysis device means for performing natural language processing to recognize the user's emotions, and a response adjustment device means for adjusting information according to the user's emotional state. This makes it possible to provide life support optimized according to the user's emotional state.

[0158] An "input device" is a device used to receive inquiries and requests from users.

[0159] An "analysis device" is a device used to analyze inquiries and requests received.

[0160] A "search device" is a device used to find information related to daily living activities based on the results of analysis.

[0161] A "response generation device" is a device that presents the searched information to the user.

[0162] A "notification device" is a device that sends out notifications to encourage the performance of daily living activities.

[0163] An "emotion analysis device" is a device that performs natural language processing to recognize the emotions of its users.

[0164] A "response adjustment device" is a device that adjusts information according to the user's emotional state.

[0165] This invention embodies a system that recognizes the user's emotional state and provides support for daily living activities accordingly. This system allows users to send questions and requests related to daily living activities using a smartphone or personal computer (terminal). By using a dedicated application, users can operate the system through an intuitive and natural interface.

[0166] The terminal receives input from the user and sends that data to the server. The server analyzes the data using natural language processing technology. This analysis utilizes an emotion analysis device to extract emotional nuances from the user's input data. Emotion analysis is a crucial process for recognizing what emotional state the user is in, such as stress or relaxation.

[0167] Based on the analysis results, the server searches its internal database for relevant life skills information. This database contains procedures, explanatory videos, and other information that addresses the user's questions. After the information is retrieved, the server generates a response appropriate to the user's emotional state. For example, if the user is confused, it provides a detailed guide; if they are calm, it presents concise information.

[0168] Ultimately, the server sends the generated response to the terminal and presents it to the user. The terminal updates its interface based on the received information, displaying it clearly. The terminal also has a push notification function, which can prompt the user to perform daily tasks at the appropriate time. The content and timing of notifications are also adjusted according to the user's emotional state, so support is provided without causing excessive stress.

[0169] For example, if a user asks, "How do I clean?", the server, through its emotion engine, recognizes that the user is tired. As a result, the system suggests a simple and effective way to clean, and generates a response that also includes a link to a relaxing audio guide.

[0170] An example of a prompt message might be: "The user wants to know how to clean, but appears tired. How should you respond?"

[0171] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0172] Step 1:

[0173] Users open a dedicated application via their smartphone or personal computer and input questions and requests related to daily life tasks. The input data is in text or voice format and uses a natural interface. The terminal immediately transmits this input data to the server. The input data includes the user's questions and voice data. The output is the transmitted user data.

[0174] Step 2:

[0175] The server passes the data received from the terminal to the emotion analysis device. The emotion analysis device uses natural language processing technology to analyze the text or audio data and recognize the user's emotions. The analysis evaluates the frequency of positive or negative words and the tone of the text to determine the degree of stress or relaxation. The input is the user's text or audio data, and the output is the result of the determination of the user's emotional state.

[0176] Step 3:

[0177] The server analyzes the user's inquiry based on their emotional state, which is obtained through sentiment analysis. It utilizes natural language processing techniques to extract key keywords from the questions and requests. This information is then used to search the server's database for information related to daily tasks. The input is the user's inquiry and emotional state, and the output is a list of related information.

[0178] Step 4:

[0179] The server uses a response generator based on the search results to create a response tailored to the user. It adjusts the information to match the user's emotional state and generates a message that includes detailed instructions or concise guides. It may also include relevant images, video links, or audio guides. The input is the searched information and emotional state, and the output is the generated response message.

[0180] Step 5:

[0181] The server sends the generated response to the terminal. The terminal receives this information and updates the interface in a user-friendly format. In addition to visual displays, push notifications deliver timely messages to encourage daily tasks. The content and timing of notifications are also adjusted based on the user's emotional state. The input is the response message sent from the server, and the output is the updated interface and the sent notifications.

[0182] (Application Example 2)

[0183] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0184] It is important for modern consumers to be able to easily obtain product information and have a comfortable shopping experience. However, when customers ask questions about products in stores, they may experience stress if the information provided does not take into account their individual emotional state. This invention aims to enable the provision of optimal product information based on an analysis of the customer's emotional state.

[0185] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0186] In this invention, the server includes an input means for receiving questions and requests from the user, an emotion analysis means for analyzing the received questions and requests and recognizing the user's emotional state, a search means for searching for product-related information based on the analysis results, and a response generation means for providing the searched information in an optimal form according to the user's emotional state. This makes it possible to provide product information that matches the emotional state of each individual customer and realize a comfortable purchasing experience.

[0187] An "input method" is an interface for receiving questions and requests from users.

[0188] "Emotional analysis tools" are functions that analyze received questions and requests to identify the user's emotional state.

[0189] A "search tool" is a function that extracts and provides relevant information based on the analyzed results.

[0190] A "response generation means" is a process for providing the user with extracted information in the most optimal form, according to the user's emotional state.

[0191] "Notification methods" refer to a function that sends necessary information and guidance as push notifications, tailored to the user's emotional state.

[0192] This embodiment includes a server, a user terminal, and a network for exchanging data between them. The user first uses the terminal to input questions or requests for product information or assistance with household chores. This data is then sent to the server via the input means. The server uses sentiment analysis means to identify the user's emotional state from this data. Natural language processing tools such as TextBlob are used in this process.

[0193] Based on the analysis results, the server uses search tools to find relevant information. This information is extracted from existing resources stored in the database. The response generation tool optimizes the information according to the user's emotional state and constructs specific content to be provided to the user. The information provided at this time takes the form of a detailed explanation or a concise summary, depending on the user's emotions.

[0194] For example, if a user enters "I want to know more about this product, but I'm in a hurry" into a terminal in a store, the server detects that the user is experiencing stress. As a result, the response generation mechanism quickly presents concise information summarizing the product's main features. The notification mechanism sends this information to the terminal as a push notification, timely and efficiently delivered to the user.

[0195] In this way, the system can provide information tailored to each customer's emotional state and appropriately meet their needs.

[0196] An example of a prompt message for a generative AI model might be: "Calculate emotional polarity from customer input, determine the emotional state, and dynamically generate information based on the result. For example, if the polarity is -0.3, determine that the customer is stressed and provide detailed guidance."

[0197] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0198] Step 1:

[0199] The user enters questions or requests about the product via a terminal. This data is then sent to the server. This step includes the process up to the point where the user's input is sent to the server as text data.

[0200] Step 2:

[0201] The server passes the received text data to a sentiment analysis tool. Here, the text data is analyzed using a natural language processing library (e.g., TextBlob), and its polarity value is calculated. Based on this polarity value, the user's emotional state is classified as "relaxed," "stressed," or "neutral."

[0202] Step 3:

[0203] The server uses search methods to retrieve relevant information from the database based on the emotional state. If the emotional state is stress, only concise and essential product information is extracted.

[0204] Step 4:

[0205] The server's response generation mechanism constructs content to provide the user with information obtained in the most optimal format, depending on their emotional state. Specifically, if the user is experiencing stress, concise content emphasizing the most important points is generated.

[0206] Step 5:

[0207] Content generated by the notification system is sent to the device as a push notification. The user receives the notification and can obtain the information appropriately. This ensures that the information the user needs is provided in a timely manner, guaranteeing a smooth user experience.

[0208] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0209] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0210] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0211] [Second Embodiment]

[0212] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0213] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0214] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0215] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0216] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0217] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0218] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0219] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0220] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0221] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0222] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0223] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0224] This invention provides a system to support users with household chore-related challenges. First, the user operates a dedicated application using a device such as a smartphone or PC. The application is designed to allow easy input of questions and requests regarding household chores through its user interface. Once the user completes the input, the information is immediately sent to a server for analysis.

[0225] Next, the server analyzes the information submitted by the user using natural language processing techniques to extract important keywords. The results obtained from this analysis form the basis for the subsequent search process. Based on the extracted keywords, the server searches the database for relevant household information, such as cleaning procedures and instructional videos. This search process is performed quickly and accurately to select the most relevant information for the user.

[0226] The server then uses the retrieved information to generate a response for the user. This response includes links to resources and concise advice, designed to help the user perform household chores effectively. The generated response is sent to the terminal and displayed to the user.

[0227] Furthermore, to ensure users don't forget to do household chores, the device is equipped with a push notification function. Users can set notification schedules within the application, for example, to receive cleaning reminders on specific days. These notifications are managed through coordination with a server.

[0228] As a concrete example, consider a scenario where a user wants to know how to clean their living room. The user inputs "living room cleaning procedure," and the server, after analysis, generates a response including relevant videos and instructions, which are then displayed on the device. This process enables the user to efficiently perform household chores and receive support for a more comfortable life.

[0229] The following describes the processing flow.

[0230] Step 1:

[0231] The user accesses the application on their device and enters questions or requests related to household chores. The entered information is sent to the server when the submit button is pressed.

[0232] Step 2:

[0233] The terminal converts the data entered by the user into JSON format and sends an HTTP request to the server's API endpoint.

[0234] Step 3:

[0235] The server receives data from the terminal for analysis. The received data is first validated to confirm that it is in the correct format.

[0236] Step 4:

[0237] The server uses a natural language processing (NLP) module to analyze the content of incoming questions and requests. It then extracts relevant keywords and phrases.

[0238] Step 5:

[0239] The server uses the extracted keywords to search a database containing information about household chores. The search results include resources such as instruction manuals and explanatory videos.

[0240] Step 6:

[0241] After retrieving the appropriate information, the server generates a response to provide to the user. The generated response includes the URL and description of the searched resource.

[0242] Step 7:

[0243] The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0244] Step 8:

[0245] The terminal analyzes the received response data and displays it on the user interface. This allows the user to access suggested household chore methods and resources.

[0246] Step 9:

[0247] Users can configure push notification settings through their device. Once the settings are complete, that information is sent back to the server.

[0248] Step 10:

[0249] The server registers the user's notification settings as a schedule and sends push notifications to the device at the specified time to remind the user to perform household chores.

[0250] (Example 1)

[0251] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0252] In today's busy lifestyle, efficiently managing household tasks and providing users with necessary information quickly and accurately is challenging. In particular, there is a need to significantly reduce the time and effort required in the process of acquiring useful data from multiple sources, appropriately analyzing it, and communicating beneficial results to users.

[0253] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0254] In this invention, the server includes terminal means for receiving information input from users, data processing means for analyzing the received information, and information extraction means for searching for life management-related information based on the analysis results. This enables users to efficiently and easily obtain the necessary information and appropriately manage their daily tasks.

[0255] "Terminal means" refers to a device used by users to input information, such as a smartphone or personal computer.

[0256] "Data processing means" refers to a function that analyzes input information and identifies important attributes.

[0257] An "information extraction method" is a process that has the function of searching for necessary information from relevant databases based on the results of the analysis.

[0258] "Result display means" refers to an output function that conveys searched information to the user in an easy-to-understand manner, and includes systems that display text and links.

[0259] "Communication means" refers to a method of providing a communication function to send notifications to users regarding daily life activities.

[0260] Generative modeling is a machine learning technique used to extract specific patterns or attributes from data.

[0261] An "information link" is connection information that indicates the location of information in a way that users can access.

[0262] This invention is a system for efficiently acquiring information necessary for daily life and managing daily living tasks. First, the user accesses the system using a terminal. This terminal consists of information input devices such as smartphones and personal computers. Through the terminal, the user can operate a dedicated application and input questions and requests related to daily living tasks.

[0263] When a device transmits information, the server receives it. The server utilizes natural language processing technology, specifically generative modeling. For example, it uses generative models such as BERT or GPT to identify important attributes and keywords from the input information. This allows for an accurate understanding of the user's intent.

[0264] Subsequently, the server uses information extraction means to search relevant databases based on the identified keywords. These databases contain procedures and explanatory materials related to the target daily living activity, and the server quickly selects the most relevant information. The results obtained through this process are presented to the user via a results display means.

[0265] The server generates results that include text information and resource links, which can be viewed on the user's device. Finally, using communication methods, push notifications are sent to the device based on the user's set time for daily activities. This helps users remember to perform necessary activities.

[0266] For example, when a user enters a prompt such as, "Please tell me the steps for cleaning the living room," the system provides relevant cleaning instructions and links to explanatory videos. This entire process minimizes the time users spend searching for information, allowing them to efficiently carry out household tasks.

[0267] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0268] Step 1:

[0269] The user launches a dedicated application on their device and enters information. The device receives a natural language prompt, such as, "Please tell me the steps for cleaning the living room." It then prepares to send this input information to the system.

[0270] Step 2:

[0271] The terminal sends the entered prompt text to the server. The server receives the data and verifies the input information as received data. This input data becomes the basis for proceeding to the next analysis step.

[0272] Step 3:

[0273] The server analyzes the input prompt sentence using natural language processing techniques. This analysis utilizes generative AI models such as BERT and GPT to extract important keywords from the sentence, such as "cleaning" and "procedure." The extracted keywords are then output and used in the next search process.

[0274] Step 4:

[0275] The server searches the database based on the keywords obtained in the analysis step. It quickly finds relevant lifestyle information, such as cleaning procedures and links to explanatory videos. The output of this search is information selected as the most suitable resource.

[0276] Step 5:

[0277] The server uses the search results to generate a response message to present to the user. This message includes text information and links and is ready to be sent to the terminal. The generated message is output and ready for display.

[0278] Step 6:

[0279] The terminal displays the response message received from the server in an easy-to-understand format for the user. The user can easily access the information they need. This output allows the user to review the steps for cleaning the living room.

[0280] Step 7:

[0281] When a user sets a schedule for daily task notifications, the server sends push notifications based on specific dates and times. These notifications help users remember to complete the tasks they have set.

[0282] (Application Example 1)

[0283] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart glasses 214 are referred to as a "terminal".

[0284] Due to the lack of efficient work management in physical stores, operations such as cleaning and inventory replenishment may not be properly carried out. As a result, the efficiency of the store's operation may decrease, potentially compromising customer satisfaction. Therefore, a system for supporting and optimizing work in physical stores is necessary.

[0285] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0286] In this invention, the server includes an input means for receiving tasks and services from users, an analysis device means for analyzing the received tasks and services, and a facility work support device means for managing work plans. This enables effective and rapid work management within the store.

[0287] The "user" refers to a person who actually manages home-based work or in-facility work using the system.

[0288] The "tasks and services" refer to specific requirements and questions related to home-based work or in-facility work that the user wishes to solve or implement.

[0289] The "input means" refers to an interface for the user to input tasks and services into the system.

[0290] The "analysis device" refers to a device for understanding the input tasks and services and extracting important information.

[0291] The "detection device" refers to a device for searching for optimal home-based work-related data based on the analyzed information.

[0292] The "response generation device" refers to a device for configuring the detected information in a form to be provided to the user.

[0293] A "reporting device" refers to a device that sends push notifications to users when they are performing household tasks.

[0294] "In-facility work support equipment" refers to equipment that manages work plans within a physical store and supports the effective execution of tasks.

[0295] To realize this invention, a program is needed to construct the entire system. The processing flow of that program is described below in natural language.

[0296] The server is implemented using Node.js and the Express framework. This server receives tasks and services entered by users using their smartphones. The data is sent from a user interface on the device developed using React Native.

[0297] The server then uses the Google Cloud Natural Language API to analyze the tasks and services and identify key concepts. This analysis extracts information related to home-based tasks and in-store operations.

[0298] The server searches the data stored in MongoDB based on the extracted information and finds the relevant information. Then, the response generator constructs the information to be provided to the user as video links or instruction manual links.

[0299] Furthermore, the reporting device sends push notifications to the terminal, prompting users to perform household chores.

[0300] For example, if a store staff member types in, "Please tell me the procedure for wiping dust off shelves. I would also like to know what tools are needed," the server will quickly and accurately support the task by providing appropriate procedure videos and a list of tools.

[0301] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0302] Step 1:

[0303] The user uses a smartphone to input the tasks or services to be performed. The React Native app installed on the terminal receives this input and sends it to the server. The input data includes the content and requirements of the task. A data request to the server is generated as the output.

[0304] Step 2:

[0305] The server uses Node.js and Express to receive the data received from the terminal. The received data is analyzed using the Google Cloud Natural Language API, and important concepts and keywords are extracted. The analyzed input data generates a list of concepts as the output.

[0306] Step 3:

[0307] Based on the concepts obtained from the analysis, the server searches the MongoDB database. The database stores information related to home-based work and in-facility work, and relevant resources are selected. Using the list of concepts as the input, relevant information is obtained as the output.

[0308] Step 4:

[0309] Based on the information obtained, the server generates a response. The response includes links to procedure videos and specific procedure manuals. The information is constructed using the prompt text generated by the generation AI model and provided to the user as the response.

[0310] Step 5:

[0311] The server uses the response information to send a push notification to the terminal. This notification prompts the user to perform the work at an appropriate timing. A notification displayed on the user's terminal is generated as the output.

[0312] Step 6:

[0313] Users efficiently perform household chores and facility-based tasks based on information displayed on their devices. Based on user input, appropriate resources are used to complete the tasks.

[0314] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0315] This invention relates to a system that recognizes the user's emotional state and provides household support. Users can send questions and requests regarding household chores to the system from a device such as a smartphone or PC. These requests can be made through a dedicated application with a natural interface.

[0316] When the terminal receives user input, that information is immediately sent to the server. The server then processes the received data. A particularly noteworthy aspect here is the use of an emotion engine to recognize the user's emotional state. The emotion engine analyzes emotional nuances from the input language data to identify the user's current emotions. This allows for determinations such as whether the user is stressed or relaxed.

[0317] The server then uses natural language processing technology to analyze the question and retrieve relevant household information from its database. The search results include appropriate instructions and explanatory videos. The retrieved information is then used to generate a response that is tailored to the user's emotional state. For example, if the user is confused, a more detailed explanation is provided; if they are calm, only concise information is provided.

[0318] The generated response is sent from the server to the terminal and provided to the user. The terminal updates its interface based on this information, and it is designed to be visually user-friendly. In addition, a push notification function is included, allowing the user to be prompted to do household chores at the appropriate time. The content and timing of the notifications are also adjusted according to the user's emotional state, so appropriate support is provided without causing the user excessive stress.

[0319] For example, suppose a user asks, "How do I clean?" and the server's emotion engine recognizes that the user is tired. In that case, the system selects a simple and effective cleaning method, generates a response that includes a link to a relaxing audio guide, and presents it to the user. In this way, it enables household support that takes the user's emotions into consideration.

[0320] The following describes the processing flow.

[0321] Step 1:

[0322] The user opens the application on their device and enters questions or requests related to household chores. After completing the input, they press the submit button to send the information to the server.

[0323] Step 2:

[0324] The terminal formats the user's input data and sends an HTTP request to the server. During this process, the data is converted to JSON format.

[0325] Step 3:

[0326] The server receives data sent from the terminal and uses an emotion engine to analyze the user's emotional state from their input. For example, it detects emotions from contexts such as "tired" or "stressed."

[0327] Step 4:

[0328] The server uses natural language processing technology to analyze input questions and requests and extract relevant keywords. This allows the type of information the user is specifically seeking to be identified.

[0329] Step 5:

[0330] Based on the analysis results, the server searches the database for household-related information. Appropriate resources such as instruction manuals and explanatory videos are selected.

[0331] Step 6:

[0332] The server considers the emotional state provided by the emotion engine and generates a response based on the retrieved information. For example, if the user is feeling stressed, the response will include encouraging messages and information that can help with relaxation.

[0333] Step 7:

[0334] The server sends the generated response to the terminal in JSON format.

[0335] Step 8:

[0336] The terminal receives a response from the server and displays it on the user interface. The user then reviews the information provided and uses it to help with household chores.

[0337] Step 9:

[0338] Users can schedule appropriate push notifications through their devices. This information is then sent back to the server.

[0339] Step 10:

[0340] The server is configured to send push notifications at specific times based on the user's notification settings. The timing and content of notifications are optimized according to the user's emotional state.

[0341] (Example 2)

[0342] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0343] In recent years, there has been a growing demand for efficient performance of various tasks in daily life. However, conventional systems that provide support while considering the emotional state of the user are not yet fully established. When a user is stressed or in a particular emotional state, the information provided may not always be appropriate. Therefore, there is a need for a system that can facilitate daily life tasks by providing information tailored to the user's emotions.

[0344] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0345] In this invention, the server includes an input device means for receiving inquiries and requests from users, an emotion analysis device means for performing natural language processing to recognize the user's emotions, and a response adjustment device means for adjusting information according to the user's emotional state. This makes it possible to provide life support optimized according to the user's emotional state.

[0346] An "input device" is a device used to receive inquiries and requests from users.

[0347] An "analysis device" is a device used to analyze inquiries and requests received.

[0348] A "search device" is a device used to find information related to daily living activities based on the results of analysis.

[0349] A "response generation device" is a device that presents the searched information to the user.

[0350] A "notification device" is a device that sends out notifications to encourage the performance of daily living activities.

[0351] An "emotion analysis device" is a device that performs natural language processing to recognize the emotions of its users.

[0352] A "response adjustment device" is a device that adjusts information according to the user's emotional state.

[0353] This invention embodies a system that recognizes the user's emotional state and provides support for daily living activities accordingly. This system allows users to send questions and requests related to daily living activities using a smartphone or personal computer (terminal). By using a dedicated application, users can operate the system through an intuitive and natural interface.

[0354] The terminal receives input from the user and sends that data to the server. The server analyzes the data using natural language processing technology. This analysis utilizes an emotion analysis device to extract emotional nuances from the user's input data. Emotion analysis is a crucial process for recognizing what emotional state the user is in, such as stress or relaxation.

[0355] Based on the analysis results, the server searches its internal database for relevant life skills information. This database contains procedures, explanatory videos, and other information that addresses the user's questions. After the information is retrieved, the server generates a response appropriate to the user's emotional state. For example, if the user is confused, it provides a detailed guide; if they are calm, it presents concise information.

[0356] Ultimately, the server sends the generated response to the terminal and presents it to the user. The terminal updates its interface based on the received information, displaying it clearly. The terminal also has a push notification function, which can prompt the user to perform daily tasks at the appropriate time. The content and timing of notifications are also adjusted according to the user's emotional state, so support is provided without causing excessive stress.

[0357] For example, if a user asks, "How do I clean?", the server, through its emotion engine, recognizes that the user is tired. As a result, the system suggests a simple and effective way to clean, and generates a response that also includes a link to a relaxing audio guide.

[0358] An example of a prompt message might be: "The user wants to know how to clean, but appears tired. How should you respond?"

[0359] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0360] Step 1:

[0361] Users open a dedicated application via their smartphone or personal computer and input questions and requests related to daily life tasks. The input data is in text or voice format and uses a natural interface. The terminal immediately transmits this input data to the server. The input data includes the user's questions and voice data. The output is the transmitted user data.

[0362] Step 2:

[0363] The server passes the data received from the terminal to the emotion analysis device. The emotion analysis device uses natural language processing technology to analyze the text or audio data and recognize the user's emotions. The analysis evaluates the frequency of positive or negative words and the tone of the text to determine the degree of stress or relaxation. The input is the user's text or audio data, and the output is the result of the determination of the user's emotional state.

[0364] Step 3:

[0365] The server analyzes the user's inquiry based on their emotional state, which is obtained through sentiment analysis. It utilizes natural language processing techniques to extract key keywords from the questions and requests. This information is then used to search the server's database for information related to daily tasks. The input is the user's inquiry and emotional state, and the output is a list of related information.

[0366] Step 4:

[0367] The server uses a response generator based on the search results to create a response tailored to the user. It adjusts the information to match the user's emotional state and generates a message that includes detailed instructions or concise guides. It may also include relevant images, video links, or audio guides. The input is the searched information and emotional state, and the output is the generated response message.

[0368] Step 5:

[0369] The server sends the generated response to the terminal. The terminal receives this information and updates the interface in a user-friendly format. In addition to visual displays, push notifications deliver timely messages to encourage daily tasks. The content and timing of notifications are also adjusted based on the user's emotional state. The input is the response message sent from the server, and the output is the updated interface and the sent notifications.

[0370] (Application Example 2)

[0371] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0372] It is important for modern consumers to be able to easily obtain product information and have a comfortable shopping experience. However, when customers ask questions about products in stores, they may experience stress if the information provided does not take into account their individual emotional state. This invention aims to enable the provision of optimal product information based on an analysis of the customer's emotional state.

[0373] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0374] In this invention, the server includes an input means for receiving questions and requests from the user, an emotion analysis means for analyzing the received questions and requests and recognizing the user's emotional state, a search means for searching for product-related information based on the analysis results, and a response generation means for providing the searched information in an optimal form according to the user's emotional state. This makes it possible to provide product information that matches the emotional state of each individual customer and realize a comfortable purchasing experience.

[0375] An "input method" is an interface for receiving questions and requests from users.

[0376] "Emotional analysis tools" are functions that analyze received questions and requests to identify the user's emotional state.

[0377] A "search tool" is a function that extracts and provides relevant information based on the analyzed results.

[0378] A "response generation means" is a process for providing the user with extracted information in the most optimal form, according to the user's emotional state.

[0379] "Notification methods" refer to a function that sends necessary information and guidance as push notifications, tailored to the user's emotional state.

[0380] This embodiment includes a server, a user terminal, and a network for exchanging data between them. The user first uses the terminal to input questions or requests for product information or assistance with household chores. This data is then sent to the server via the input means. The server uses sentiment analysis means to identify the user's emotional state from this data. Natural language processing tools such as TextBlob are used in this process.

[0381] Based on the analysis results, the server uses search tools to find relevant information. This information is extracted from existing resources stored in the database. The response generation tool optimizes the information according to the user's emotional state and constructs specific content to be provided to the user. The information provided at this time takes the form of a detailed explanation or a concise summary, depending on the user's emotions.

[0382] For example, if a user enters "I want to know more about this product, but I'm in a hurry" into a terminal in a store, the server detects that the user is experiencing stress. As a result, the response generation mechanism quickly presents concise information summarizing the product's main features. The notification mechanism sends this information to the terminal as a push notification, timely and efficiently delivered to the user.

[0383] In this way, the system can provide information tailored to each customer's emotional state and appropriately meet their needs.

[0384] An example of a prompt message for a generative AI model might be: "Calculate emotional polarity from customer input, determine the emotional state, and dynamically generate information based on the result. For example, if the polarity is -0.3, determine that the customer is stressed and provide detailed guidance."

[0385] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0386] Step 1:

[0387] The user enters questions or requests about the product via a terminal. This data is then sent to the server. This step includes the process up to the point where the user's input is sent to the server as text data.

[0388] Step 2:

[0389] The server passes the received text data to a sentiment analysis tool. Here, the text data is analyzed using a natural language processing library (e.g., TextBlob), and its polarity value is calculated. Based on this polarity value, the user's emotional state is classified as "relaxed," "stressed," or "neutral."

[0390] Step 3:

[0391] The server uses search methods to retrieve relevant information from the database based on the emotional state. If the emotional state is stress, only concise and essential product information is extracted.

[0392] Step 4:

[0393] The server's response generation mechanism constructs content to provide the user with information obtained in the most optimal format, depending on their emotional state. Specifically, if the user is experiencing stress, concise content emphasizing the most important points is generated.

[0394] Step 5:

[0395] Content generated by the notification system is sent to the device as a push notification. The user receives the notification and can obtain the information appropriately. This ensures that the information the user needs is provided in a timely manner, guaranteeing a smooth user experience.

[0396] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0397] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0398] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0399] [Third Embodiment]

[0400] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0401] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0402] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0403] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0404] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0405] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0406] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0407] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0408] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0409] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0410] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0411] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0412] This invention provides a system to support users with household chore-related challenges. First, the user operates a dedicated application using a device such as a smartphone or PC. The application is designed to allow easy input of questions and requests regarding household chores through its user interface. Once the user completes the input, the information is immediately sent to a server for analysis.

[0413] Next, the server analyzes the information submitted by the user using natural language processing techniques to extract important keywords. The results obtained from this analysis form the basis for the subsequent search process. Based on the extracted keywords, the server searches the database for relevant household information, such as cleaning procedures and instructional videos. This search process is performed quickly and accurately to select the most relevant information for the user.

[0414] The server then uses the retrieved information to generate a response for the user. This response includes links to resources and concise advice, designed to help the user perform household chores effectively. The generated response is sent to the terminal and displayed to the user.

[0415] Furthermore, to ensure users don't forget to do household chores, the device is equipped with a push notification function. Users can set notification schedules within the application, for example, to receive cleaning reminders on specific days. These notifications are managed through coordination with a server.

[0416] As a concrete example, consider a scenario where a user wants to know how to clean their living room. The user inputs "living room cleaning procedure," and the server, after analysis, generates a response including relevant videos and instructions, which are then displayed on the device. This process enables the user to efficiently perform household chores and receive support for a more comfortable life.

[0417] The following describes the processing flow.

[0418] Step 1:

[0419] The user accesses the application on their device and enters questions or requests related to household chores. The entered information is sent to the server when the submit button is pressed.

[0420] Step 2:

[0421] The terminal converts the data entered by the user into JSON format and sends an HTTP request to the server's API endpoint.

[0422] Step 3:

[0423] The server receives data from the terminal for analysis. The received data is first validated to confirm that it is in the correct format.

[0424] Step 4:

[0425] The server uses a natural language processing (NLP) module to analyze the content of incoming questions and requests. It then extracts relevant keywords and phrases.

[0426] Step 5:

[0427] The server uses the extracted keywords to search a database containing information about household chores. The search results include resources such as instruction manuals and explanatory videos.

[0428] Step 6:

[0429] After retrieving the appropriate information, the server generates a response to provide to the user. The generated response includes the URL and description of the searched resource.

[0430] Step 7:

[0431] The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0432] Step 8:

[0433] The terminal analyzes the received response data and displays it on the user interface. This allows the user to access suggested household chore methods and resources.

[0434] Step 9:

[0435] Users can configure push notification settings through their device. Once the settings are complete, that information is sent back to the server.

[0436] Step 10:

[0437] The server registers the user's notification settings as a schedule and sends push notifications to the device at the specified time to remind the user to perform household chores.

[0438] (Example 1)

[0439] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0440] In today's busy lifestyle, efficiently managing household tasks and providing users with necessary information quickly and accurately is challenging. In particular, there is a need to significantly reduce the time and effort required in the process of acquiring useful data from multiple sources, appropriately analyzing it, and communicating beneficial results to users.

[0441] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0442] In this invention, the server includes terminal means for receiving information input from users, data processing means for analyzing the received information, and information extraction means for searching for life management-related information based on the analysis results. This enables users to efficiently and easily obtain the necessary information and appropriately manage their daily tasks.

[0443] "Terminal means" refers to a device used by users to input information, such as a smartphone or personal computer.

[0444] "Data processing means" refers to a function that analyzes input information and identifies important attributes.

[0445] An "information extraction method" is a process that has the function of searching for necessary information from relevant databases based on the results of the analysis.

[0446] "Result display means" refers to an output function that conveys searched information to the user in an easy-to-understand manner, and includes systems that display text and links.

[0447] "Communication means" refers to a method of providing a communication function to send notifications to users regarding daily life activities.

[0448] Generative modeling is a machine learning technique used to extract specific patterns or attributes from data.

[0449] An "information link" is connection information that indicates the location of information in a way that users can access.

[0450] This invention is a system for efficiently acquiring information necessary for daily life and managing daily living tasks. First, the user accesses the system using a terminal. This terminal consists of information input devices such as smartphones and personal computers. Through the terminal, the user can operate a dedicated application and input questions and requests related to daily living tasks.

[0451] When a device transmits information, the server receives it. The server utilizes natural language processing technology, specifically generative modeling. For example, it uses generative models such as BERT or GPT to identify important attributes and keywords from the input information. This allows for an accurate understanding of the user's intent.

[0452] Subsequently, the server uses information extraction means to search relevant databases based on the identified keywords. These databases contain procedures and explanatory materials related to the target daily living activity, and the server quickly selects the most relevant information. The results obtained through this process are presented to the user via a results display means.

[0453] The server generates results that include text information and resource links, which can be viewed on the user's device. Finally, using communication methods, push notifications are sent to the device based on the user's set time for daily activities. This helps users remember to perform necessary activities.

[0454] For example, when a user enters a prompt such as, "Please tell me the steps for cleaning the living room," the system provides relevant cleaning instructions and links to explanatory videos. This entire process minimizes the time users spend searching for information, allowing them to efficiently carry out household tasks.

[0455] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0456] Step 1:

[0457] The user launches a dedicated application on their device and enters information. The device receives a natural language prompt, such as, "Please tell me the steps for cleaning the living room." It then prepares to send this input information to the system.

[0458] Step 2:

[0459] The terminal sends the entered prompt text to the server. The server receives the data and verifies the input information as received data. This input data becomes the basis for proceeding to the next analysis step.

[0460] Step 3:

[0461] The server analyzes the input prompt sentence using natural language processing techniques. This analysis utilizes generative AI models such as BERT and GPT to extract important keywords from the sentence, such as "cleaning" and "procedure." The extracted keywords are then output and used in the next search process.

[0462] Step 4:

[0463] The server searches the database based on the keywords obtained in the analysis step. It quickly finds relevant lifestyle information, such as cleaning procedures and links to explanatory videos. The output of this search is information selected as the most suitable resource.

[0464] Step 5:

[0465] The server uses the search results to generate a response message to present to the user. This message includes text information and links and is ready to be sent to the terminal. The generated message is output and ready for display.

[0466] Step 6:

[0467] The terminal displays the response message received from the server in an easy-to-understand format for the user. The user can easily access the information they need. This output allows the user to review the steps for cleaning the living room.

[0468] Step 7:

[0469] When a user sets a schedule for daily task notifications, the server sends push notifications based on specific dates and times. These notifications help users remember to complete the tasks they have set.

[0470] (Application Example 1)

[0471] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0472] A lack of efficient work management within physical stores can lead to improperly performed tasks such as cleaning and inventory replenishment. This can result in decreased operational efficiency and potentially damage customer satisfaction. Therefore, a system is needed to support and streamline operations within physical stores.

[0473] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0474] In this invention, the server includes input means for receiving tasks and services from users, analysis device means for analyzing the received tasks and services, and in-facility work support device means for managing work plans. This enables effective and rapid work management within the store.

[0475] "Users" refer to individuals who actually use the system to manage household tasks or facility-based work.

[0476] "Issues and services" refer to specific requests or questions related to household tasks or facility work that users wish to resolve or perform.

[0477] "Input method" refers to the interface through which users input tasks or services into the system.

[0478] An "analysis device" refers to a device that understands the input tasks or services and extracts important information.

[0479] A "detection device" refers to a device used to find the most relevant household business data based on the analyzed information.

[0480] A "response generation device" refers to a device that configures detected information to be provided to the user.

[0481] A "reporting device" refers to a device that sends push notifications to users when they are performing household tasks.

[0482] "In-facility work support equipment" refers to equipment that manages work plans within a physical store and supports the effective execution of tasks.

[0483] To realize this invention, a program is needed to construct the entire system. The processing flow of that program is described below in natural language.

[0484] The server is implemented using Node.js and the Express framework. This server receives tasks and services entered by users using their smartphones. The data is sent from a user interface on the device developed using React Native.

[0485] The server then uses the Google Cloud Natural Language API to analyze the tasks and services and identify key concepts. This analysis extracts information related to home-based tasks and in-store operations.

[0486] The server searches the data stored in MongoDB based on the extracted information and finds the relevant information. Then, the response generator constructs the information to be provided to the user as video links or instruction manual links.

[0487] Furthermore, the reporting device sends push notifications to the terminal, prompting users to perform household chores.

[0488] For example, if a store staff member types in, "Please tell me the procedure for wiping dust off shelves. I would also like to know what tools are needed," the server will quickly and accurately support the task by providing appropriate procedure videos and a list of tools.

[0489] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0490] Step 1:

[0491] The user uses their smartphone to input the tasks or services they wish to perform. A React Native app installed on the device receives this input and sends it to the server. The input data includes the details of the task and the user's requests. The output is a data request sent to the server.

[0492] Step 2:

[0493] The server uses Node.js and Express to receive data from the terminal. The received data is parsed using the Google Cloud Natural Language API, and important concepts and keywords are extracted. The parsed input data generates a list of concepts as output.

[0494] Step 3:

[0495] The server searches the MongoDB database based on the concepts obtained from the analysis. The database contains information related to household tasks and facility-based tasks, and relevant resources are selected. A list of concepts is used as input, and the relevant information is retrieved as output.

[0496] Step 4:

[0497] Based on the acquired information, the server generates a response. The response includes links to instructional videos and specific instruction manuals. The information is constructed using prompts generated by the generation AI model and provided to the user as a response.

[0498] Step 5:

[0499] The server uses the response information to send a push notification to the terminal. This notification prompts the user to perform their task at the appropriate time. The output is a notification that is displayed on the user's terminal.

[0500] Step 6:

[0501] Users efficiently perform household chores and facility-based tasks based on information displayed on their devices. Based on user input, appropriate resources are used to complete the tasks.

[0502] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0503] This invention relates to a system that recognizes the user's emotional state and provides household support. Users can send questions and requests regarding household chores to the system from a device such as a smartphone or PC. These requests can be made through a dedicated application with a natural interface.

[0504] When the terminal receives user input, that information is immediately sent to the server. The server then processes the received data. A particularly noteworthy aspect here is the use of an emotion engine to recognize the user's emotional state. The emotion engine analyzes emotional nuances from the input language data to identify the user's current emotions. This allows for determinations such as whether the user is stressed or relaxed.

[0505] The server then uses natural language processing technology to analyze the question and retrieve relevant household information from its database. The search results include appropriate instructions and explanatory videos. The retrieved information is then used to generate a response that is tailored to the user's emotional state. For example, if the user is confused, a more detailed explanation is provided; if they are calm, only concise information is provided.

[0506] The generated response is sent from the server to the terminal and provided to the user. The terminal updates its interface based on this information, and it is designed to be visually user-friendly. In addition, a push notification function is included, allowing the user to be prompted to do household chores at the appropriate time. The content and timing of the notifications are also adjusted according to the user's emotional state, so appropriate support is provided without causing the user excessive stress.

[0507] For example, suppose a user asks, "How do I clean?" and the server's emotion engine recognizes that the user is tired. In that case, the system selects a simple and effective cleaning method, generates a response that includes a link to a relaxing audio guide, and presents it to the user. In this way, it enables household support that takes the user's emotions into consideration.

[0508] The following describes the processing flow.

[0509] Step 1:

[0510] The user opens the application on their device and enters questions or requests related to household chores. After completing the input, they press the submit button to send the information to the server.

[0511] Step 2:

[0512] The terminal formats the user's input data and sends an HTTP request to the server. During this process, the data is converted to JSON format.

[0513] Step 3:

[0514] The server receives data sent from the terminal and uses an emotion engine to analyze the user's emotional state from their input. For example, it detects emotions from contexts such as "tired" or "stressed."

[0515] Step 4:

[0516] The server uses natural language processing technology to analyze input questions and requests and extract relevant keywords. This allows the type of information the user is specifically seeking to be identified.

[0517] Step 5:

[0518] Based on the analysis results, the server searches the database for household-related information. Appropriate resources such as instruction manuals and explanatory videos are selected.

[0519] Step 6:

[0520] The server considers the emotional state provided by the emotion engine and generates a response based on the retrieved information. For example, if the user is feeling stressed, the response will include encouraging messages and information that can help with relaxation.

[0521] Step 7:

[0522] The server sends the generated response to the terminal in JSON format.

[0523] Step 8:

[0524] The terminal receives a response from the server and displays it on the user interface. The user then reviews the information provided and uses it to help with household chores.

[0525] Step 9:

[0526] Users can schedule appropriate push notifications through their devices. This information is then sent back to the server.

[0527] Step 10:

[0528] The server is configured to send push notifications at specific times based on the user's notification settings. The timing and content of notifications are optimized according to the user's emotional state.

[0529] (Example 2)

[0530] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0531] In recent years, there has been a growing demand for efficient performance of various tasks in daily life. However, conventional systems that provide support while considering the emotional state of the user are not yet fully established. When a user is stressed or in a particular emotional state, the information provided may not always be appropriate. Therefore, there is a need for a system that can facilitate daily life tasks by providing information tailored to the user's emotions.

[0532] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0533] In this invention, the server includes an input device means for receiving inquiries and requests from users, an emotion analysis device means for performing natural language processing to recognize the user's emotions, and a response adjustment device means for adjusting information according to the user's emotional state. This makes it possible to provide life support optimized according to the user's emotional state.

[0534] An "input device" is a device used to receive inquiries and requests from users.

[0535] An "analysis device" is a device used to analyze inquiries and requests received.

[0536] A "search device" is a device used to find information related to daily living activities based on the results of analysis.

[0537] A "response generation device" is a device that presents the searched information to the user.

[0538] A "notification device" is a device that sends out notifications to encourage the performance of daily living activities.

[0539] An "emotion analysis device" is a device that performs natural language processing to recognize the emotions of its users.

[0540] A "response adjustment device" is a device that adjusts information according to the user's emotional state.

[0541] This invention embodies a system that recognizes the user's emotional state and provides support for daily living activities accordingly. This system allows users to send questions and requests related to daily living activities using a smartphone or personal computer (terminal). By using a dedicated application, users can operate the system through an intuitive and natural interface.

[0542] The terminal receives input from the user and sends that data to the server. The server analyzes the data using natural language processing technology. This analysis utilizes an emotion analysis device to extract emotional nuances from the user's input data. Emotion analysis is a crucial process for recognizing what emotional state the user is in, such as stress or relaxation.

[0543] Based on the analysis results, the server searches its internal database for relevant life skills information. This database contains procedures, explanatory videos, and other information that addresses the user's questions. After the information is retrieved, the server generates a response appropriate to the user's emotional state. For example, if the user is confused, it provides a detailed guide; if they are calm, it presents concise information.

[0544] Ultimately, the server sends the generated response to the terminal and presents it to the user. The terminal updates its interface based on the received information, displaying it clearly. The terminal also has a push notification function, which can prompt the user to perform daily tasks at the appropriate time. The content and timing of notifications are also adjusted according to the user's emotional state, so support is provided without causing excessive stress.

[0545] For example, if a user asks, "How do I clean?", the server, through its emotion engine, recognizes that the user is tired. As a result, the system suggests a simple and effective way to clean, and generates a response that also includes a link to a relaxing audio guide.

[0546] An example of a prompt message might be: "The user wants to know how to clean, but appears tired. How should you respond?"

[0547] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0548] Step 1:

[0549] Users open a dedicated application via their smartphone or personal computer and input questions and requests related to daily life tasks. The input data is in text or voice format and uses a natural interface. The terminal immediately transmits this input data to the server. The input data includes the user's questions and voice data. The output is the transmitted user data.

[0550] Step 2:

[0551] The server passes the data received from the terminal to the emotion analysis device. The emotion analysis device uses natural language processing technology to analyze the text or audio data and recognize the user's emotions. The analysis evaluates the frequency of positive or negative words and the tone of the text to determine the degree of stress or relaxation. The input is the user's text or audio data, and the output is the result of the determination of the user's emotional state.

[0552] Step 3:

[0553] The server analyzes the user's inquiry based on their emotional state, which is obtained through sentiment analysis. It utilizes natural language processing techniques to extract key keywords from the questions and requests. This information is then used to search the server's database for information related to daily tasks. The input is the user's inquiry and emotional state, and the output is a list of related information.

[0554] Step 4:

[0555] The server uses a response generator based on the search results to create a response tailored to the user. It adjusts the information to match the user's emotional state and generates a message that includes detailed instructions or concise guides. It may also include relevant images, video links, or audio guides. The input is the searched information and emotional state, and the output is the generated response message.

[0556] Step 5:

[0557] The server sends the generated response to the terminal. The terminal receives this information and updates the interface in a user-friendly format. In addition to visual displays, push notifications deliver timely messages to encourage daily tasks. The content and timing of notifications are also adjusted based on the user's emotional state. The input is the response message sent from the server, and the output is the updated interface and the sent notifications.

[0558] (Application Example 2)

[0559] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0560] It is important for modern consumers to be able to easily obtain product information and have a comfortable shopping experience. However, when customers ask questions about products in stores, they may experience stress if the information provided does not take into account their individual emotional state. This invention aims to enable the provision of optimal product information based on an analysis of the customer's emotional state.

[0561] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0562] In this invention, the server includes an input means for receiving questions and requests from the user, an emotion analysis means for analyzing the received questions and requests and recognizing the user's emotional state, a search means for searching for product-related information based on the analysis results, and a response generation means for providing the searched information in an optimal form according to the user's emotional state. This makes it possible to provide product information that matches the emotional state of each individual customer and realize a comfortable purchasing experience.

[0563] An "input method" is an interface for receiving questions and requests from users.

[0564] "Emotional analysis tools" are functions that analyze received questions and requests to identify the user's emotional state.

[0565] A "search tool" is a function that extracts and provides relevant information based on the analyzed results.

[0566] A "response generation means" is a process for providing the user with extracted information in the most optimal form, according to the user's emotional state.

[0567] "Notification methods" refer to a function that sends necessary information and guidance as push notifications, tailored to the user's emotional state.

[0568] This embodiment includes a server, a user terminal, and a network for exchanging data between them. The user first uses the terminal to input questions or requests for product information or assistance with household chores. This data is then sent to the server via the input means. The server uses sentiment analysis means to identify the user's emotional state from this data. Natural language processing tools such as TextBlob are used in this process.

[0569] Based on the analysis results, the server uses search tools to find relevant information. This information is extracted from existing resources stored in the database. The response generation tool optimizes the information according to the user's emotional state and constructs specific content to be provided to the user. The information provided at this time takes the form of a detailed explanation or a concise summary, depending on the user's emotions.

[0570] For example, if a user enters "I want to know more about this product, but I'm in a hurry" into a terminal in a store, the server detects that the user is experiencing stress. As a result, the response generation mechanism quickly presents concise information summarizing the product's main features. The notification mechanism sends this information to the terminal as a push notification, timely and efficiently delivered to the user.

[0571] In this way, the system can provide information tailored to each customer's emotional state and appropriately meet their needs.

[0572] An example of a prompt message for a generative AI model might be: "Calculate emotional polarity from customer input, determine the emotional state, and dynamically generate information based on the result. For example, if the polarity is -0.3, determine that the customer is stressed and provide detailed guidance."

[0573] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0574] Step 1:

[0575] The user enters questions or requests about the product via a terminal. This data is then sent to the server. This step includes the process up to the point where the user's input is sent to the server as text data.

[0576] Step 2:

[0577] The server passes the received text data to a sentiment analysis tool. Here, the text data is analyzed using a natural language processing library (e.g., TextBlob), and its polarity value is calculated. Based on this polarity value, the user's emotional state is classified as "relaxed," "stressed," or "neutral."

[0578] Step 3:

[0579] The server uses search methods to retrieve relevant information from the database based on the emotional state. If the emotional state is stress, only concise and essential product information is extracted.

[0580] Step 4:

[0581] The server's response generation mechanism constructs content to provide the user with information obtained in the most optimal format, depending on their emotional state. Specifically, if the user is experiencing stress, concise content emphasizing the most important points is generated.

[0582] Step 5:

[0583] Content generated by the notification system is sent to the device as a push notification. The user receives the notification and can obtain the information appropriately. This ensures that the information the user needs is provided in a timely manner, guaranteeing a smooth user experience.

[0584] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0585] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0586] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0587] [Fourth Embodiment]

[0588] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0589] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0590] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0591] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0592] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0593] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0594] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0595] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0596] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0597] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0598] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0599] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0600] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0601] This invention provides a system to support users with household chore-related challenges. First, the user operates a dedicated application using a device such as a smartphone or PC. The application is designed to allow easy input of questions and requests regarding household chores through its user interface. Once the user completes the input, the information is immediately sent to a server for analysis.

[0602] Next, the server analyzes the information submitted by the user using natural language processing techniques to extract important keywords. The results obtained from this analysis form the basis for the subsequent search process. Based on the extracted keywords, the server searches the database for relevant household information, such as cleaning procedures and instructional videos. This search process is performed quickly and accurately to select the most relevant information for the user.

[0603] The server then uses the retrieved information to generate a response for the user. This response includes links to resources and concise advice, designed to help the user perform household chores effectively. The generated response is sent to the terminal and displayed to the user.

[0604] Furthermore, to ensure users don't forget to do household chores, the device is equipped with a push notification function. Users can set notification schedules within the application, for example, to receive cleaning reminders on specific days. These notifications are managed through coordination with a server.

[0605] As a concrete example, consider a scenario where a user wants to know how to clean their living room. The user inputs "living room cleaning procedure," and the server, after analysis, generates a response including relevant videos and instructions, which are then displayed on the device. This process enables the user to efficiently perform household chores and receive support for a more comfortable life.

[0606] The following describes the processing flow.

[0607] Step 1:

[0608] The user accesses the application on their device and enters questions or requests related to household chores. The entered information is sent to the server when the submit button is pressed.

[0609] Step 2:

[0610] The terminal converts the data entered by the user into JSON format and sends an HTTP request to the server's API endpoint.

[0611] Step 3:

[0612] The server receives data from the terminal for analysis. The received data is first validated to confirm that it is in the correct format.

[0613] Step 4:

[0614] The server uses a natural language processing (NLP) module to analyze the content of incoming questions and requests. It then extracts relevant keywords and phrases.

[0615] Step 5:

[0616] The server uses the extracted keywords to search a database containing information about household chores. The search results include resources such as instruction manuals and explanatory videos.

[0617] Step 6:

[0618] After retrieving the appropriate information, the server generates a response to provide to the user. The generated response includes the URL and description of the searched resource.

[0619] Step 7:

[0620] The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0621] Step 8:

[0622] The terminal analyzes the received response data and displays it on the user interface. This allows the user to access suggested household chore methods and resources.

[0623] Step 9:

[0624] Users can configure push notification settings through their device. Once the settings are complete, that information is sent back to the server.

[0625] Step 10:

[0626] The server registers the user's notification settings as a schedule and sends push notifications to the device at the specified time to remind the user to perform household chores.

[0627] (Example 1)

[0628] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0629] In today's busy lifestyle, efficiently managing household tasks and providing users with necessary information quickly and accurately is challenging. In particular, there is a need to significantly reduce the time and effort required in the process of acquiring useful data from multiple sources, appropriately analyzing it, and communicating beneficial results to users.

[0630] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0631] In this invention, the server includes terminal means for receiving information input from users, data processing means for analyzing the received information, and information extraction means for searching for life management-related information based on the analysis results. This enables users to efficiently and easily obtain the necessary information and appropriately manage their daily tasks.

[0632] "Terminal means" refers to a device used by users to input information, such as a smartphone or personal computer.

[0633] "Data processing means" refers to a function that analyzes input information and identifies important attributes.

[0634] An "information extraction method" is a process that has the function of searching for necessary information from relevant databases based on the results of the analysis.

[0635] "Result display means" refers to an output function that conveys searched information to the user in an easy-to-understand manner, and includes systems that display text and links.

[0636] "Communication means" refers to a method of providing a communication function to send notifications to users regarding daily life activities.

[0637] Generative modeling is a machine learning technique used to extract specific patterns or attributes from data.

[0638] An "information link" is connection information that indicates the location of information in a way that users can access.

[0639] This invention is a system for efficiently acquiring information necessary for daily life and managing daily living tasks. First, the user accesses the system using a terminal. This terminal consists of information input devices such as smartphones and personal computers. Through the terminal, the user can operate a dedicated application and input questions and requests related to daily living tasks.

[0640] When a device transmits information, the server receives it. The server utilizes natural language processing technology, specifically generative modeling. For example, it uses generative models such as BERT or GPT to identify important attributes and keywords from the input information. This allows for an accurate understanding of the user's intent.

[0641] Subsequently, the server uses information extraction means to search relevant databases based on the identified keywords. These databases contain procedures and explanatory materials related to the target daily living activity, and the server quickly selects the most relevant information. The results obtained through this process are presented to the user via a results display means.

[0642] The server generates results that include text information and resource links, which can be viewed on the user's device. Finally, using communication methods, push notifications are sent to the device based on the user's set time for daily activities. This helps users remember to perform necessary activities.

[0643] For example, when a user enters a prompt such as, "Please tell me the steps for cleaning the living room," the system provides relevant cleaning instructions and links to explanatory videos. This entire process minimizes the time users spend searching for information, allowing them to efficiently carry out household tasks.

[0644] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0645] Step 1:

[0646] The user launches a dedicated application on their device and enters information. The device receives a natural language prompt, such as, "Please tell me the steps for cleaning the living room." It then prepares to send this input information to the system.

[0647] Step 2:

[0648] The terminal sends the entered prompt text to the server. The server receives the data and verifies the input information as received data. This input data becomes the basis for proceeding to the next analysis step.

[0649] Step 3:

[0650] The server analyzes the input prompt sentence using natural language processing techniques. This analysis utilizes generative AI models such as BERT and GPT to extract important keywords from the sentence, such as "cleaning" and "procedure." The extracted keywords are then output and used in the next search process.

[0651] Step 4:

[0652] The server searches the database based on the keywords obtained in the analysis step. It quickly finds relevant lifestyle information, such as cleaning procedures and links to explanatory videos. The output of this search is information selected as the most suitable resource.

[0653] Step 5:

[0654] The server uses the search results to generate a response message to present to the user. This message includes text information and links and is ready to be sent to the terminal. The generated message is output and ready for display.

[0655] Step 6:

[0656] The terminal displays the response message received from the server in an easy-to-understand format for the user. The user can easily access the information they need. This output allows the user to review the steps for cleaning the living room.

[0657] Step 7:

[0658] When a user sets a schedule for daily task notifications, the server sends push notifications based on specific dates and times. These notifications help users remember to complete the tasks they have set.

[0659] (Application Example 1)

[0660] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0661] A lack of efficient work management within physical stores can lead to improperly performed tasks such as cleaning and inventory replenishment. This can result in decreased operational efficiency and potentially damage customer satisfaction. Therefore, a system is needed to support and streamline operations within physical stores.

[0662] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0663] In this invention, the server includes input means for receiving tasks and services from users, analysis device means for analyzing the received tasks and services, and in-facility work support device means for managing work plans. This enables effective and rapid work management within the store.

[0664] "Users" refer to individuals who actually use the system to manage household tasks or facility-based work.

[0665] "Issues and services" refer to specific requests or questions related to household tasks or facility work that users wish to resolve or perform.

[0666] "Input method" refers to the interface through which users input tasks or services into the system.

[0667] An "analysis device" refers to a device that understands the input tasks or services and extracts important information.

[0668] A "detection device" refers to a device used to find the most relevant household business data based on the analyzed information.

[0669] A "response generation device" refers to a device that configures detected information to be provided to the user.

[0670] A "reporting device" refers to a device that sends push notifications to users when they are performing household tasks.

[0671] "In-facility work support equipment" refers to equipment that manages work plans within a physical store and supports the effective execution of tasks.

[0672] To realize this invention, a program is needed to construct the entire system. The processing flow of that program is described below in natural language.

[0673] The server is implemented using Node.js and the Express framework. This server receives tasks and services entered by users using their smartphones. The data is sent from a user interface on the device developed using React Native.

[0674] The server then uses the Google Cloud Natural Language API to analyze the tasks and services and identify key concepts. This analysis extracts information related to home-based tasks and in-store operations.

[0675] The server searches the data stored in MongoDB based on the extracted information and finds the relevant information. Then, the response generator constructs the information to be provided to the user as video links or instruction manual links.

[0676] Furthermore, the reporting device sends push notifications to the terminal, prompting users to perform household chores.

[0677] For example, if a store staff member types in, "Please tell me the procedure for wiping dust off shelves. I would also like to know what tools are needed," the server will quickly and accurately support the task by providing appropriate procedure videos and a list of tools.

[0678] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0679] Step 1:

[0680] The user uses their smartphone to input the tasks or services they wish to perform. A React Native app installed on the device receives this input and sends it to the server. The input data includes the details of the task and the user's requests. The output is a data request sent to the server.

[0681] Step 2:

[0682] The server uses Node.js and Express to receive data from the terminal. The received data is parsed using the Google Cloud Natural Language API, and important concepts and keywords are extracted. The parsed input data generates a list of concepts as output.

[0683] Step 3:

[0684] The server searches the MongoDB database based on the concepts obtained from the analysis. The database contains information related to household tasks and facility-based tasks, and relevant resources are selected. A list of concepts is used as input, and the relevant information is retrieved as output.

[0685] Step 4:

[0686] Based on the acquired information, the server generates a response. The response includes links to instructional videos and specific instruction manuals. The information is constructed using prompts generated by the generation AI model and provided to the user as a response.

[0687] Step 5:

[0688] The server uses the response information to send a push notification to the terminal. This notification prompts the user to perform their task at the appropriate time. The output is a notification that is displayed on the user's terminal.

[0689] Step 6:

[0690] Users efficiently perform household chores and facility-based tasks based on information displayed on their devices. Based on user input, appropriate resources are used to complete the tasks.

[0691] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0692] This invention relates to a system that recognizes the user's emotional state and provides household support. Users can send questions and requests regarding household chores to the system from a device such as a smartphone or PC. These requests can be made through a dedicated application with a natural interface.

[0693] When the terminal receives user input, that information is immediately sent to the server. The server then processes the received data. A particularly noteworthy aspect here is the use of an emotion engine to recognize the user's emotional state. The emotion engine analyzes emotional nuances from the input language data to identify the user's current emotions. This allows for determinations such as whether the user is stressed or relaxed.

[0694] The server then uses natural language processing technology to analyze the question and retrieve relevant household information from its database. The search results include appropriate instructions and explanatory videos. The retrieved information is then used to generate a response that is tailored to the user's emotional state. For example, if the user is confused, a more detailed explanation is provided; if they are calm, only concise information is provided.

[0695] The generated response is sent from the server to the terminal and provided to the user. The terminal updates its interface based on this information, and it is designed to be visually user-friendly. In addition, a push notification function is included, allowing the user to be prompted to do household chores at the appropriate time. The content and timing of the notifications are also adjusted according to the user's emotional state, so appropriate support is provided without causing the user excessive stress.

[0696] For example, suppose a user asks, "How do I clean?" and the server's emotion engine recognizes that the user is tired. In that case, the system selects a simple and effective cleaning method, generates a response that includes a link to a relaxing audio guide, and presents it to the user. In this way, it enables household support that takes the user's emotions into consideration.

[0697] The following describes the processing flow.

[0698] Step 1:

[0699] The user opens the application on their device and enters questions or requests related to household chores. After completing the input, they press the submit button to send the information to the server.

[0700] Step 2:

[0701] The terminal formats the user's input data and sends an HTTP request to the server. During this process, the data is converted to JSON format.

[0702] Step 3:

[0703] The server receives data sent from the terminal and uses an emotion engine to analyze the user's emotional state from their input. For example, it detects emotions from contexts such as "tired" or "stressed."

[0704] Step 4:

[0705] The server uses natural language processing technology to analyze input questions and requests and extract relevant keywords. This allows the type of information the user is specifically seeking to be identified.

[0706] Step 5:

[0707] Based on the analysis results, the server searches the database for household-related information. Appropriate resources such as instruction manuals and explanatory videos are selected.

[0708] Step 6:

[0709] The server considers the emotional state provided by the emotion engine and generates a response based on the retrieved information. For example, if the user is feeling stressed, the response will include encouraging messages and information that can help with relaxation.

[0710] Step 7:

[0711] The server sends the generated response to the terminal in JSON format.

[0712] Step 8:

[0713] The terminal receives a response from the server and displays it on the user interface. The user then reviews the information provided and uses it to help with household chores.

[0714] Step 9:

[0715] Users can schedule appropriate push notifications through their devices. This information is then sent back to the server.

[0716] Step 10:

[0717] The server is configured to send push notifications at specific times based on the user's notification settings. The timing and content of notifications are optimized according to the user's emotional state.

[0718] (Example 2)

[0719] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0720] In recent years, there has been a growing demand for efficient performance of various tasks in daily life. However, conventional systems that provide support while considering the emotional state of the user are not yet fully established. When a user is stressed or in a particular emotional state, the information provided may not always be appropriate. Therefore, there is a need for a system that can facilitate daily life tasks by providing information tailored to the user's emotions.

[0721] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0722] In this invention, the server includes an input device means for receiving inquiries and requests from users, an emotion analysis device means for performing natural language processing to recognize the user's emotions, and a response adjustment device means for adjusting information according to the user's emotional state. This makes it possible to provide life support optimized according to the user's emotional state.

[0723] An "input device" is a device used to receive inquiries and requests from users.

[0724] An "analysis device" is a device used to analyze inquiries and requests received.

[0725] A "search device" is a device used to find information related to daily living activities based on the results of analysis.

[0726] A "response generation device" is a device that presents the searched information to the user.

[0727] A "notification device" is a device that sends out notifications to encourage the performance of daily living activities.

[0728] An "emotion analysis device" is a device that performs natural language processing to recognize the emotions of its users.

[0729] A "response adjustment device" is a device that adjusts information according to the user's emotional state.

[0730] This invention embodies a system that recognizes the user's emotional state and provides support for daily living activities accordingly. This system allows users to send questions and requests related to daily living activities using a smartphone or personal computer (terminal). By using a dedicated application, users can operate the system through an intuitive and natural interface.

[0731] The terminal receives input from the user and sends that data to the server. The server analyzes the data using natural language processing technology. This analysis utilizes an emotion analysis device to extract emotional nuances from the user's input data. Emotion analysis is a crucial process for recognizing what emotional state the user is in, such as stress or relaxation.

[0732] Based on the analysis results, the server searches its internal database for relevant life skills information. This database contains procedures, explanatory videos, and other information that addresses the user's questions. After the information is retrieved, the server generates a response appropriate to the user's emotional state. For example, if the user is confused, it provides a detailed guide; if they are calm, it presents concise information.

[0733] Ultimately, the server sends the generated response to the terminal and presents it to the user. The terminal updates its interface based on the received information, displaying it clearly. The terminal also has a push notification function, which can prompt the user to perform daily tasks at the appropriate time. The content and timing of notifications are also adjusted according to the user's emotional state, so support is provided without causing excessive stress.

[0734] For example, if a user asks, "How do I clean?", the server, through its emotion engine, recognizes that the user is tired. As a result, the system suggests a simple and effective way to clean, and generates a response that also includes a link to a relaxing audio guide.

[0735] An example of a prompt message might be: "The user wants to know how to clean, but appears tired. How should you respond?"

[0736] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0737] Step 1:

[0738] Users open a dedicated application via their smartphone or personal computer and input questions and requests related to daily life tasks. The input data is in text or voice format and uses a natural interface. The terminal immediately transmits this input data to the server. The input data includes the user's questions and voice data. The output is the transmitted user data.

[0739] Step 2:

[0740] The server passes the data received from the terminal to the emotion analysis device. The emotion analysis device uses natural language processing technology to analyze the text or audio data and recognize the user's emotions. The analysis evaluates the frequency of positive or negative words and the tone of the text to determine the degree of stress or relaxation. The input is the user's text or audio data, and the output is the result of the determination of the user's emotional state.

[0741] Step 3:

[0742] The server analyzes the user's inquiry based on their emotional state, which is obtained through sentiment analysis. It utilizes natural language processing techniques to extract key keywords from the questions and requests. This information is then used to search the server's database for information related to daily tasks. The input is the user's inquiry and emotional state, and the output is a list of related information.

[0743] Step 4:

[0744] The server uses a response generator based on the search results to create a response tailored to the user. It adjusts the information to match the user's emotional state and generates a message that includes detailed instructions or concise guides. It may also include relevant images, video links, or audio guides. The input is the searched information and emotional state, and the output is the generated response message.

[0745] Step 5:

[0746] The server sends the generated response to the terminal. The terminal receives this information and updates the interface in a user-friendly format. In addition to visual displays, push notifications deliver timely messages to encourage daily tasks. The content and timing of notifications are also adjusted based on the user's emotional state. The input is the response message sent from the server, and the output is the updated interface and the sent notifications.

[0747] (Application Example 2)

[0748] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0749] It is important for modern consumers to be able to easily obtain product information and have a comfortable shopping experience. However, when customers ask questions about products in stores, they may experience stress if the information provided does not take into account their individual emotional state. This invention aims to enable the provision of optimal product information based on an analysis of the customer's emotional state.

[0750] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0751] In this invention, the server includes an input means for receiving questions and requests from the user, an emotion analysis means for analyzing the received questions and requests and recognizing the user's emotional state, a search means for searching for product-related information based on the analysis results, and a response generation means for providing the searched information in an optimal form according to the user's emotional state. This makes it possible to provide product information that matches the emotional state of each individual customer and realize a comfortable purchasing experience.

[0752] An "input method" is an interface for receiving questions and requests from users.

[0753] "Emotional analysis tools" are functions that analyze received questions and requests to identify the user's emotional state.

[0754] A "search tool" is a function that extracts and provides relevant information based on the analyzed results.

[0755] A "response generation means" is a process for providing the user with extracted information in the most optimal form, according to the user's emotional state.

[0756] "Notification methods" refer to a function that sends necessary information and guidance as push notifications, tailored to the user's emotional state.

[0757] This embodiment includes a server, a user terminal, and a network for exchanging data between them. The user first uses the terminal to input questions or requests for product information or assistance with household chores. This data is then sent to the server via the input means. The server uses sentiment analysis means to identify the user's emotional state from this data. Natural language processing tools such as TextBlob are used in this process.

[0758] Based on the analysis results, the server uses search tools to find relevant information. This information is extracted from existing resources stored in the database. The response generation tool optimizes the information according to the user's emotional state and constructs specific content to be provided to the user. The information provided at this time takes the form of a detailed explanation or a concise summary, depending on the user's emotions.

[0759] For example, if a user enters "I want to know more about this product, but I'm in a hurry" into a terminal in a store, the server detects that the user is experiencing stress. As a result, the response generation mechanism quickly presents concise information summarizing the product's main features. The notification mechanism sends this information to the terminal as a push notification, timely and efficiently delivered to the user.

[0760] In this way, the system can provide information tailored to each customer's emotional state and appropriately meet their needs.

[0761] An example of a prompt message for a generative AI model might be: "Calculate emotional polarity from customer input, determine the emotional state, and dynamically generate information based on the result. For example, if the polarity is -0.3, determine that the customer is stressed and provide detailed guidance."

[0762] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0763] Step 1:

[0764] The user enters questions or requests about the product via a terminal. This data is then sent to the server. This step includes the process up to the point where the user's input is sent to the server as text data.

[0765] Step 2:

[0766] The server passes the received text data to a sentiment analysis tool. Here, the text data is analyzed using a natural language processing library (e.g., TextBlob), and its polarity value is calculated. Based on this polarity value, the user's emotional state is classified as "relaxed," "stressed," or "neutral."

[0767] Step 3:

[0768] The server uses search methods to retrieve relevant information from the database based on the emotional state. If the emotional state is stress, only concise and essential product information is extracted.

[0769] Step 4:

[0770] The server's response generation mechanism constructs content to provide the user with information obtained in the most optimal format, depending on their emotional state. Specifically, if the user is experiencing stress, concise content emphasizing the most important points is generated.

[0771] Step 5:

[0772] Content generated by the notification system is sent to the device as a push notification. The user receives the notification and can obtain the information appropriately. This ensures that the information the user needs is provided in a timely manner, guaranteeing a smooth user experience.

[0773] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0774] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0775] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0776] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0777] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0778] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0779] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0780] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0781] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0782] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0783] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0784] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0785] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0786] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0787] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0788] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0789] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0790] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0791] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0792] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0793] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0794] The following is further disclosed regarding the embodiments described above.

[0795] (Claim 1)

[0796] An input method for receiving questions and requests from users,

[0797] An analysis tool for analyzing received questions and requests,

[0798] A search method for searching for household chore-related information based on the analysis results,

[0799] A response generation means that provides the searched information to the user,

[0800] A notification method that sends push notifications to encourage household chores to be performed,

[0801] A system that includes this.

[0802] (Claim 2)

[0803] The system according to claim 1, wherein the analysis means extracts important keywords from questions or requests using natural language processing technology.

[0804] (Claim 3)

[0805] The system according to claim 1, wherein the response generating means generates a message that includes a resource link providing a video or a procedure manual.

[0806] "Example 1"

[0807] (Claim 1)

[0808] A terminal device for receiving information input from users,

[0809] A data processing means for analyzing the received information,

[0810] An information extraction means for searching for lifestyle management-related information based on the analysis results,

[0811] A means for displaying search results and providing the searched information to the user,

[0812] A means of communication that sends notifications to encourage the performance of daily living tasks,

[0813] A system that includes this.

[0814] (Claim 2)

[0815] The system according to claim 1, wherein the data processing means uses generative model technology to identify attributes from information.

[0816] (Claim 3)

[0817] The system according to claim 1, wherein the result display means generates a message that includes an information link providing visual information or a document.

[0818] "Application Example 1"

[0819] (Claim 1)

[0820] The means of accepting challenges and services from users,

[0821] An analytical device and means for analyzing the tasks and services received,

[0822] A detection device means for searching for information related to household tasks by comparing it with the analysis results,

[0823] A response generation device means that provides the explored information to the user,

[0824] A reporting device means for sending notifications to encourage the performance of household chores,

[0825] A facility-based work support device for managing work plans,

[0826] A system that includes this.

[0827] (Claim 2)

[0828] The apparatus according to claim 1, wherein the analysis device extracts important concepts from problems and services using natural language processing technology.

[0829] (Claim 3)

[0830] The system according to claim 1, wherein the response generation device creates communication that includes links for supplying viewing resources and instruction manuals.

[0831] "Example 2 of combining an emotion engine"

[0832] (Claim 1)

[0833] An input device for receiving inquiries and requests from users,

[0834] An analytical device means for analyzing inquiries and requests received,

[0835] A search device means for finding information related to daily life activities based on the analyzed results,

[0836] A response generation device means that presents the searched information to the user,

[0837] A notification device means that sends out a notification to encourage the performance of daily living activities,

[0838] An emotion analysis device means that performs natural language processing to recognize the emotions of the user,

[0839] A response adjustment device means that adjusts information according to the user's emotional state,

[0840] A system that includes this.

[0841] (Claim 2)

[0842] The system according to claim 1, wherein the analysis device extracts key words from inquiries and requests using natural language processing technology.

[0843] (Claim 3)

[0844] The system according to claim 1, wherein the response generating device generates communication including a link to a document providing images or instructions.

[0845] "Application example 2 when combining with an emotional engine"

[0846] (Claim 1)

[0847] An input method for receiving questions and requests from users,

[0848] A means of sentiment analysis to analyze received questions and requests and recognize the user's emotional state,

[0849] A search method for searching for household chore-related information based on the analysis results,

[0850] A response generation means that provides the searched information in an optimal form according to the user's emotional state,

[0851] A notification method that sends push notifications tailored to the user's emotional state in order to prompt the user to perform household chores at the appropriate time,

[0852] A system that includes this.

[0853] (Claim 2)

[0854] The system according to claim 1, wherein the analysis means extracts important keywords from questions or requests using natural language processing technology and identifies emotional states.

[0855] (Claim 3)

[0856] The system according to claim 1, wherein the response generating means generates a message that includes a resource link providing a video or instruction manual with a level of detail appropriate to the emotional state. [Explanation of Symbols]

[0857] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. An input method for receiving questions and requests from users, An analysis tool for analyzing received questions and requests, A search method for searching for household chore-related information based on the analysis results, A response generation means that provides the searched information to the user, A notification method that sends push notifications to encourage household chores to be performed, A system that includes this.

2. The system according to claim 1, wherein the analysis means extracts important keywords from questions or requests using natural language processing technology.

3. The system according to claim 1, wherein the response generating means generates a message that includes a resource link providing a video or a procedure manual.

Images