system

Abstract

We provide the system. [Solution] A means of collecting information to analyze the degree of cleanliness in each room and the behavioral patterns of the residents, A means for generating an optimal cleaning plan based on the aforementioned information, A means for transmitting the aforementioned plan to an information terminal and controlling the cleaning machine, A means including a cleaning machine that detects dirt in a room in real time and performs cleaning, A means to receive feedback after cleaning is complete and to improve the artificial intelligence model, A means including an application for an information terminal to notify the user of the optimal cleaning timing, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern ordinary households, especially in dual-income households and busy households, problems include limited cleaning time, poor cleaning efficiency and being laborious, and not knowing the appropriate cleaning methods for each room and location. As a result, it is difficult to maintain the cleanliness within the home, and there is a risk of deterioration in the quality of the living environment.

Means for Solving the Problems

[0005] This invention enhances cleaning efficiency by providing a system that collects data to analyze the degree of dirtiness in each room and the behavioral patterns of residents, and generates an optimal cleaning schedule. Specifically, the server generates a cleaning schedule based on the collected data and sends it to a terminal to control a cleaning robot. This cleaning robot detects dirt in the room in real time and performs focused cleaning. Furthermore, it receives feedback after cleaning is completed and continuously improves the AI ​​model to achieve optimal cleaning at all times. In addition, users can send operations and feedback to the system via a smartphone app or voice assistant, enabling flexible system operation that meets the needs of residents.

[0006] "Data" refers to a collection of information gathered to analyze things like the cleanliness of a room and the behavioral patterns of its inhabitants.

[0007] "Analysis" is the process of using collected data to clarify the nature and structure of a subject and deepen our understanding of it.

[0008] A "cleaning schedule" is a plan outlining the time and route for cleaning, based on the condition of the room and the residents' daily routines.

[0009] A "terminal" refers to equipment or devices that perform actual cleaning based on data and instructions received from a server, and includes cleaning robots and smart speakers.

[0010] A "cleaning robot" is an autonomous robotic device that moves automatically around the home, uses sensors to detect dirt, and cleans accordingly.

[0011] "Feedback" refers to information and opinions about the cleaning results and equipment performance that the system receives after cleaning is complete.

[0012] An "AI model" is a computational model of an artificial intelligence system designed to suggest the optimal cleaning method and schedule based on data collection and analysis.

[0013] A "smartphone app" is software that users use from their smart devices to operate a cleaning system and send feedback.

[0014] A "voice assistant" is software or a digital device that receives user instructions through voice recognition and assists in operating cleaning systems and providing information. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12]It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention provides a smart system that utilizes AI technology to streamline household cleaning. Specifically, it constructs a system that analyzes the level of dirtiness in each room and the residents' daily routines, and then proposes and executes an optimal cleaning schedule.

[0037] Server Functions

[0038] The server collects data from multiple devices regarding the degree of dirtiness in each room, furniture arrangement, and residents' behavior patterns. Data collection is performed through cleaning robots, smartphone apps, and voice assistants. The server uses this data to analyze it and build an AI model. Based on the AI ​​model, it generates an optimal cleaning schedule and sends it to each device.

[0039] Device functions

[0040] The system includes a robotic vacuum cleaner and a smart speaker. The robotic vacuum cleaner automatically starts cleaning according to a schedule received from the server. As the robot moves around the room, its built-in sensors detect dust and debris in real time, and it cleans intensively along the optimal route.

[0041] The smart speaker sends voice commands from the user to a server to check if any corrections are needed to the AI ​​model. The device also feeds back the status and results to the server after cleaning is complete to help adjust future schedules.

[0042] User actions

[0043] Users can access the system through a smartphone app and input individual requirements such as room layout and cleaning frequency. Users can monitor the cleaning progress through the app and adjust schedules and settings as needed. Using a voice assistant makes it even easier for users to issue cleaning commands and check results.

[0044] Specific example

[0045] For example, a dual-income household might want to set a schedule to finish cleaning the floors by 6 p.m. on weekdays. Based on past data, the server creates a schedule to run the cleaning robot while the residents are at the office, identifying the times when cleaning is needed. Once the user sets room priorities in the app, the robot starts cleaning from the living room and efficiently covers the entire house. After completion, the robot reports its cleaning status to the server, which is then used to improve future schedules.

[0046] Thus, the present invention makes it possible to automate and streamline household cleaning, thereby improving the quality of the living environment.

[0047] The following describes the processing flow.

[0048] Step 1:

[0049] The server collects data from terminals regarding the level of cleanliness in each room, furniture arrangement, and residents' behavior patterns. This includes sensor data and user input data.

[0050] Step 2:

[0051] The server uses an AI model to analyze the collected data and understand the level of dirtiness in each room and the residents' behavior patterns. This provides the basic information needed to determine cleaning priorities and timing.

[0052] Step 3:

[0053] The server generates an optimal cleaning schedule based on the analysis results. This schedule includes the time slots and specific routes for cleaning.

[0054] Step 4:

[0055] The server sends the generated cleaning schedule to the terminal and provides specific instructions to the cleaning robot.

[0056] Step 5:

[0057] The cleaning robot included in the device starts cleaning at the designated time according to the schedule received from the server. The robot uses its built-in sensors to detect dust and debris in real time, moving efficiently around the room and cleaning.

[0058] Step 6:

[0059] Users can monitor the cleaning progress through a smartphone app or voice assistant. They can also adjust cleaning schedules and settings via the app as needed.

[0060] Step 7:

[0061] After cleaning is complete, the device sends the cleaning results to the server. This information is used to improve the AI ​​model for the next cleaning. Based on the feedback, the server continues to optimize the schedule and cleaning procedure.

[0062] (Example 1)

[0063] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0064] In modern homes, daily cleaning tasks are often not performed efficiently amidst busy lifestyles, creating a need for improved living environments. Furthermore, conventional cleaning equipment and services struggle to provide flexible schedules based on the degree of dirtiness in rooms and the residents' daily routines, highlighting the need for automated systems.

[0065] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0066] In this invention, the server includes means for collecting dimensional data from information terminals and analyzing the state of each spatial domain and human behavior patterns; means for constructing a generative AI model that optimizes the operation schedule based on the data; and means for transmitting the optimized schedule to a control device and controlling the work equipment. This makes it possible to perform cleaning activities automatically and efficiently, improving the quality of the living environment.

[0067] An "information terminal" is a device that has the function of collecting data and transmitting it to a server, and includes smartphones, tablets, and personal computers.

[0068] "Dimensional data" refers to information collected to represent the state and environment of a spatial domain, as well as human activity, and is data that can capture physical arrangements and temporal changes.

[0069] "Spatial area" refers to the physical area that is the subject of cleaning, and includes the interior space of rooms and buildings.

[0070] "Human behavior patterns" refer to the tendencies of activities that residents engage in on a daily basis, and understanding these patterns can help optimize schedules.

[0071] "Operation schedule" refers to a plan that includes the date, time, and order in which cleaning tasks will be performed, and is optimized by a generative AI model.

[0072] A "generative AI model" is an algorithm that uses machine learning techniques to analyze data and create an optimal cleaning schedule.

[0073] A "control device" is an electronic device that operates work equipment based on instructions received from a server, and can operate multiple devices, including cleaning robots.

[0074] "Work equipment" refers to automated devices used to perform cleaning, and includes cleaning robots and other cleaning equipment.

[0075] This invention provides a system for automating and streamlining household cleaning. Specifically, it is achieved through a series of processes in which a server, terminals, and users work together.

[0076] Server operation:

[0077] The server collects dimensional data from information terminals. The data collected relates to the state of a spatial domain and human behavior patterns. This includes information such as the cleanliness of a room, the arrangement of furniture, and the daily routines of the inhabitants. Based on this data, the server preprocesses the data using Python's Pandas and NumPy, and builds a generative AI model using Scikit-learn and TENSORFLOW®. This model is capable of generating an optimal schedule of activities.

[0078] Device operation:

[0079] The system includes a cleaning robot and a smart speaker. The cleaning robot automatically begins cleaning according to the schedule sent from the server. Equipped with LIDAR sensors and cameras, the robot detects particles in the surrounding space in real time, enabling highly accurate cleaning. This allows for efficient and waste-free cleaning.

[0080] User actions:

[0081] Users can check the current cleaning schedule using a smartphone app or voice control device. They can also set cleaning conditions and priorities as needed through prompts. For example, they can give instructions such as, "Please set a new schedule that increases the frequency of cleaning the living room, taking past feedback into consideration, and does not clean at night."

[0082] This allows daily cleaning tasks to be performed automatically in accordance with the residents' daily routines, ensuring a consistently comfortable living environment and reducing the burden on users.

[0083] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0084] Step 1:

[0085] The server collects dimensional data through information terminals. It receives data such as dirt and furniture placement information from sensors in the spatial domain, as well as resident behavior patterns, as input. This data is stored in storage and prepared for later analysis.

[0086] Step 2:

[0087] The server preprocesses and analyzes the collected data. The input is the raw data obtained in step 1, which is cleaned up using Python's Pandas and NumPy to prepare it for analysis. The output is the analyzed dataset, which is used to optimize the operation schedule.

[0088] Step 3:

[0089] The server uses the analyzed data to build a generative AI model and generate an optimal operating schedule. The input is the dataset obtained in the previous step, and the model is trained using Scikit-learn or TensorFlow. The output is an optimized cleaning schedule, which determines the operating time and order of the cleaning robots.

[0090] Step 4:

[0091] The server sends the generated operation schedule to the terminal. The terminal, specifically the cleaning robot, receives this schedule and prepares to execute it. The output is instruction data for the cleaning robot, which then starts the automated cleaning process.

[0092] Step 5:

[0093] The terminal performs the actual cleaning based on the received schedule. The input is an operation instruction from the server, and the cleaning robot uses this to scan the site environment using LIDAR sensors and cameras and then performs the cleaning. The output is cleaning completion status data, which is fed back to the server.

[0094] Step 6:

[0095] The server receives feedback from the terminal indicating that cleaning is complete, and updates the AI ​​model. The input is feedback information from the terminal, which is saved as new data and used to retrain the model. This improves the model, which is then used to generate more efficient schedules for future cleanings.

[0096] Step 7:

[0097] The user uses a smartphone app to check the schedule and settings and make adjustments as needed. For example, they might use a prompt message such as, "Please increase the cleaning frequency in the living room and disable nighttime cleaning." The output is a new schedule and settings tailored to the user's needs.

[0098] (Application Example 1)

[0099] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0100] Household cleaning is time-consuming and often a burden for residents with busy lifestyles. Furthermore, creating an efficient cleaning schedule requires considering residents' lifestyles and the condition of their rooms, which is difficult to manage manually. Therefore, there is a need for a system that automatically performs optimal cleaning based on the level of dirt and the resident's lifestyle.

[0101] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0102] In this invention, the server includes means for collecting information to analyze the degree of dirtiness in each room and the behavioral patterns of the residents, means for generating an optimal cleaning plan based on the information, and means including an information terminal application for notifying the user of the optimal cleaning timing. As a result, residents will have an optimal cleaning schedule suggested and executed according to their lifestyle, enabling them to live an efficient and stress-free life.

[0103] "Room-specific cleanliness" is an indicator of the level of cleaning required for each room, and is determined by measuring the amount of dust and debris.

[0104] "Resident behavior patterns" refer to the tendencies of residents' movement and activities in their daily lives, and this allows us to understand which rooms are used at which times of the day.

[0105] An "information terminal" refers to a portable computing device such as a smartphone or tablet that sends and receives data via the internet or applications.

[0106] A "cleaning machine" is a robotic device that automatically cleans floors, using internal sensors to detect dirt and obstacles and performing cleaning efficiently.

[0107] An "artificial intelligence model" is an algorithm that performs pattern recognition and prediction based on a large amount of data, with the aim of optimizing cleaning plans and improving them through feedback.

[0108] "Cleaning timing" refers to the optimal time to begin cleaning, and is determined by considering the resident's schedule and the room's usage.

[0109] An "information terminal application" is software that runs on an information terminal and provides cleaning management and notification functions.

[0110] To realize this invention, the server collects and analyzes data on the degree of dirtiness in each room and the behavioral patterns of the residents. Smartphones and tablets are used as information terminals. Specifically, these information terminals transmit the status of each room to the server via sensors. Based on this, the server uses AI technology such as TensorFlow to generate an optimal cleaning plan. The generated plan is notified to the user through an application on the information terminal, allowing residents to adjust the cleaning timing according to their own schedule and lifestyle.

[0111] The cleaning machine, or automated cleaning robot, receives instructions from a server and cleans along an efficient route while detecting dirt in real time. Once cleaning is complete, feedback is sent back to the server, and the artificial intelligence model is improved. This optimizes the cleaning schedule for future cleaning sessions.

[0112] For example, if a user wants their living room to be cleaned intensively on weekend afternoons, they can specify this through the application. They can also input prompts for the generating AI model, such as, "Based on resident A's lifestyle, please suggest the optimal cleaning schedule for this week."

[0113] Thus, the system provided by the invention can automate and optimize cleaning according to the user's lifestyle, thereby improving the quality of the living environment.

[0114] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0115] Step 1:

[0116] Users perform initial setup via a terminal, entering cleaning requirements and room priorities. This input includes resident schedules and acceptable levels of dirtiness for each room. This data is sent to the server and registered as the initial dataset.

[0117] Step 2:

[0118] Based on the initial setup data, the server collects data to understand the level of dirtiness in each room and the residents' lifestyle patterns. It collects sensor information from cleaning machines and location information from smartphones and stores it in a database. The data collected in this process will be used later for the analysis of AI models.

[0119] Step 3:

[0120] The server analyzes the collected data using AI software such as TensorFlow and generates an optimal cleaning plan using a generative AI model. Input data includes the room's level of dirtiness and usage frequency, while output is specific cleaning timings and routes. This plan is then fed back to the user.

[0121] Step 4:

[0122] The terminal receives the cleaning plan sent from the server and notifies the user of the cleaning timing. The terminal's application controls the cleaning machine and starts cleaning according to the plan. At this step, the user can check the progress of the cleaning in real time via notifications.

[0123] Step 5:

[0124] Once cleaning is complete, the cleaning machine sends feedback on its performance and the degree of soiling to a server. The server uses this feedback to improve its artificial intelligence model, thereby increasing the accuracy of the next cleaning plan. This allows for the automatic suggestion of a more optimal plan, even with the same information.

[0125] This process allows users to achieve efficient cleaning without hassle and maintain a comfortable living environment.

[0126] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0127] This invention is a smart system for streamlining household cleaning tasks, and in particular, it combines an emotion engine that recognizes user emotions to provide a more personalized cleaning experience. Specifically, this system collects data based on the degree of dirtiness in a room and the behavioral patterns of the inhabitants to provide an optimal cleaning schedule, and also has the function of detecting the user's emotions and reflecting them in the cleaning process.

[0128] Server Functions

[0129] The server collects and analyzes data from terminals, and uses an AI model to understand the level of dirtiness in each room and the residents' behavior patterns. Based on the results of this analysis, it generates an optimal cleaning schedule and sends it to the various terminals. Furthermore, it analyzes the user's voice input through an emotion engine and adjusts the schedule and cleaning methods based on that emotion.

[0130] Device functions

[0131] The devices include a robotic vacuum cleaner and a smart speaker. Through an emotion engine, the devices analyze the user's voice to understand their emotions and transmit this information to a server. The robotic vacuum cleaner cleans according to the received schedule, detecting the level of dirt in the room in real time and cleaning along an optimized route. It also sends feedback to the server to help improve future schedules.

[0132] User actions

[0133] Users can access the system through a smartphone app or voice assistant. Users can adjust the cleaning pace and timing based on their emotional state; for example, they can instruct the system to increase cleaning frequency during stressful periods. The system adapts the cleaning plan based on this input, providing a more satisfying cleaning experience.

[0134] Specific example

[0135] For example, consider a dual-income household where the user wants to clean efficiently when they come home tired. When the user expresses their emotions through voice input such as "I'm tired," the emotion engine recognizes this, and the server adjusts the cleaning robot's operation to reduce stress. For example, it might clean in silent mode, providing an environment that doesn't cause stress to the user. After completion, the cleaning robot sends feedback to the server, which is then used to improve future situations.

[0136] Thus, this invention makes it possible to achieve personalized cleaning that is attentive to the user's emotions and improve the living environment within the home.

[0137] The following describes the processing flow.

[0138] Step 1:

[0139] Users initiate interaction with the system by expressing emotions through a smartphone app or voice input. This can be done simply by reporting their physical condition or mood through voice.

[0140] Step 2:

[0141] The emotion engine built into the device analyzes the user's voice input and identifies emotions from that voice. For example, it detects voice tone and specific keywords to extract emotions such as "tired" or "want to relax."

[0142] Step 3:

[0143] The device sends the analysis results to the server and requests that it generate cleaning requirements that have been adjusted based on the user's current sentiment data.

[0144] Step 4:

[0145] The server generates a new cleaning schedule based on received emotional data, the current level of dirtiness in the room, and the residents' behavior patterns. For example, if the user is tired, it may increase the cleaning frequency or recommend cleaning in silent mode.

[0146] Step 5:

[0147] The server sends the generated cleaning schedule and any emotionally-based adjustments to the device's cleaning robot.

[0148] Step 6:

[0149] The cleaning robot starts cleaning in the specified mode according to a schedule received from the server. In practice, the robot moves around the room, updating information about the room in real time using sensors to clean efficiently.

[0150] Step 7:

[0151] After cleaning is complete, the device sends the cleaning results and user feedback back to the server. This allows the server to use the information to improve the AI ​​model.

[0152] Step 8:

[0153] The user receives a cleaning completion notification via an app or similar means, confirming that the cleaning was performed appropriately according to their emotional needs. The system evaluates whether it performed as expected and makes further adjustments as needed.

[0154] Through these steps, the system provides a flexible and personalized cleaning process that takes user emotions into consideration.

[0155] (Example 2)

[0156] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0157] In modern homes, cleaning is an important and time-consuming task, but conventional automated cleaning systems often lack efficiency because they fail to adequately consider the lifestyles and emotional states of residents. Furthermore, the operation of cleaning robots can sometimes increase residents' stress levels. Therefore, there is a need for cleaning systems that are adapted to the behavior and emotions of residents.

[0158] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0159] In this invention, the server includes means for collecting information to analyze the pollution status of each environment and the behavioral characteristics of the residents, means for creating an optimal cleaning schedule based on the information, and means for analyzing emotions from the user's voice instructions and adjusting the cleaning process and schedule accordingly. This makes it possible to provide residents with an individually optimized cleaning experience and improve the quality of the living environment.

[0160] "Environment" refers to the individual rooms or areas that are to be cleaned, and is a target area related to its level of contamination and the behavioral characteristics of the residents.

[0161] "Contamination level" refers to the degree of accumulation of impurities such as dust and debris within the environment being cleaned.

[0162] "Resident behavioral characteristics" refer to the activity patterns and lifestyle habits of residents within a specific environment, and are used to optimize cleaning schedules.

[0163] "Means of collecting information" refers to methods of acquiring data on environmental pollution levels and the behavioral characteristics of residents using sensors and cameras.

[0164] A "cleaning schedule" refers to the operating schedule of cleaning equipment created based on acquired information, and represents a plan for effectively cleaning the environment.

[0165] "Methods for analyzing emotions from voice commands" refers to technologies that use algorithms and models to analyze a user's voice input and estimate the user's emotions and stress levels.

[0166] "Means of adjusting the cleaning process and schedule" refers to methods for appropriately changing the operating modes and schedules of cleaning equipment based on analyzed emotional data.

[0167] This invention is a system for efficiently performing cleaning tasks in a residential environment, combining information technology and emotion analysis technology to provide a more personalized cleaning experience.

[0168] The server functions as a device that receives and analyzes data collected from connected devices in real time. This data includes the level of contamination in rooms obtained from sensors and the behavioral characteristics of residents captured by cameras. A generative AI model is used for analysis, automatically generating an optimal cleaning schedule from each data point. In addition, the server utilizes a voice analysis engine to analyze emotions from the user's voice obtained through the device, and has the ability to adjust the cleaning process and schedule as needed.

[0169] The system includes a cleaning robot and a smart speaker for receiving user voice commands. The cleaning robot cleans rooms efficiently according to a cleaning schedule received from the server. During this process, the robot detects environmental contamination in real time and sends this data back to the server as feedback. This data is then used to optimize the next cleaning plan.

[0170] Users can access the system through a smartphone application or voice assistant and easily give cleaning instructions tailored to their emotional state. In particular, when stressed, entering prompts such as "I'm tired" into a smart speaker will instruct the cleaning robot to operate in quiet mode, thus reducing stress for residents.

[0171] This invention overcomes the limitations of conventional automatic cleaning devices, enabling customization based on lifestyle habits and individual emotional states. This significantly improves the quality of the living environment.

[0172] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0173] Step 1:

[0174] The terminal collects data on environmental pollution levels and resident behavioral characteristics via sensors and cameras installed in the room. The collected data includes real-time information such as temperature, humidity, and camera footage. This input data is used to assess pollution levels and analyze resident behavior, and is then transmitted to a server.

[0175] Step 2:

[0176] The server inputs environmental data received from the terminal into a generating AI model, which then analyzes the data. The AI ​​model utilizes machine learning algorithms to evaluate the degree of contamination in each room and the behavioral patterns of the residents. This analysis identifies the time and methods required for cleaning and generates an optimal cleaning schedule.

[0177] Step 3:

[0178] The server analyzes the emotional information entered by the user via voice through a voice analysis engine. The voice data is used to infer the user's emotional state. The emotional analysis results obtained from the prompt text are compared with the current cleaning plan, and output is generated to readjust the cleaning schedule and cleaning methods as needed.

[0179] Step 4:

[0180] Upon receiving instructions from the server, the cleaning robot at the terminal begins operation according to the predetermined cleaning schedule. The robot uses sensors to detect contamination in the room in real time and cleans along an optimized route. It monitors the progress of cleaning and the condition of the room, and dynamically adjusts the route and cleaning mode.

[0181] Step 5:

[0182] After cleaning is complete, the terminal reports completion to the server, along with feedback on the cleaning results and the latest environmental data. The server receives this feedback and processes the data to improve the accuracy of the next cleaning plan. It also stores the data as training data for the generated AI model, enabling continuous performance improvement.

[0183] This series of processes makes it possible to provide residents with an efficient and comfortable cleaning experience.

[0184] (Application Example 2)

[0185] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0186] In recent years, household cleaning machines have been required to be highly efficient and comfortable to use, but conventional technology has struggled to flexibly adapt to the user's lifestyle and emotional state. In particular, a cleaning system that can appropriately respond to specific emotional states, such as when the user is fatigued or stressed, has yet to be established. Therefore, there is a need to develop a new cleaning system that can recognize emotions and appropriately adjust the cleaning process according to the user's situation.

[0187] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0188] In this invention, the server includes means for collecting data to analyze the degree of dirtiness in each room and the behavioral patterns of the residents, means for generating an optimal cleaning schedule, and means for detecting the emotional state of the residents and adjusting the operation of the cleaning machine based on that. This makes it possible to optimize the operation of the cleaning machine according to the emotional state of the user and provide a comfortable living environment.

[0189] "Room-specific cleanliness" refers to the condition or level of cleanliness in each individual room, and is measured based on the frequency and type of dirt that occurs.

[0190] "Resident behavior patterns" refer to patterns of behavior and actions of people living within a residence, and include information about their daily activities and movement tendencies.

[0191] An "optimal cleaning schedule" is a set of times and sequences planned to perform cleaning tasks with maximum efficiency, and is formulated based on the degree of soiling and the user's daily schedule.

[0192] "Terminal" is a general term for electronic devices related to this system, and includes cleaning machines and devices that handle information display and communication.

[0193] A "cleaning machine" is a mechanical device designed for cleaning indoor spaces, with the purpose of removing dirt and purifying the environment.

[0194] "Resident's emotional state" refers to the user's psychological or emotional condition and is determined based on information from their voice and behavior.

[0195] An "artificial intelligence model" is a mathematical or computational model designed to perform a specific task based on data, and possesses the ability to learn from experience.

[0196] The system that realizes this application example is constructed as follows: This system is an advanced home cleaning system that provides a personalized cleaning experience based on the user's emotions. This system consists of a server, terminals (cleaning machines and information display devices), and a user interface.

[0197] The server is the core of the system, responsible for all data analysis and schedule generation. It utilizes data collected from various sensors and input devices to analyze the level of dirtiness in each room and the residents' behavioral patterns. This data is then analyzed by an AI model to generate an optimal cleaning schedule. Furthermore, the server has the ability to detect the user's emotional state from their voice and behavior using an emotion engine, and adjust the cleaning machine's operation accordingly.

[0198] The device includes a vacuum cleaner and a voice assistance device. The vacuum cleaner uses robotic cleaning technology to clean the room and transmits status data to the server in real time. The voice assistance device analyzes the user's voice and automatically transmits their emotional state to the server. This allows the cleaning mode to be adjusted according to the user's emotional state.

[0199] Users can access and operate the system via a mobile device or voice assistance system. Users can set the cleaning pace and timing by directly inputting their emotional state or giving voice instructions. Through this, the server adaptively trains its AI model to improve the accuracy of future cleaning plans.

[0200] For example, if a user gives a voice command saying, "I want it to be quiet today," the server will interpret the voice as indicating the user is seeking silence and operate the vacuum cleaner in silent mode. Furthermore, if the user is feeling stressed, a relaxation mode will be suggested.

[0201] For this system to function, the following prompt can be used: "Design an app that detects the user's emotions and adjusts the operation of the vacuum cleaner accordingly."

[0202] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0203] Step 1:

[0204] The server collects sensor data to measure residents' behavior patterns and the level of dirtiness in each room. The input is environmental data obtained from various sensors, and the output is a dataset organized for necessary analysis. The server integrates this data and prepares it as foundational data for quantifying behavioral patterns.

[0205] Step 2:

[0206] The server uses an emotion engine to analyze the user's voice input and detect their emotional state. The input is the user's voice data, and the output is an emotion label. The voice waveform is analyzed via a speech processing algorithm, and the emotion is classified based on an emotion model. This result is then used in the next step.

[0207] Step 3:

[0208] The server generates an optimal cleaning schedule based on the behavioral pattern data obtained in Step 1 and the emotional state labels obtained in Step 2. The input is the behavioral patterns and emotional labels obtained in the previous step, and the output is a specific cleaning schedule. An AI model is used to analyze this data and create individually customized schedules.

[0209] Step 4:

[0210] The vacuum cleaner in the terminal starts cleaning according to the cleaning schedule sent from the server. The input is the schedule data received from the server, and the output is feedback data after cleaning is complete. The vacuum cleaner proceeds with cleaning while detecting the room conditions with its built-in sensors, and when cleaning is complete, it sends the results to the server.

[0211] Step 5:

[0212] Users operate the system and send feedback via mobile devices or voice assistance systems. Inputs are the user's experiences and requests, while outputs are data used to train the next AI model. User feedback is reflected on the server and used to improve the next schedule generation.

[0213] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0214] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0215] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0216] [Second Embodiment]

[0217] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0218] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0219] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0220] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0221] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0222] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0223] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0224] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0225] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0226] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0227] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0228] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0229] This invention provides a smart system that utilizes AI technology to streamline household cleaning. Specifically, it constructs a system that analyzes the level of dirtiness in each room and the residents' daily routines, and then proposes and executes an optimal cleaning schedule.

[0230] Server Functions

[0231] The server collects data from multiple devices regarding the degree of dirtiness in each room, furniture arrangement, and residents' behavior patterns. Data collection is performed through cleaning robots, smartphone apps, and voice assistants. The server uses this data to analyze it and build an AI model. Based on the AI ​​model, it generates an optimal cleaning schedule and sends it to each device.

[0232] Device functions

[0233] The system includes a robotic vacuum cleaner and a smart speaker. The robotic vacuum cleaner automatically starts cleaning according to a schedule received from the server. As the robot moves around the room, its built-in sensors detect dust and debris in real time, and it cleans intensively along the optimal route.

[0234] The smart speaker sends voice commands from the user to a server to check if any corrections are needed to the AI ​​model. The device also feeds back the status and results to the server after cleaning is complete to help adjust future schedules.

[0235] User actions

[0236] Users can access the system through a smartphone app and input individual requirements such as room layout and cleaning frequency. Users can monitor the cleaning progress through the app and adjust schedules and settings as needed. Using a voice assistant makes it even easier for users to issue cleaning commands and check results.

[0237] Specific example

[0238] For example, a dual-income household might want to set a schedule to finish cleaning the floors by 6 p.m. on weekdays. Based on past data, the server creates a schedule to run the cleaning robot while the residents are at the office, identifying the times when cleaning is needed. Once the user sets room priorities in the app, the robot starts cleaning from the living room and efficiently covers the entire house. After completion, the robot reports its cleaning status to the server, which is then used to improve future schedules.

[0239] Thus, the present invention makes it possible to automate and streamline household cleaning, thereby improving the quality of the living environment.

[0240] The following describes the processing flow.

[0241] Step 1:

[0242] The server collects data from terminals regarding the level of cleanliness in each room, furniture arrangement, and residents' behavior patterns. This includes sensor data and user input data.

[0243] Step 2:

[0244] The server uses an AI model to analyze the collected data and understand the level of dirtiness in each room and the residents' behavior patterns. This provides the basic information needed to determine cleaning priorities and timing.

[0245] Step 3:

[0246] The server generates an optimal cleaning schedule based on the analysis results. This schedule includes the time slots and specific routes for cleaning.

[0247] Step 4:

[0248] The server sends the generated cleaning schedule to the terminal and provides specific instructions to the cleaning robot.

[0249] Step 5:

[0250] The cleaning robot included in the device starts cleaning at the designated time according to the schedule received from the server. The robot uses its built-in sensors to detect dust and debris in real time, moving efficiently around the room and cleaning.

[0251] Step 6:

[0252] Users can monitor the cleaning progress through a smartphone app or voice assistant. They can also adjust cleaning schedules and settings via the app as needed.

[0253] Step 7:

[0254] After cleaning is complete, the device sends the cleaning results to the server. This information is used to improve the AI ​​model for the next cleaning. Based on the feedback, the server continues to optimize the schedule and cleaning procedure.

[0255] (Example 1)

[0256] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0257] In modern homes, daily cleaning tasks are often not performed efficiently amidst busy lifestyles, creating a need for improved living environments. Furthermore, conventional cleaning equipment and services struggle to provide flexible schedules based on the degree of dirtiness in rooms and the residents' daily routines, highlighting the need for automated systems.

[0258] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0259] In this invention, the server includes means for collecting dimensional data from information terminals and analyzing the state of each spatial domain and human behavior patterns; means for constructing a generative AI model that optimizes the operation schedule based on the data; and means for transmitting the optimized schedule to a control device and controlling the work equipment. This makes it possible to perform cleaning activities automatically and efficiently, improving the quality of the living environment.

[0260] An "information terminal" is a device that has the function of collecting data and transmitting it to a server, and includes smartphones, tablets, and personal computers.

[0261] "Dimensional data" refers to information collected to represent the state and environment of a spatial domain, as well as human activity, and is data that can capture physical arrangements and temporal changes.

[0262] "Spatial area" refers to the physical area that is the subject of cleaning, and includes the interior space of rooms and buildings.

[0263] "Human behavior patterns" refer to the tendencies of activities that residents engage in on a daily basis, and understanding these patterns can help optimize schedules.

[0264] "Operation schedule" refers to a plan that includes the date, time, and order in which cleaning tasks will be performed, and is optimized by a generative AI model.

[0265] A "generative AI model" is an algorithm that uses machine learning techniques to analyze data and create an optimal cleaning schedule.

[0266] A "control device" is an electronic device that operates work equipment based on instructions received from a server, and can operate multiple devices, including cleaning robots.

[0267] "Work equipment" refers to automated devices used to perform cleaning, and includes cleaning robots and other cleaning equipment.

[0268] This invention provides a system for automating and streamlining household cleaning. Specifically, it is achieved through a series of processes in which a server, terminals, and users work together.

[0269] Server operation:

[0270] The server collects dimensional data from information terminals. The data collected relates to the state of a spatial domain and human behavior patterns. This includes information such as the cleanliness of a room, the arrangement of furniture, and the daily routines of the inhabitants. Based on this data, the server preprocesses the data using Python's Pandas and NumPy, and builds a generative AI model using Scikit-learn and TensorFlow. This model is capable of generating an optimal schedule.

[0271] Device operation:

[0272] The system includes a cleaning robot and a smart speaker. The cleaning robot automatically begins cleaning according to the schedule sent from the server. Equipped with LIDAR sensors and cameras, the robot detects particles in the surrounding space in real time, enabling highly accurate cleaning. This allows for efficient and waste-free cleaning.

[0273] User actions:

[0274] Users can check the current cleaning schedule using a smartphone app or voice control device. They can also set cleaning conditions and priorities as needed through prompts. For example, they can give instructions such as, "Please set a new schedule that increases the frequency of cleaning the living room, taking past feedback into consideration, and does not clean at night."

[0275] This allows daily cleaning tasks to be performed automatically in accordance with the residents' daily routines, ensuring a consistently comfortable living environment and reducing the burden on users.

[0276] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0277] Step 1:

[0278] The server collects dimensional data through information terminals. It receives data such as dirt and furniture placement information from sensors in the spatial domain, as well as resident behavior patterns, as input. This data is stored in storage and prepared for later analysis.

[0279] Step 2:

[0280] The server preprocesses and analyzes the collected data. The input is the raw data obtained in step 1, which is cleaned up using Python's Pandas and NumPy to prepare it for analysis. The output is the analyzed dataset, which is used to optimize the operation schedule.

[0281] Step 3:

[0282] The server uses the analyzed data to build a generative AI model and generate an optimal operating schedule. The input is the dataset obtained in the previous step, and the model is trained using Scikit-learn or TensorFlow. The output is an optimized cleaning schedule, which determines the operating time and order of the cleaning robots.

[0283] Step 4:

[0284] The server will send the generated operation schedule to the terminal. The terminal, specifically the cleaning robot, will receive this schedule and prepare for execution. The output is the instruction data for the cleaning robot, based on which the robot will start automated cleaning.

[0285] Step 5:

[0286] The terminal will perform actual cleaning based on the received schedule. The input is the operation instruction from the server. The cleaning robot will scan the on-site environment using the LIDAR sensor and camera based on this and execute the cleaning. The output is the status data of cleaning completion, which will be fed back to the server.

[0287] Step 6:

[0288] The server will receive the feedback of cleaning completion obtained from the terminal and update the AI model. The input is the feedback information from the terminal, which is saved as new data and used for the retraining of the model. Thereby, the model is improved and utilized for more efficient schedule generation in subsequent times.

[0289] Step 7:

[0290] The user will use the smartphone app to check the schedule and settings and give instructions for modification if necessary. For example, the operation is performed in the form of a prompt sentence "I want to increase the cleaning frequency of the living room and make it not operate at night". The output is the new schedule and settings that meet the user's needs.

[0291] (Application Example 1)

[0292] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0293] Household cleaning is time-consuming and often a burden for residents with busy lifestyles. Furthermore, creating an efficient cleaning schedule requires considering residents' lifestyles and the condition of their rooms, which is difficult to manage manually. Therefore, there is a need for a system that automatically performs optimal cleaning based on the level of dirt and the resident's lifestyle.

[0294] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0295] In this invention, the server includes means for collecting information to analyze the degree of dirtiness in each room and the behavioral patterns of the residents, means for generating an optimal cleaning plan based on the information, and means including an information terminal application for notifying the user of the optimal cleaning timing. As a result, residents will have an optimal cleaning schedule suggested and executed according to their lifestyle, enabling them to live an efficient and stress-free life.

[0296] "Room-specific cleanliness" is an indicator of the level of cleaning required for each room, and is determined by measuring the amount of dust and debris.

[0297] "Resident behavior patterns" refer to the tendencies of residents' movement and activities in their daily lives, and this allows us to understand which rooms are used at which times of the day.

[0298] An "information terminal" refers to a portable computing device such as a smartphone or tablet that sends and receives data via the internet or applications.

[0299] A "cleaning machine" is a robotic device that automatically cleans floors, using internal sensors to detect dirt and obstacles and performing cleaning efficiently.

[0300] An "artificial intelligence model" is an algorithm that performs pattern recognition and prediction based on a large amount of data, with the aim of optimizing cleaning plans and improving them through feedback.

[0301] "Cleaning timing" refers to the optimal time to begin cleaning, which is determined by considering the resident's schedule and the room's usage.

[0302] An "information terminal application" is software that runs on an information terminal and provides cleaning management and notification functions.

[0303] To realize this invention, the server collects and analyzes data on the degree of dirtiness in each room and the behavioral patterns of the residents. Smartphones and tablets are used as information terminals. Specifically, these information terminals transmit the status of each room to the server via sensors. Based on this, the server uses AI technology such as TensorFlow to generate an optimal cleaning plan. The generated plan is notified to the user through an application on the information terminal, allowing residents to adjust the cleaning timing according to their own schedule and lifestyle.

[0304] The cleaning machine, or automated cleaning robot, receives instructions from a server and cleans along an efficient route while detecting dirt in real time. Once cleaning is complete, feedback is sent back to the server, and the artificial intelligence model is improved. This optimizes the cleaning schedule for future cleaning sessions.

[0305] For example, if a user wants their living room to be cleaned intensively on weekend afternoons, they can specify this through the application. They can also input prompts for the generating AI model, such as, "Based on resident A's lifestyle, please suggest the optimal cleaning schedule for this week."

[0306] Thus, the system provided by the invention can automate and optimize cleaning according to the user's lifestyle, improving the quality of the living environment.

[0307] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0308] Step 1:

[0309] The user makes initial settings through the terminal and inputs cleaning requirements and room priorities. The input includes the schedule of the residents, the allowable range of dirt for each room, etc. This data is sent to the server and registered as an initial dataset.

[0310] Step 2:

[0311] The server collects data for understanding the dirt condition of each room and the living pattern of the residents based on the data obtained in the initial settings. It collects sensor information of the cleaning machine and location information of the smartphone and stores them in the database. The data collected in this process is used for the analysis of the later AI model.

[0312] Step 3:

[0313] The server analyzes the collected data using AI software such as TensorFlow and generates an optimal cleaning plan by the generated AI model. The input data is the dirt state and usage frequency of the room, and the output is the specific cleaning timing and route. This plan is fed back to the user.

[0314] Step 4:

[0315] The terminal receives the cleaning plan sent from the server and notifies the user of the cleaning timing. The application of the terminal controls the cleaning machine and starts cleaning based on the plan. The user can confirm the progress of cleaning in real time through the notification at this step.

[0316] Step 5:

[0317] Once cleaning is complete, the cleaning machine sends feedback on its performance and the degree of soiling to a server. The server uses this feedback to improve its artificial intelligence model, thereby increasing the accuracy of the next cleaning plan. This allows for the automatic suggestion of a more optimal plan, even with the same information.

[0318] This process allows users to achieve efficient cleaning without hassle and maintain a comfortable living environment.

[0319] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0320] This invention is a smart system for streamlining household cleaning tasks, and in particular, it combines an emotion engine that recognizes user emotions to provide a more personalized cleaning experience. Specifically, this system collects data based on the degree of dirtiness in a room and the behavioral patterns of the inhabitants to provide an optimal cleaning schedule, and also has the function of detecting the user's emotions and reflecting them in the cleaning process.

[0321] Server Functions

[0322] The server collects and analyzes data from terminals, and uses an AI model to understand the level of dirtiness in each room and the residents' behavior patterns. Based on the results of this analysis, it generates an optimal cleaning schedule and sends it to the various terminals. Furthermore, it analyzes the user's voice input through an emotion engine and adjusts the schedule and cleaning methods based on that emotion.

[0323] Device functions

[0324] The devices include a robotic vacuum cleaner and a smart speaker. Through an emotion engine, the devices analyze the user's voice to understand their emotions and transmit this information to a server. The robotic vacuum cleaner cleans according to the received schedule, detecting the level of dirt in the room in real time and cleaning along an optimized route. It also sends feedback to the server to help improve future schedules.

[0325] User actions

[0326] Users can access the system through a smartphone app or voice assistant. Users can adjust the cleaning pace and timing based on their emotional state; for example, they can instruct the system to increase cleaning frequency during stressful periods. The system adapts the cleaning plan based on this input, providing a more satisfying cleaning experience.

[0327] Specific example

[0328] For example, consider a dual-income household where the user wants to clean efficiently when they come home tired. When the user expresses their emotions through voice input such as "I'm tired," the emotion engine recognizes this, and the server adjusts the cleaning robot's operation to reduce stress. For example, it might clean in silent mode, providing an environment that doesn't cause stress to the user. After completion, the cleaning robot sends feedback to the server, which is then used to improve future situations.

[0329] Thus, this invention makes it possible to achieve personalized cleaning that is attentive to the user's emotions and improve the living environment within the home.

[0330] The following describes the processing flow.

[0331] Step 1:

[0332] Users initiate interaction with the system by expressing emotions through a smartphone app or voice input. This can be done simply by reporting their physical condition or mood through voice.

[0333] Step 2:

[0334] The emotion engine built into the device analyzes the user's voice input and identifies emotions from that voice. For example, it detects voice tone and specific keywords to extract emotions such as "tired" or "want to relax."

[0335] Step 3:

[0336] The device sends the analysis results to the server and requests that it generate cleaning requirements that have been adjusted based on the user's current sentiment data.

[0337] Step 4:

[0338] The server generates a new cleaning schedule based on received emotional data, the current level of dirtiness in the room, and the residents' behavior patterns. For example, if the user is tired, it may increase the cleaning frequency or recommend cleaning in silent mode.

[0339] Step 5:

[0340] The server sends the generated cleaning schedule and any emotionally-based adjustments to the device's cleaning robot.

[0341] Step 6:

[0342] The cleaning robot starts cleaning in the specified mode according to a schedule received from the server. In practice, the robot moves around the room, updating information about the room in real time using sensors to clean efficiently.

[0343] Step 7:

[0344] After cleaning is complete, the device sends the cleaning results and user feedback back to the server. This allows the server to use the information to improve the AI ​​model.

[0345] Step 8:

[0346] The user receives a cleaning completion notification via an app or similar means, confirming that the cleaning was performed appropriately according to their emotional needs. The system evaluates whether it performed as expected and makes further adjustments as needed.

[0347] Through these steps, the system provides a flexible and personalized cleaning process that takes user emotions into consideration.

[0348] (Example 2)

[0349] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0350] In modern homes, cleaning is an important and time-consuming task, but conventional automated cleaning systems often lack efficiency because they fail to adequately consider the lifestyles and emotional states of residents. Furthermore, the operation of cleaning robots can sometimes increase residents' stress levels. Therefore, there is a need for cleaning systems that are adapted to the behavior and emotions of residents.

[0351] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0352] In this invention, the server includes means for collecting information to analyze the pollution status of each environment and the behavioral characteristics of the residents, means for creating an optimal cleaning schedule based on the information, and means for analyzing emotions from the user's voice instructions and adjusting the cleaning process and schedule accordingly. This makes it possible to provide residents with an individually optimized cleaning experience and improve the quality of the living environment.

[0353] "Environment" refers to the individual rooms or areas that are to be cleaned, and is a target area related to its level of contamination and the behavioral characteristics of the residents.

[0354] "Contamination level" refers to the degree of accumulation of impurities such as dust and debris within the environment being cleaned.

[0355] "Resident behavioral characteristics" refer to the activity patterns and lifestyle habits of residents within a specific environment, and are used to optimize cleaning schedules.

[0356] "Means of collecting information" refers to methods of acquiring data on environmental pollution levels and the behavioral characteristics of residents using sensors and cameras.

[0357] A "cleaning schedule" refers to the operating schedule of cleaning equipment created based on acquired information, and represents a plan for effectively cleaning the environment.

[0358] "Methods for analyzing emotions from voice commands" refers to technologies that use algorithms and models to analyze a user's voice input and estimate the user's emotions and stress levels.

[0359] "Means of adjusting the cleaning process and schedule" refers to methods for appropriately changing the operating modes and schedules of cleaning equipment based on analyzed emotional data.

[0360] This invention is a system for efficiently performing cleaning tasks in a residential environment, combining information technology and emotion analysis technology to provide a more personalized cleaning experience.

[0361] The server functions as a device that receives and analyzes data collected from connected devices in real time. This data includes the level of contamination in rooms obtained from sensors and the behavioral characteristics of residents captured by cameras. A generative AI model is used for analysis, automatically generating an optimal cleaning schedule from each data point. In addition, the server utilizes a voice analysis engine to analyze emotions from the user's voice obtained through the device, and has the ability to adjust the cleaning process and schedule as needed.

[0362] The system includes a cleaning robot and a smart speaker for receiving user voice commands. The cleaning robot cleans rooms efficiently according to a cleaning schedule received from the server. During this process, the robot detects environmental contamination in real time and sends this data back to the server as feedback. This data is then used to optimize the next cleaning plan.

[0363] Users can access the system through a smartphone application or voice assistant and easily give cleaning instructions tailored to their emotional state. In particular, when stressed, entering prompts such as "I'm tired" into a smart speaker will instruct the cleaning robot to operate in quiet mode, thus reducing stress for residents.

[0364] This invention overcomes the limitations of conventional automatic cleaning devices, enabling customization based on lifestyle habits and individual emotional states. This significantly improves the quality of the living environment.

[0365] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0366] Step 1:

[0367] The terminal collects data on environmental pollution levels and resident behavioral characteristics via sensors and cameras installed in the room. The collected data includes real-time information such as temperature, humidity, and camera footage. This input data is used to assess pollution levels and analyze resident behavior, and is then transmitted to a server.

[0368] Step 2:

[0369] The server inputs environmental data received from the terminal into a generating AI model, which then analyzes the data. The AI ​​model utilizes machine learning algorithms to evaluate the degree of contamination in each room and the behavioral patterns of the residents. This analysis identifies the time and methods required for cleaning and generates an optimal cleaning schedule.

[0370] Step 3:

[0371] The server analyzes the emotional information entered by the user via voice through a voice analysis engine. The voice data is used to infer the user's emotional state. The emotional analysis results obtained from the prompt text are compared with the current cleaning plan, and output is generated to readjust the cleaning schedule and cleaning methods as needed.

[0372] Step 4:

[0373] Upon receiving instructions from the server, the cleaning robot at the terminal begins operation according to the predetermined cleaning schedule. The robot uses sensors to detect contamination in the room in real time and cleans along an optimized route. It monitors the progress of cleaning and the condition of the room, and dynamically adjusts the route and cleaning mode.

[0374] Step 5:

[0375] After cleaning is complete, the terminal reports completion to the server, along with feedback on the cleaning results and the latest environmental data. The server receives this feedback and processes the data to improve the accuracy of the next cleaning plan. It also stores the data as training data for the generated AI model, enabling continuous performance improvement.

[0376] This series of processes makes it possible to provide residents with an efficient and comfortable cleaning experience.

[0377] (Application Example 2)

[0378] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0379] In recent years, household cleaning machines have been required to be highly efficient and comfortable to use, but conventional technology has struggled to flexibly adapt to the user's lifestyle and emotional state. In particular, a cleaning system that can appropriately respond to specific emotional states, such as when the user is fatigued or stressed, has yet to be established. Therefore, there is a need to develop a new cleaning system that can recognize emotions and appropriately adjust the cleaning process according to the user's situation.

[0380] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0381] In this invention, the server includes means for collecting data to analyze the degree of dirtiness in each room and the behavioral patterns of the residents, means for generating an optimal cleaning schedule, and means for detecting the emotional state of the residents and adjusting the operation of the cleaning machine based on that. This makes it possible to optimize the operation of the cleaning machine according to the emotional state of the user and provide a comfortable living environment.

[0382] "Room-specific cleanliness" refers to the condition or level of cleanliness in each individual room, and is measured based on the frequency and type of dirt that occurs.

[0383] "Resident behavior patterns" refer to patterns of behavior and actions of people living within a residence, and include information about their daily activities and movement tendencies.

[0384] An "optimal cleaning schedule" is a set of times and sequences planned to perform cleaning tasks with maximum efficiency, and is formulated based on the degree of soiling and the user's daily schedule.

[0385] "Terminal" is a general term for electronic devices related to this system, and includes cleaning machines and devices that handle information display and communication.

[0386] A "cleaning machine" is a mechanical device designed for cleaning indoor spaces, with the purpose of removing dirt and purifying the environment.

[0387] "Resident's emotional state" refers to the user's psychological or emotional condition and is determined based on information from their voice and behavior.

[0388] An "artificial intelligence model" is a mathematical or computational model designed to perform a specific task based on data, and possesses the ability to learn from experience.

[0389] The system that realizes this application example is constructed as follows: This system is an advanced home cleaning system that provides a personalized cleaning experience based on the user's emotions. This system consists of a server, terminals (cleaning machines and information display devices), and a user interface.

[0390] The server is the core of the system, responsible for all data analysis and schedule generation. It utilizes data collected from various sensors and input devices to analyze the level of dirtiness in each room and the residents' behavioral patterns. This data is then analyzed by an AI model to generate an optimal cleaning schedule. Furthermore, the server has the ability to detect the user's emotional state from their voice and behavior using an emotion engine, and adjust the cleaning machine's operation accordingly.

[0391] The device includes a vacuum cleaner and a voice assistance device. The vacuum cleaner uses robotic cleaning technology to clean the room and transmits status data to the server in real time. The voice assistance device analyzes the user's voice and automatically transmits their emotional state to the server. This allows the cleaning mode to be adjusted according to the user's emotional state.

[0392] Users can access and operate the system via a mobile device or voice assistance system. Users can set the cleaning pace and timing by directly inputting their emotional state or giving voice instructions. Through this, the server adaptively trains its AI model to improve the accuracy of future cleaning plans.

[0393] For example, if a user gives a voice command saying, "I want it to be quiet today," the server will interpret the voice as indicating the user is seeking silence and operate the vacuum cleaner in silent mode. Furthermore, if the user is feeling stressed, a relaxation mode will be suggested.

[0394] For this system to function, the following prompt can be used: "Design an app that detects the user's emotions and adjusts the operation of the vacuum cleaner accordingly."

[0395] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0396] Step 1:

[0397] The server collects sensor data to measure residents' behavior patterns and the level of dirtiness in each room. The input is environmental data obtained from various sensors, and the output is a dataset organized for necessary analysis. The server integrates this data and prepares it as foundational data for quantifying behavioral patterns.

[0398] Step 2:

[0399] The server uses an emotion engine to analyze the user's voice input and detect their emotional state. The input is the user's voice data, and the output is an emotion label. The voice waveform is analyzed via a speech processing algorithm, and the emotion is classified based on an emotion model. This result is then used in the next step.

[0400] Step 3:

[0401] The server generates an optimal cleaning schedule based on the behavioral pattern data obtained in Step 1 and the emotional state labels obtained in Step 2. The input is the behavioral patterns and emotional labels obtained in the previous step, and the output is a specific cleaning schedule. An AI model is used to analyze this data and create individually customized schedules.

[0402] Step 4:

[0403] The vacuum cleaner in the terminal starts cleaning according to the cleaning schedule sent from the server. The input is the schedule data received from the server, and the output is feedback data after cleaning is complete. The vacuum cleaner proceeds with cleaning while detecting the room conditions with its built-in sensors, and when cleaning is complete, it sends the results to the server.

[0404] Step 5:

[0405] Users operate the system and send feedback via mobile devices or voice assistance systems. Inputs are the user's experiences and requests, while outputs are data used to train the next AI model. User feedback is reflected on the server and used to improve the next schedule generation.

[0406] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0407] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0408] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0409] [Third Embodiment]

[0410] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0411] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0412] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0413] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0414] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0415] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0416] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0417] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0418] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0419] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0420] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0421] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0422] This invention provides a smart system that utilizes AI technology to streamline household cleaning. Specifically, it constructs a system that analyzes the level of dirtiness in each room and the residents' daily routines, and then proposes and executes an optimal cleaning schedule.

[0423] Server Functions

[0424] The server collects data from multiple devices regarding the degree of dirtiness in each room, furniture arrangement, and residents' behavior patterns. Data collection is performed through cleaning robots, smartphone apps, and voice assistants. The server uses this data to analyze it and build an AI model. Based on the AI ​​model, it generates an optimal cleaning schedule and sends it to each device.

[0425] Device functions

[0426] The system includes a robotic vacuum cleaner and a smart speaker. The robotic vacuum cleaner automatically starts cleaning according to a schedule received from the server. As the robot moves around the room, its built-in sensors detect dust and debris in real time, and it cleans intensively along the optimal route.

[0427] The smart speaker sends voice commands from the user to a server to check if any corrections are needed to the AI ​​model. The device also feeds back the status and results to the server after cleaning is complete to help adjust future schedules.

[0428] User actions

[0429] Users can access the system through a smartphone app and input individual requirements such as room layout and cleaning frequency. Users can monitor the cleaning progress through the app and adjust schedules and settings as needed. Using a voice assistant makes it even easier for users to issue cleaning commands and check results.

[0430] Specific example

[0431] For example, a dual-income household might want to set a schedule to finish cleaning the floors by 6 p.m. on weekdays. Based on past data, the server creates a schedule to run the cleaning robot while the residents are at the office, identifying the times when cleaning is needed. Once the user sets room priorities in the app, the robot starts cleaning from the living room and efficiently covers the entire house. After completion, the robot reports its cleaning status to the server, which is then used to improve future schedules.

[0432] Thus, the present invention makes it possible to automate and streamline household cleaning, thereby improving the quality of the living environment.

[0433] The following describes the processing flow.

[0434] Step 1:

[0435] The server collects data from terminals regarding the level of cleanliness in each room, furniture arrangement, and residents' behavior patterns. This includes sensor data and user input data.

[0436] Step 2:

[0437] The server uses an AI model to analyze the collected data and understand the level of dirtiness in each room and the residents' behavior patterns. This provides the basic information needed to determine cleaning priorities and timing.

[0438] Step 3:

[0439] The server generates an optimal cleaning schedule based on the analysis results. This schedule includes the time slots and specific routes for cleaning.

[0440] Step 4:

[0441] The server sends the generated cleaning schedule to the terminal and provides specific instructions to the cleaning robot.

[0442] Step 5:

[0443] The cleaning robot included in the device starts cleaning at the designated time according to the schedule received from the server. The robot uses its built-in sensors to detect dust and debris in real time, moving efficiently around the room and cleaning.

[0444] Step 6:

[0445] Users can monitor the cleaning progress through a smartphone app or voice assistant. They can also adjust cleaning schedules and settings via the app as needed.

[0446] Step 7:

[0447] After cleaning is complete, the device sends the cleaning results to the server. This information is used to improve the AI ​​model for the next cleaning. Based on the feedback, the server continues to optimize the schedule and cleaning procedure.

[0448] (Example 1)

[0449] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0450] In modern homes, daily cleaning tasks are often not performed efficiently amidst busy lifestyles, creating a need for improved living environments. Furthermore, conventional cleaning equipment and services struggle to provide flexible schedules based on the degree of dirtiness in rooms and the residents' daily routines, highlighting the need for automated systems.

[0451] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0452] In this invention, the server includes means for collecting dimensional data from information terminals and analyzing the state of each spatial domain and human behavior patterns; means for constructing a generative AI model that optimizes the operation schedule based on the data; and means for transmitting the optimized schedule to a control device and controlling the work equipment. This makes it possible to perform cleaning activities automatically and efficiently, improving the quality of the living environment.

[0453] An "information terminal" is a device that has the function of collecting data and transmitting it to a server, and includes smartphones, tablets, and personal computers.

[0454] "Dimensional data" refers to information collected to represent the state and environment of a spatial domain, as well as human activity, and is data that can capture physical arrangements and temporal changes.

[0455] "Spatial area" refers to the physical area that is the subject of cleaning, and includes the interior space of rooms and buildings.

[0456] "Human behavior patterns" refer to the tendencies of activities that residents engage in on a daily basis, and understanding these patterns can help optimize schedules.

[0457] "Operation schedule" refers to a plan that includes the date, time, and order in which cleaning tasks will be performed, and is optimized by a generative AI model.

[0458] A "generative AI model" is an algorithm that uses machine learning techniques to analyze data and create an optimal cleaning schedule.

[0459] A "control device" is an electronic device that operates work equipment based on instructions received from a server, and can operate multiple devices, including cleaning robots.

[0460] "Work equipment" refers to automated devices used to perform cleaning, and includes cleaning robots and other cleaning equipment.

[0461] This invention provides a system for automating and streamlining household cleaning. Specifically, it is achieved through a series of processes in which a server, terminals, and users work together.

[0462] Server operation:

[0463] The server collects dimensional data from information terminals. The data collected relates to the state of a spatial domain and human behavior patterns. This includes information such as the cleanliness of a room, the arrangement of furniture, and the daily routines of the inhabitants. Based on this data, the server preprocesses the data using Python's Pandas and NumPy, and builds a generative AI model using Scikit-learn and TensorFlow. This model is capable of generating an optimal schedule.

[0464] Device operation:

[0465] The system includes a cleaning robot and a smart speaker. The cleaning robot automatically begins cleaning according to the schedule sent from the server. Equipped with LIDAR sensors and cameras, the robot detects particles in the surrounding space in real time, enabling highly accurate cleaning. This allows for efficient and waste-free cleaning.

[0466] User actions:

[0467] Users can check the current cleaning schedule using a smartphone app or voice control device. They can also set cleaning conditions and priorities as needed through prompts. For example, they can give instructions such as, "Please set a new schedule that increases the frequency of cleaning the living room, taking past feedback into consideration, and does not clean at night."

[0468] This allows daily cleaning tasks to be performed automatically in accordance with the residents' daily routines, ensuring a consistently comfortable living environment and reducing the burden on users.

[0469] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0470] Step 1:

[0471] The server collects dimensional data through information terminals. It receives data such as dirt and furniture placement information from sensors in the spatial domain, as well as resident behavior patterns, as input. This data is stored in storage and prepared for later analysis.

[0472] Step 2:

[0473] The server preprocesses and analyzes the collected data. The input is the raw data obtained in step 1, which is cleaned up using Python's Pandas and NumPy to prepare it for analysis. The output is the analyzed dataset, which is used to optimize the operation schedule.

[0474] Step 3:

[0475] The server uses the analyzed data to build a generative AI model and generate an optimal operating schedule. The input is the dataset obtained in the previous step, and the model is trained using Scikit-learn or TensorFlow. The output is an optimized cleaning schedule, which determines the operating time and order of the cleaning robots.

[0476] Step 4:

[0477] The server sends the generated operation schedule to the terminal. The terminal, specifically the cleaning robot, receives this schedule and prepares to execute it. The output is instruction data for the cleaning robot, which then starts the automated cleaning process.

[0478] Step 5:

[0479] The terminal performs the actual cleaning based on the received schedule. The input is an operation instruction from the server, and the cleaning robot uses this to scan the site environment using LIDAR sensors and cameras and then performs the cleaning. The output is cleaning completion status data, which is fed back to the server.

[0480] Step 6:

[0481] The server receives feedback from the terminal indicating that cleaning is complete, and updates the AI ​​model. The input is feedback information from the terminal, which is saved as new data and used to retrain the model. This improves the model, which is then used to generate more efficient schedules for future cleanings.

[0482] Step 7:

[0483] The user uses a smartphone app to check the schedule and settings and make adjustments as needed. For example, they might use a prompt message such as, "Please increase the cleaning frequency in the living room and disable nighttime cleaning." The output is a new schedule and settings tailored to the user's needs.

[0484] (Application Example 1)

[0485] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0486] Household cleaning is time-consuming and often a burden for residents with busy lifestyles. Furthermore, creating an efficient cleaning schedule requires considering residents' lifestyles and the condition of their rooms, which is difficult to manage manually. Therefore, there is a need for a system that automatically performs optimal cleaning based on the level of dirt and the resident's lifestyle.

[0487] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0488] In this invention, the server includes means for collecting information to analyze the degree of dirtiness in each room and the behavioral patterns of the residents, means for generating an optimal cleaning plan based on the information, and means including an information terminal application for notifying the user of the optimal cleaning timing. As a result, residents will have an optimal cleaning schedule suggested and executed according to their lifestyle, enabling them to live an efficient and stress-free life.

[0489] "Room-specific cleanliness" is an indicator of the level of cleaning required for each room, and is determined by measuring the amount of dust and debris.

[0490] "Resident behavior patterns" refer to the tendencies of residents' movement and activities in their daily lives, and this allows us to understand which rooms are used at which times of the day.

[0491] An "information terminal" refers to a portable computing device such as a smartphone or tablet that sends and receives data via the internet or applications.

[0492] A "cleaning machine" is a robotic device that automatically cleans floors, using internal sensors to detect dirt and obstacles and performing cleaning efficiently.

[0493] An "artificial intelligence model" is an algorithm that performs pattern recognition and prediction based on a large amount of data, with the aim of optimizing cleaning plans and improving them through feedback.

[0494] "Cleaning timing" refers to the optimal time to begin cleaning, and is determined by considering the resident's schedule and the room's usage.

[0495] An "information terminal application" is software that runs on an information terminal and provides cleaning management and notification functions.

[0496] To realize this invention, the server collects and analyzes data on the degree of dirtiness in each room and the behavioral patterns of the residents. Smartphones and tablets are used as information terminals. Specifically, these information terminals transmit the status of each room to the server via sensors. Based on this, the server uses AI technology such as TensorFlow to generate an optimal cleaning plan. The generated plan is notified to the user through an application on the information terminal, allowing residents to adjust the cleaning timing according to their own schedule and lifestyle.

[0497] The cleaning machine, or automated cleaning robot, receives instructions from a server and cleans along an efficient route while detecting dirt in real time. Once cleaning is complete, feedback is sent back to the server, and the artificial intelligence model is improved. This optimizes the cleaning schedule for future cleaning sessions.

[0498] For example, if a user wants their living room to be cleaned intensively on weekend afternoons, they can specify this through the application. They can also input prompts for the generating AI model, such as, "Based on resident A's lifestyle, please suggest the optimal cleaning schedule for this week."

[0499] Thus, the system provided by the invention can automate and optimize cleaning according to the user's lifestyle, thereby improving the quality of the living environment.

[0500] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0501] Step 1:

[0502] Users perform initial setup via a terminal, entering cleaning requirements and room priorities. This input includes resident schedules and acceptable levels of dirtiness for each room. This data is sent to the server and registered as the initial dataset.

[0503] Step 2:

[0504] Based on the initial setup data, the server collects data to understand the level of dirtiness in each room and the residents' lifestyle patterns. It collects sensor information from cleaning machines and location information from smartphones and stores it in a database. The data collected in this process will be used later for the analysis of AI models.

[0505] Step 3:

[0506] The server analyzes the collected data using AI software such as TensorFlow and generates an optimal cleaning plan using a generative AI model. Input data includes the room's level of dirtiness and usage frequency, while output is specific cleaning timings and routes. This plan is then fed back to the user.

[0507] Step 4:

[0508] The terminal receives the cleaning plan sent from the server and notifies the user of the cleaning timing. The terminal's application controls the cleaning machine and starts cleaning according to the plan. At this step, the user can check the progress of the cleaning in real time via notifications.

[0509] Step 5:

[0510] Once cleaning is complete, the cleaning machine sends feedback on its performance and the degree of soiling to a server. The server uses this feedback to improve its artificial intelligence model, thereby increasing the accuracy of the next cleaning plan. This allows for the automatic suggestion of a more optimal plan, even with the same information.

[0511] This process allows users to achieve efficient cleaning without hassle and maintain a comfortable living environment.

[0512] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0513] This invention is a smart system for streamlining household cleaning tasks, and in particular, it combines an emotion engine that recognizes user emotions to provide a more personalized cleaning experience. Specifically, this system collects data based on the degree of dirtiness in a room and the behavioral patterns of the inhabitants to provide an optimal cleaning schedule, and also has the function of detecting the user's emotions and reflecting them in the cleaning process.

[0514] Server Functions

[0515] The server collects and analyzes data from terminals, and uses an AI model to understand the level of dirtiness in each room and the residents' behavior patterns. Based on the results of this analysis, it generates an optimal cleaning schedule and sends it to the various terminals. Furthermore, it analyzes the user's voice input through an emotion engine and adjusts the schedule and cleaning methods based on that emotion.

[0516] Device functions

[0517] The devices include a robotic vacuum cleaner and a smart speaker. Through an emotion engine, the devices analyze the user's voice to understand their emotions and transmit this information to a server. The robotic vacuum cleaner cleans according to the received schedule, detecting the level of dirt in the room in real time and cleaning along an optimized route. It also sends feedback to the server to help improve future schedules.

[0518] User actions

[0519] Users can access the system through a smartphone app or voice assistant. Users can adjust the cleaning pace and timing based on their emotional state; for example, they can instruct the system to increase cleaning frequency during stressful periods. The system adapts the cleaning plan based on this input, providing a more satisfying cleaning experience.

[0520] Specific example

[0521] For example, consider a dual-income household where the user wants to clean efficiently when they come home tired. When the user expresses their emotions through voice input such as "I'm tired," the emotion engine recognizes this, and the server adjusts the cleaning robot's operation to reduce stress. For example, it might clean in silent mode, providing an environment that doesn't cause stress to the user. After completion, the cleaning robot sends feedback to the server, which is then used to improve future situations.

[0522] Thus, this invention makes it possible to achieve personalized cleaning that is attentive to the user's emotions and improve the living environment within the home.

[0523] The following describes the processing flow.

[0524] Step 1:

[0525] Users initiate interaction with the system by expressing emotions through a smartphone app or voice input. This can be done simply by reporting their physical condition or mood through voice.

[0526] Step 2:

[0527] The emotion engine built into the device analyzes the user's voice input and identifies emotions from that voice. For example, it detects voice tone and specific keywords to extract emotions such as "tired" or "want to relax."

[0528] Step 3:

[0529] The device sends the analysis results to the server and requests that it generate cleaning requirements that have been adjusted based on the user's current sentiment data.

[0530] Step 4:

[0531] The server generates a new cleaning schedule based on received emotional data, the current level of dirtiness in the room, and the residents' behavior patterns. For example, if the user is tired, it may increase the cleaning frequency or recommend cleaning in silent mode.

[0532] Step 5:

[0533] The server sends the generated cleaning schedule and any emotionally-based adjustments to the device's cleaning robot.

[0534] Step 6:

[0535] The cleaning robot starts cleaning in the specified mode according to a schedule received from the server. In practice, the robot moves around the room, updating information about the room in real time using sensors to clean efficiently.

[0536] Step 7:

[0537] After cleaning is complete, the device sends the cleaning results and user feedback back to the server. This allows the server to use the information to improve the AI ​​model.

[0538] Step 8:

[0539] The user receives a cleaning completion notification via an app or similar means, confirming that the cleaning was performed appropriately according to their emotional needs. The system evaluates whether it performed as expected and makes further adjustments as needed.

[0540] Through these steps, the system provides a flexible and personalized cleaning process that takes user emotions into consideration.

[0541] (Example 2)

[0542] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0543] In modern homes, cleaning is an important and time-consuming task, but conventional automated cleaning systems often lack efficiency because they fail to adequately consider the lifestyles and emotional states of residents. Furthermore, the operation of cleaning robots can sometimes increase residents' stress levels. Therefore, there is a need for cleaning systems that are adapted to the behavior and emotions of residents.

[0544] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0545] In this invention, the server includes means for collecting information to analyze the pollution status of each environment and the behavioral characteristics of the residents, means for creating an optimal cleaning schedule based on the information, and means for analyzing emotions from the user's voice instructions and adjusting the cleaning process and schedule accordingly. This makes it possible to provide residents with an individually optimized cleaning experience and improve the quality of the living environment.

[0546] "Environment" refers to the individual rooms or areas that are to be cleaned, and is a target area related to its level of contamination and the behavioral characteristics of the residents.

[0547] "Contamination level" refers to the degree of accumulation of impurities such as dust and debris within the environment being cleaned.

[0548] "Resident behavioral characteristics" refer to the activity patterns and lifestyle habits of residents within a specific environment, and are used to optimize cleaning schedules.

[0549] "Means of collecting information" refers to methods of acquiring data on environmental pollution levels and the behavioral characteristics of residents using sensors and cameras.

[0550] A "cleaning schedule" refers to the operating schedule of cleaning equipment created based on acquired information, and represents a plan for effectively cleaning the environment.

[0551] "Methods for analyzing emotions from voice commands" refers to technologies that use algorithms and models to analyze a user's voice input and estimate the user's emotions and stress levels.

[0552] "Means of adjusting the cleaning process and schedule" refers to methods for appropriately changing the operating modes and schedules of cleaning equipment based on analyzed emotional data.

[0553] This invention is a system for efficiently performing cleaning tasks in a residential environment, combining information technology and emotion analysis technology to provide a more personalized cleaning experience.

[0554] The server functions as a device that receives and analyzes data collected from connected devices in real time. This data includes the level of contamination in rooms obtained from sensors and the behavioral characteristics of residents captured by cameras. A generative AI model is used for analysis, automatically generating an optimal cleaning schedule from each data point. In addition, the server utilizes a voice analysis engine to analyze emotions from the user's voice obtained through the device, and has the ability to adjust the cleaning process and schedule as needed.

[0555] The system includes a cleaning robot and a smart speaker for receiving user voice commands. The cleaning robot cleans rooms efficiently according to a cleaning schedule received from the server. During this process, the robot detects environmental contamination in real time and sends this data back to the server as feedback. This data is then used to optimize the next cleaning plan.

[0556] Users can access the system through a smartphone application or voice assistant and easily give cleaning instructions tailored to their emotional state. In particular, when stressed, entering prompts such as "I'm tired" into a smart speaker will instruct the cleaning robot to operate in quiet mode, thus reducing stress for residents.

[0557] This invention overcomes the limitations of conventional automatic cleaning devices, enabling customization based on lifestyle habits and individual emotional states. This significantly improves the quality of the living environment.

[0558] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0559] Step 1:

[0560] The terminal collects data on environmental pollution levels and resident behavioral characteristics via sensors and cameras installed in the room. The collected data includes real-time information such as temperature, humidity, and camera footage. This input data is used to assess pollution levels and analyze resident behavior, and is then transmitted to a server.

[0561] Step 2:

[0562] The server inputs environmental data received from the terminal into a generating AI model, which then analyzes the data. The AI ​​model utilizes machine learning algorithms to evaluate the degree of contamination in each room and the behavioral patterns of the residents. This analysis identifies the time and methods required for cleaning and generates an optimal cleaning schedule.

[0563] Step 3:

[0564] The server analyzes the emotional information entered by the user via voice through a voice analysis engine. The voice data is used to infer the user's emotional state. The emotional analysis results obtained from the prompt text are compared with the current cleaning plan, and output is generated to readjust the cleaning schedule and cleaning methods as needed.

[0565] Step 4:

[0566] Upon receiving instructions from the server, the cleaning robot at the terminal begins operation according to the predetermined cleaning schedule. The robot uses sensors to detect contamination in the room in real time and cleans along an optimized route. It monitors the progress of cleaning and the condition of the room, and dynamically adjusts the route and cleaning mode.

[0567] Step 5:

[0568] After cleaning is complete, the terminal reports completion to the server, along with feedback on the cleaning results and the latest environmental data. The server receives this feedback and processes the data to improve the accuracy of the next cleaning plan. It also stores the data as training data for the generated AI model, enabling continuous performance improvement.

[0569] This series of processes makes it possible to provide residents with an efficient and comfortable cleaning experience.

[0570] (Application Example 2)

[0571] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0572] In recent years, household cleaning machines have been required to be highly efficient and comfortable to use, but conventional technology has struggled to flexibly adapt to the user's lifestyle and emotional state. In particular, a cleaning system that can appropriately respond to specific emotional states, such as when the user is fatigued or stressed, has yet to be established. Therefore, there is a need to develop a new cleaning system that can recognize emotions and appropriately adjust the cleaning process according to the user's situation.

[0573] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0574] In this invention, the server includes means for collecting data to analyze the degree of dirtiness in each room and the behavioral patterns of the residents, means for generating an optimal cleaning schedule, and means for detecting the emotional state of the residents and adjusting the operation of the cleaning machine based on that. This makes it possible to optimize the operation of the cleaning machine according to the emotional state of the user and provide a comfortable living environment.

[0575] "Room-specific cleanliness" refers to the condition or level of cleanliness in each individual room, and is measured based on the frequency and type of dirt that occurs.

[0576] "Resident behavior patterns" refer to patterns of behavior and actions of people living within a residence, and include information about their daily activities and movement tendencies.

[0577] An "optimal cleaning schedule" is a set of times and sequences planned to perform cleaning tasks with maximum efficiency, and is formulated based on the degree of soiling and the user's daily schedule.

[0578] "Terminal" is a general term for electronic devices related to this system, and includes cleaning machines and devices that handle information display and communication.

[0579] A "cleaning machine" is a mechanical device designed for cleaning indoor spaces, with the purpose of removing dirt and purifying the environment.

[0580] "Resident's emotional state" refers to the user's psychological or emotional condition and is determined based on information from their voice and behavior.

[0581] An "artificial intelligence model" is a mathematical or computational model designed to perform a specific task based on data, and possesses the ability to learn from experience.

[0582] The system that realizes this application example is constructed as follows: This system is an advanced home cleaning system that provides a personalized cleaning experience based on the user's emotions. This system consists of a server, terminals (cleaning machines and information display devices), and a user interface.

[0583] The server is the core of the system, responsible for all data analysis and schedule generation. It utilizes data collected from various sensors and input devices to analyze the level of dirtiness in each room and the residents' behavioral patterns. This data is then analyzed by an AI model to generate an optimal cleaning schedule. Furthermore, the server has the ability to detect the user's emotional state from their voice and behavior using an emotion engine, and adjust the cleaning machine's operation accordingly.

[0584] The device includes a vacuum cleaner and a voice assistance device. The vacuum cleaner uses robotic cleaning technology to clean the room and transmits status data to the server in real time. The voice assistance device analyzes the user's voice and automatically transmits their emotional state to the server. This allows the cleaning mode to be adjusted according to the user's emotional state.

[0585] Users can access and operate the system via a mobile device or voice assistance system. Users can set the cleaning pace and timing by directly inputting their emotional state or giving voice instructions. Through this, the server adaptively trains its AI model to improve the accuracy of future cleaning plans.

[0586] For example, if a user gives a voice command saying, "I want it to be quiet today," the server will interpret the voice as indicating the user is seeking silence and operate the vacuum cleaner in silent mode. Furthermore, if the user is feeling stressed, a relaxation mode will be suggested.

[0587] For this system to function, the following prompt can be used: "Design an app that detects the user's emotions and adjusts the operation of the vacuum cleaner accordingly."

[0588] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0589] Step 1:

[0590] The server collects sensor data to measure residents' behavior patterns and the level of dirtiness in each room. The input is environmental data obtained from various sensors, and the output is a dataset organized for necessary analysis. The server integrates this data and prepares it as foundational data for quantifying behavioral patterns.

[0591] Step 2:

[0592] The server uses an emotion engine to analyze the user's voice input and detect their emotional state. The input is the user's voice data, and the output is an emotion label. The voice waveform is analyzed via a speech processing algorithm, and the emotion is classified based on an emotion model. This result is then used in the next step.

[0593] Step 3:

[0594] The server generates an optimal cleaning schedule based on the behavioral pattern data obtained in Step 1 and the emotional state labels obtained in Step 2. The input is the behavioral patterns and emotional labels obtained in the previous step, and the output is a specific cleaning schedule. An AI model is used to analyze this data and create individually customized schedules.

[0595] Step 4:

[0596] The vacuum cleaner in the terminal starts cleaning according to the cleaning schedule sent from the server. The input is the schedule data received from the server, and the output is feedback data after cleaning is complete. The vacuum cleaner proceeds with cleaning while detecting the room conditions with its built-in sensors, and when cleaning is complete, it sends the results to the server.

[0597] Step 5:

[0598] Users operate the system and send feedback via mobile devices or voice assistance systems. Inputs are the user's experiences and requests, while outputs are data used to train the next AI model. User feedback is reflected on the server and used to improve the next schedule generation.

[0599] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0600] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0601] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0602] [Fourth Embodiment]

[0603] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0604] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0605] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0606] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0607] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0608] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0609] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0610] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0611] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0612] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0613] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0614] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0615] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0616] This invention provides a smart system that utilizes AI technology to streamline household cleaning. Specifically, it constructs a system that analyzes the level of dirtiness in each room and the residents' daily routines, and then proposes and executes an optimal cleaning schedule.

[0617] Server Functions

[0618] The server collects data from multiple devices regarding the degree of dirtiness in each room, furniture arrangement, and residents' behavior patterns. Data collection is performed through cleaning robots, smartphone apps, and voice assistants. The server uses this data to analyze it and build an AI model. Based on the AI ​​model, it generates an optimal cleaning schedule and sends it to each device.

[0619] Device functions

[0620] The system includes a robotic vacuum cleaner and a smart speaker. The robotic vacuum cleaner automatically starts cleaning according to a schedule received from the server. As the robot moves around the room, its built-in sensors detect dust and debris in real time, and it cleans intensively along the optimal route.

[0621] The smart speaker sends voice commands from the user to a server to check if any corrections are needed to the AI ​​model. The device also feeds back the status and results to the server after cleaning is complete to help adjust future schedules.

[0622] User actions

[0623] Users can access the system through a smartphone app and input individual requirements such as room layout and cleaning frequency. Users can monitor the cleaning progress through the app and adjust schedules and settings as needed. Using a voice assistant makes it even easier for users to issue cleaning commands and check results.

[0624] Specific example

[0625] For example, a dual-income household might want to set a schedule to finish cleaning the floors by 6 p.m. on weekdays. Based on past data, the server creates a schedule to run the cleaning robot while the residents are at the office, identifying the times when cleaning is needed. Once the user sets room priorities in the app, the robot starts cleaning from the living room and efficiently covers the entire house. After completion, the robot reports its cleaning status to the server, which is then used to improve future schedules.

[0626] Thus, the present invention makes it possible to automate and streamline household cleaning, thereby improving the quality of the living environment.

[0627] The following describes the processing flow.

[0628] Step 1:

[0629] The server collects data from terminals regarding the level of cleanliness in each room, furniture arrangement, and residents' behavior patterns. This includes sensor data and user input data.

[0630] Step 2:

[0631] The server uses an AI model to analyze the collected data and understand the level of dirtiness in each room and the residents' behavior patterns. This provides the basic information needed to determine cleaning priorities and timing.

[0632] Step 3:

[0633] The server generates an optimal cleaning schedule based on the analysis results. This schedule includes the time slots and specific routes for cleaning.

[0634] Step 4:

[0635] The server sends the generated cleaning schedule to the terminal and provides specific instructions to the cleaning robot.

[0636] Step 5:

[0637] The cleaning robot included in the device starts cleaning at the designated time according to the schedule received from the server. The robot uses its built-in sensors to detect dust and debris in real time, moving efficiently around the room and cleaning.

[0638] Step 6:

[0639] Users can monitor the cleaning progress through a smartphone app or voice assistant. They can also adjust cleaning schedules and settings via the app as needed.

[0640] Step 7:

[0641] After cleaning is complete, the device sends the cleaning results to the server. This information is used to improve the AI ​​model for the next cleaning. Based on the feedback, the server continues to optimize the schedule and cleaning procedure.

[0642] (Example 1)

[0643] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0644] In modern homes, daily cleaning tasks are often not performed efficiently amidst busy lifestyles, creating a need for improved living environments. Furthermore, conventional cleaning equipment and services struggle to provide flexible schedules based on the degree of dirtiness in rooms and the residents' daily routines, highlighting the need for automated systems.

[0645] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0646] In this invention, the server includes means for collecting dimensional data from information terminals and analyzing the state of each spatial domain and human behavior patterns; means for constructing a generative AI model that optimizes the operation schedule based on the data; and means for transmitting the optimized schedule to a control device and controlling the work equipment. This makes it possible to perform cleaning activities automatically and efficiently, improving the quality of the living environment.

[0647] An "information terminal" is a device that has the function of collecting data and transmitting it to a server, and includes smartphones, tablets, and personal computers.

[0648] "Dimensional data" refers to information collected to represent the state and environment of a spatial domain, as well as human activity, and is data that can capture physical arrangements and temporal changes.

[0649] "Spatial area" refers to the physical area that is the subject of cleaning, and includes the interior space of rooms and buildings.

[0650] "Human behavior patterns" refer to the tendencies of activities that residents engage in on a daily basis, and understanding these patterns can help optimize schedules.

[0651] "Operation schedule" refers to a plan that includes the date, time, and order in which cleaning tasks will be performed, and is optimized by a generative AI model.

[0652] A "generative AI model" is an algorithm that uses machine learning techniques to analyze data and create an optimal cleaning schedule.

[0653] A "control device" is an electronic device that operates work equipment based on instructions received from a server, and can operate multiple devices, including cleaning robots.

[0654] "Work equipment" refers to automated devices used to perform cleaning, and includes cleaning robots and other cleaning equipment.

[0655] This invention provides a system for automating and streamlining household cleaning. Specifically, it is achieved through a series of processes in which a server, terminals, and users work together.

[0656] Server operation:

[0657] The server collects dimensional data from information terminals. The data collected relates to the state of a spatial domain and human behavior patterns. This includes information such as the cleanliness of a room, the arrangement of furniture, and the daily routines of the inhabitants. Based on this data, the server preprocesses the data using Python's Pandas and NumPy, and builds a generative AI model using Scikit-learn and TensorFlow. This model is capable of generating an optimal schedule.

[0658] Device operation:

[0659] The system includes a cleaning robot and a smart speaker. The cleaning robot automatically begins cleaning according to the schedule sent from the server. Equipped with LIDAR sensors and cameras, the robot detects particles in the surrounding space in real time, enabling highly accurate cleaning. This allows for efficient and waste-free cleaning.

[0660] User actions:

[0661] Users can check the current cleaning schedule using a smartphone app or voice control device. They can also set cleaning conditions and priorities as needed through prompts. For example, they can give instructions such as, "Please set a new schedule that increases the frequency of cleaning the living room, taking past feedback into consideration, and does not clean at night."

[0662] This allows daily cleaning tasks to be performed automatically in accordance with the residents' daily routines, ensuring a consistently comfortable living environment and reducing the burden on users.

[0663] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0664] Step 1:

[0665] The server collects dimensional data through information terminals. It receives data such as dirt and furniture placement information from sensors in the spatial domain, as well as resident behavior patterns, as input. This data is stored in storage and prepared for later analysis.

[0666] Step 2:

[0667] The server preprocesses and analyzes the collected data. The input is the raw data obtained in step 1, which is cleaned up using Python's Pandas and NumPy to prepare it for analysis. The output is the analyzed dataset, which is used to optimize the operation schedule.

[0668] Step 3:

[0669] The server uses the analyzed data to build a generative AI model and generate an optimal operating schedule. The input is the dataset obtained in the previous step, and the model is trained using Scikit-learn or TensorFlow. The output is an optimized cleaning schedule, which determines the operating time and order of the cleaning robots.

[0670] Step 4:

[0671] The server sends the generated operation schedule to the terminal. The terminal, specifically the cleaning robot, receives this schedule and prepares to execute it. The output is instruction data for the cleaning robot, which then starts the automated cleaning process.

[0672] Step 5:

[0673] The terminal performs the actual cleaning based on the received schedule. The input is an operation instruction from the server, and the cleaning robot uses this to scan the site environment using LIDAR sensors and cameras and then performs the cleaning. The output is cleaning completion status data, which is fed back to the server.

[0674] Step 6:

[0675] The server receives feedback from the terminal indicating that cleaning is complete, and updates the AI ​​model. The input is feedback information from the terminal, which is saved as new data and used to retrain the model. This improves the model, which is then used to generate more efficient schedules for future cleanings.

[0676] Step 7:

[0677] The user uses a smartphone app to check the schedule and settings and make adjustments as needed. For example, they might use a prompt message such as, "Please increase the cleaning frequency in the living room and disable nighttime cleaning." The output is a new schedule and settings tailored to the user's needs.

[0678] (Application Example 1)

[0679] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0680] Household cleaning is time-consuming and often a burden for residents with busy lifestyles. Furthermore, creating an efficient cleaning schedule requires considering residents' lifestyles and the condition of their rooms, which is difficult to manage manually. Therefore, there is a need for a system that automatically performs optimal cleaning based on the level of dirt and the resident's lifestyle.

[0681] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0682] In this invention, the server includes means for collecting information to analyze the degree of dirtiness in each room and the behavioral patterns of the residents, means for generating an optimal cleaning plan based on the information, and means including an information terminal application for notifying the user of the optimal cleaning timing. As a result, residents will have an optimal cleaning schedule suggested and executed according to their lifestyle, enabling them to live an efficient and stress-free life.

[0683] "Room-specific cleanliness" is an indicator of the level of cleaning required for each room, and is determined by measuring the amount of dust and debris.

[0684] "Resident behavior patterns" refer to the tendencies of residents' movement and activities in their daily lives, and this allows us to understand which rooms are used at which times of the day.

[0685] An "information terminal" refers to a portable computing device such as a smartphone or tablet that sends and receives data via the internet or applications.

[0686] A "cleaning machine" is a robotic device that automatically cleans floors, using internal sensors to detect dirt and obstacles and performing cleaning efficiently.

[0687] An "artificial intelligence model" is an algorithm that performs pattern recognition and prediction based on a large amount of data, with the aim of optimizing cleaning plans and improving them through feedback.

[0688] "Cleaning timing" refers to the optimal time to begin cleaning, and is determined by considering the resident's schedule and the room's usage.

[0689] An "information terminal application" is software that runs on an information terminal and provides cleaning management and notification functions.

[0690] To realize this invention, the server collects and analyzes data on the degree of dirtiness in each room and the behavioral patterns of the residents. Smartphones and tablets are used as information terminals. Specifically, these information terminals transmit the status of each room to the server via sensors. Based on this, the server uses AI technology such as TensorFlow to generate an optimal cleaning plan. The generated plan is notified to the user through an application on the information terminal, allowing residents to adjust the cleaning timing according to their own schedule and lifestyle.

[0691] The cleaning machine, or automated cleaning robot, receives instructions from a server and cleans along an efficient route while detecting dirt in real time. Once cleaning is complete, feedback is sent back to the server, and the artificial intelligence model is improved. This optimizes the cleaning schedule for future cleaning sessions.

[0692] For example, if a user wants their living room to be cleaned intensively on weekend afternoons, they can specify this through the application. They can also input prompts for the generating AI model, such as, "Based on resident A's lifestyle, please suggest the optimal cleaning schedule for this week."

[0693] Thus, the system provided by the invention can automate and optimize cleaning according to the user's lifestyle, thereby improving the quality of the living environment.

[0694] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0695] Step 1:

[0696] Users perform initial setup via a terminal, entering cleaning requirements and room priorities. This input includes resident schedules and acceptable levels of dirtiness for each room. This data is sent to the server and registered as the initial dataset.

[0697] Step 2:

[0698] Based on the initial setup data, the server collects data to understand the level of dirtiness in each room and the residents' lifestyle patterns. It collects sensor information from cleaning machines and location information from smartphones and stores it in a database. The data collected in this process will be used later for the analysis of AI models.

[0699] Step 3:

[0700] The server analyzes the collected data using AI software such as TensorFlow and generates an optimal cleaning plan using a generative AI model. Input data includes the room's level of dirtiness and usage frequency, while output is specific cleaning timings and routes. This plan is then fed back to the user.

[0701] Step 4:

[0702] The terminal receives the cleaning plan sent from the server and notifies the user of the cleaning timing. The terminal's application controls the cleaning machine and starts cleaning according to the plan. At this step, the user can check the progress of the cleaning in real time via notifications.

[0703] Step 5:

[0704] Once cleaning is complete, the cleaning machine sends feedback on its performance and the degree of soiling to a server. The server uses this feedback to improve its artificial intelligence model, thereby increasing the accuracy of the next cleaning plan. This allows for the automatic suggestion of a more optimal plan, even with the same information.

[0705] This process allows users to achieve efficient cleaning without hassle and maintain a comfortable living environment.

[0706] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0707] This invention is a smart system for streamlining household cleaning tasks, and in particular, it combines an emotion engine that recognizes user emotions to provide a more personalized cleaning experience. Specifically, this system collects data based on the degree of dirtiness in a room and the behavioral patterns of the inhabitants to provide an optimal cleaning schedule, and also has the function of detecting the user's emotions and reflecting them in the cleaning process.

[0708] Server Functions

[0709] The server collects and analyzes data from terminals, and uses an AI model to understand the level of dirtiness in each room and the residents' behavior patterns. Based on the results of this analysis, it generates an optimal cleaning schedule and sends it to the various terminals. Furthermore, it analyzes the user's voice input through an emotion engine and adjusts the schedule and cleaning methods based on that emotion.

[0710] Device functions

[0711] The devices include a robotic vacuum cleaner and a smart speaker. Through an emotion engine, the devices analyze the user's voice to understand their emotions and transmit this information to a server. The robotic vacuum cleaner cleans according to the received schedule, detecting the level of dirt in the room in real time and cleaning along an optimized route. It also sends feedback to the server to help improve future schedules.

[0712] User actions

[0713] Users can access the system through a smartphone app or voice assistant. Users can adjust the cleaning pace and timing based on their emotional state; for example, they can instruct the system to increase cleaning frequency during stressful periods. The system adapts the cleaning plan based on this input, providing a more satisfying cleaning experience.

[0714] Specific example

[0715] For example, consider a dual-income household where the user wants to clean efficiently when they come home tired. When the user expresses their emotions through voice input such as "I'm tired," the emotion engine recognizes this, and the server adjusts the cleaning robot's operation to reduce stress. For example, it might clean in silent mode, providing an environment that doesn't cause stress to the user. After completion, the cleaning robot sends feedback to the server, which is then used to improve future situations.

[0716] Thus, this invention makes it possible to achieve personalized cleaning that is attentive to the user's emotions and improve the living environment within the home.

[0717] The following describes the processing flow.

[0718] Step 1:

[0719] Users initiate interaction with the system by expressing emotions through a smartphone app or voice input. This can be done simply by reporting their physical condition or mood through voice.

[0720] Step 2:

[0721] The emotion engine built into the device analyzes the user's voice input and identifies emotions from that voice. For example, it detects voice tone and specific keywords to extract emotions such as "tired" or "want to relax."

[0722] Step 3:

[0723] The device sends the analysis results to the server and requests that it generate cleaning requirements that have been adjusted based on the user's current sentiment data.

[0724] Step 4:

[0725] The server generates a new cleaning schedule based on received emotional data, the current level of dirtiness in the room, and the residents' behavior patterns. For example, if the user is tired, it may increase the cleaning frequency or recommend cleaning in silent mode.

[0726] Step 5:

[0727] The server sends the generated cleaning schedule and any emotionally-based adjustments to the device's cleaning robot.

[0728] Step 6:

[0729] The cleaning robot starts cleaning in the specified mode according to a schedule received from the server. In practice, the robot moves around the room, updating information about the room in real time using sensors to clean efficiently.

[0730] Step 7:

[0731] After cleaning is complete, the device sends the cleaning results and user feedback back to the server. This allows the server to use the information to improve the AI ​​model.

[0732] Step 8:

[0733] The user receives a cleaning completion notification via an app or similar means, confirming that the cleaning was performed appropriately according to their emotional needs. The system evaluates whether it performed as expected and makes further adjustments as needed.

[0734] Through these steps, the system provides a flexible and personalized cleaning process that takes user emotions into consideration.

[0735] (Example 2)

[0736] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0737] In modern homes, cleaning is an important and time-consuming task, but conventional automated cleaning systems often lack efficiency because they fail to adequately consider the lifestyles and emotional states of residents. Furthermore, the operation of cleaning robots can sometimes increase residents' stress levels. Therefore, there is a need for cleaning systems that are adapted to the behavior and emotions of residents.

[0738] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0739] In this invention, the server includes means for collecting information to analyze the pollution status of each environment and the behavioral characteristics of the residents, means for creating an optimal cleaning schedule based on the information, and means for analyzing emotions from the user's voice instructions and adjusting the cleaning process and schedule accordingly. This makes it possible to provide residents with an individually optimized cleaning experience and improve the quality of the living environment.

[0740] "Environment" refers to the individual rooms or areas that are to be cleaned, and is a target area related to its level of contamination and the behavioral characteristics of the residents.

[0741] "Contamination level" refers to the degree of accumulation of impurities such as dust and debris within the environment being cleaned.

[0742] "Resident behavioral characteristics" refer to the activity patterns and lifestyle habits of residents within a specific environment, and are used to optimize cleaning schedules.

[0743] "Means of collecting information" refers to methods of acquiring data on environmental pollution levels and the behavioral characteristics of residents using sensors and cameras.

[0744] A "cleaning schedule" refers to the operating schedule of cleaning equipment created based on acquired information, and represents a plan for effectively cleaning the environment.

[0745] "Methods for analyzing emotions from voice commands" refers to technologies that use algorithms and models to analyze a user's voice input and estimate the user's emotions and stress levels.

[0746] "Means of adjusting the cleaning process and schedule" refers to methods for appropriately changing the operating modes and schedules of cleaning equipment based on analyzed emotional data.

[0747] This invention is a system for efficiently performing cleaning tasks in a residential environment, combining information technology and emotion analysis technology to provide a more personalized cleaning experience.

[0748] The server functions as a device that receives and analyzes data collected from connected devices in real time. This data includes the level of contamination in rooms obtained from sensors and the behavioral characteristics of residents captured by cameras. A generative AI model is used for analysis, automatically generating an optimal cleaning schedule from each data point. In addition, the server utilizes a voice analysis engine to analyze emotions from the user's voice obtained through the device, and has the ability to adjust the cleaning process and schedule as needed.

[0749] The system includes a cleaning robot and a smart speaker for receiving user voice commands. The cleaning robot cleans rooms efficiently according to a cleaning schedule received from the server. During this process, the robot detects environmental contamination in real time and sends this data back to the server as feedback. This data is then used to optimize the next cleaning plan.

[0750] Users can access the system through a smartphone application or voice assistant and easily give cleaning instructions tailored to their emotional state. In particular, when stressed, entering prompts such as "I'm tired" into a smart speaker will instruct the cleaning robot to operate in quiet mode, thus reducing stress for residents.

[0751] This invention overcomes the limitations of conventional automatic cleaning devices, enabling customization based on lifestyle habits and individual emotional states. This significantly improves the quality of the living environment.

[0752] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0753] Step 1:

[0754] The terminal collects data on environmental pollution levels and resident behavioral characteristics via sensors and cameras installed in the room. The collected data includes real-time information such as temperature, humidity, and camera footage. This input data is used to assess pollution levels and analyze resident behavior, and is then transmitted to a server.

[0755] Step 2:

[0756] The server inputs environmental data received from the terminal into a generating AI model, which then analyzes the data. The AI ​​model utilizes machine learning algorithms to evaluate the degree of contamination in each room and the behavioral patterns of the residents. This analysis identifies the time and methods required for cleaning and generates an optimal cleaning schedule.

[0757] Step 3:

[0758] The server analyzes the emotional information entered by the user via voice through a voice analysis engine. The voice data is used to infer the user's emotional state. The emotional analysis results obtained from the prompt text are compared with the current cleaning plan, and output is generated to readjust the cleaning schedule and cleaning methods as needed.

[0759] Step 4:

[0760] Upon receiving instructions from the server, the cleaning robot at the terminal begins operation according to the predetermined cleaning schedule. The robot uses sensors to detect contamination in the room in real time and cleans along an optimized route. It monitors the progress of cleaning and the condition of the room, and dynamically adjusts the route and cleaning mode.

[0761] Step 5:

[0762] After cleaning is complete, the terminal reports completion to the server, along with feedback on the cleaning results and the latest environmental data. The server receives this feedback and processes the data to improve the accuracy of the next cleaning plan. It also stores the data as training data for the generated AI model, enabling continuous performance improvement.

[0763] This series of processes makes it possible to provide residents with an efficient and comfortable cleaning experience.

[0764] (Application Example 2)

[0765] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0766] In recent years, household cleaning machines have been required to be highly efficient and comfortable to use, but conventional technology has struggled to flexibly adapt to the user's lifestyle and emotional state. In particular, a cleaning system that can appropriately respond to specific emotional states, such as when the user is fatigued or stressed, has yet to be established. Therefore, there is a need to develop a new cleaning system that can recognize emotions and appropriately adjust the cleaning process according to the user's situation.

[0767] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0768] In this invention, the server includes means for collecting data to analyze the degree of dirtiness in each room and the behavioral patterns of the residents, means for generating an optimal cleaning schedule, and means for detecting the emotional state of the residents and adjusting the operation of the cleaning machine based on that. This makes it possible to optimize the operation of the cleaning machine according to the emotional state of the user and provide a comfortable living environment.

[0769] "Room-specific cleanliness" refers to the condition or level of cleanliness in each individual room, and is measured based on the frequency and type of dirt that occurs.

[0770] "Resident behavior patterns" refer to patterns of behavior and actions of people living within a residence, and include information about their daily activities and movement tendencies.

[0771] An "optimal cleaning schedule" is a set of times and sequences planned to perform cleaning tasks with maximum efficiency, and is formulated based on the degree of soiling and the user's daily schedule.

[0772] "Terminal" is a general term for electronic devices related to this system, and includes cleaning machines and devices that handle information display and communication.

[0773] A "cleaning machine" is a mechanical device designed for cleaning indoor spaces, with the purpose of removing dirt and purifying the environment.

[0774] "Resident's emotional state" refers to the user's psychological or emotional condition and is determined based on information from their voice and behavior.

[0775] An "artificial intelligence model" is a mathematical or computational model designed to perform a specific task based on data, and possesses the ability to learn from experience.

[0776] The system that realizes this application example is constructed as follows: This system is an advanced home cleaning system that provides a personalized cleaning experience based on the user's emotions. This system consists of a server, terminals (cleaning machines and information display devices), and a user interface.

[0777] The server is the core of the system, responsible for all data analysis and schedule generation. It utilizes data collected from various sensors and input devices to analyze the level of dirtiness in each room and the residents' behavioral patterns. This data is then analyzed by an AI model to generate an optimal cleaning schedule. Furthermore, the server has the ability to detect the user's emotional state from their voice and behavior using an emotion engine, and adjust the cleaning machine's operation accordingly.

[0778] The device includes a vacuum cleaner and a voice assistance device. The vacuum cleaner uses robotic cleaning technology to clean the room and transmits status data to the server in real time. The voice assistance device analyzes the user's voice and automatically transmits their emotional state to the server. This allows the cleaning mode to be adjusted according to the user's emotional state.

[0779] Users can access and operate the system via a mobile device or voice assistance system. Users can set the cleaning pace and timing by directly inputting their emotional state or giving voice instructions. Through this, the server adaptively trains its AI model to improve the accuracy of future cleaning plans.

[0780] For example, if a user gives a voice command saying, "I want it to be quiet today," the server will interpret the voice as indicating the user is seeking silence and operate the vacuum cleaner in silent mode. Furthermore, if the user is feeling stressed, a relaxation mode will be suggested.

[0781] For this system to function, the following prompt can be used: "Design an app that detects the user's emotions and adjusts the operation of the vacuum cleaner accordingly."

[0782] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0783] Step 1:

[0784] The server collects sensor data to measure residents' behavior patterns and the level of dirtiness in each room. The input is environmental data obtained from various sensors, and the output is a dataset organized for necessary analysis. The server integrates this data and prepares it as foundational data for quantifying behavioral patterns.

[0785] Step 2:

[0786] The server uses an emotion engine to analyze the user's voice input and detect their emotional state. The input is the user's voice data, and the output is an emotion label. The voice waveform is analyzed via a speech processing algorithm, and the emotion is classified based on an emotion model. This result is then used in the next step.

[0787] Step 3:

[0788] The server generates an optimal cleaning schedule based on the behavioral pattern data obtained in Step 1 and the emotional state labels obtained in Step 2. The input is the behavioral patterns and emotional labels obtained in the previous step, and the output is a specific cleaning schedule. An AI model is used to analyze this data and create individually customized schedules.

[0789] Step 4:

[0790] The vacuum cleaner in the terminal starts cleaning according to the cleaning schedule sent from the server. The input is the schedule data received from the server, and the output is feedback data after cleaning is complete. The vacuum cleaner proceeds with cleaning while detecting the room conditions with its built-in sensors, and when cleaning is complete, it sends the results to the server.

[0791] Step 5:

[0792] Users operate the system and send feedback via mobile devices or voice assistance systems. Inputs are the user's experiences and requests, while outputs are data used to train the next AI model. User feedback is reflected on the server and used to improve the next schedule generation.

[0793] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0794] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0795] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0796] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0797] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0798] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0799] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0800] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0801] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0802] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0803] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0804] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0805] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0806] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0807] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0808] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0809] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0810] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0811] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0812] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0813] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0814] The following is further disclosed regarding the embodiments described above.

[0815] (Claim 1)

[0816] A means of collecting data to analyze the degree of cleanliness in each room and the behavioral patterns of the residents,

[0817] A means for generating an optimal cleaning schedule based on the aforementioned data,

[0818] A means for transmitting the aforementioned schedule to a terminal and controlling the cleaning robot,

[0819] A means including a cleaning robot that detects dirt in a room in real time and cleans it,

[0820] A means to receive feedback after cleaning is complete and improve the AI ​​model,

[0821] A system that includes this.

[0822] (Claim 2)

[0823] The system according to claim 1, which initializes cleaning requirements based on input from residents.

[0824] (Claim 3)

[0825] The system according to claim 1, wherein the system is operated and feedback is sent using a smartphone app or voice assistant.

[0826] "Example 1"

[0827] (Claim 1)

[0828] A means of collecting dimensional data from information terminals and analyzing the state of each spatial domain and human behavior patterns,

[0829] A means for constructing a generative AI model that optimizes the operation schedule based on the aforementioned data,

[0830] A means for transmitting the optimized schedule to a control device and controlling the work equipment,

[0831] A means including work equipment that detects particles in a space in real time and performs cleaning,

[0832] A means of receiving evaluation data after cleaning is completed and updating the generated AI model,

[0833] A system that includes this.

[0834] (Claim 2)

[0835] The system according to claim 1, which uses an information terminal to set initial conditions for cleaning based on human input.

[0836] (Claim 3)

[0837] The system according to claim 1, wherein communication equipment or voice processing equipment is used to operate the system and transmit evaluation data.

[0838] "Application Example 1"

[0839] (Claim 1)

[0840] A means of collecting information to analyze the degree of cleanliness in each room and the behavioral patterns of the residents,

[0841] A means for generating an optimal cleaning plan based on the aforementioned information,

[0842] A means for transmitting the aforementioned plan to an information terminal and controlling the cleaning machine,

[0843] A means including a cleaning machine that detects dirt in a room in real time and performs cleaning,

[0844] A means to receive feedback after cleaning is complete and to improve the artificial intelligence model,

[0845] A means including an application for an information terminal to notify the user of the optimal cleaning timing,

[0846] A system that includes this.

[0847] (Claim 2)

[0848] The system according to claim 1, which sets initial cleaning requirements based on input from residents and optimizes cleaning according to the residents' lifestyles.

[0849] (Claim 3)

[0850] The system according to claim 1, which allows residents to operate the system and send feedback using an information terminal application or voice control function, and to instruct the next cleaning.

[0851] "Example 2 of combining an emotion engine"

[0852] (Claim 1)

[0853] A means of collecting information to analyze the pollution status and behavioral characteristics of residents in each environment,

[0854] Based on the aforementioned information, a means for creating an optimal cleaning schedule,

[0855] A means for transmitting the aforementioned timetable to the device and controlling the cleaning equipment,

[0856] Means including cleaning equipment that dynamically detects and cleans environmental contamination,

[0857] After cleaning is completed, an evaluation is received to update the learning model, and a means is provided to improve the learning model.

[0858] A means of analyzing emotions from the user's voice commands and adjusting the cleaning process and schedule accordingly.

[0859] A system that includes this.

[0860] (Claim 2)

[0861] The system according to claim 1, which sets the frequency and method of cleaning based on the emotional state of the residents.

[0862] (Claim 3)

[0863] The system according to claim 1, which transmits system operations and evaluations using a mobile information terminal application or voice assistance device.

[0864] "Application example 2 when combining with an emotional engine"

[0865] (Claim 1)

[0866] A means of collecting data to analyze the degree of cleanliness in each room and the behavioral patterns of the residents,

[0867] A means for generating an optimal cleaning schedule based on the aforementioned data,

[0868] A means for transmitting the aforementioned schedule to a terminal and controlling the cleaning machine,

[0869] A means including a cleaning machine that detects dirt in a room in real time and performs cleaning,

[0870] A means for detecting the emotional state of the residents and adjusting the operation of the vacuum cleaner based on that,

[0871] A means to receive feedback after cleaning is complete and to improve the artificial intelligence model,

[0872] A system that includes this.

[0873] (Claim 2)

[0874] The system according to claim 1, which initializes cleaning requirements based on input from residents.

[0875] (Claim 3)

[0876] The system according to claim 1, wherein the system is operated and feedback is transmitted using a mobile information terminal or voice support system. [Explanation of symbols]

[0877] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting information to analyze the degree of cleanliness in each room and the behavioral patterns of the residents, A means for generating an optimal cleaning plan based on the aforementioned information, A means for transmitting the aforementioned plan to an information terminal and controlling the cleaning machine, A means including a cleaning machine that detects dirt in a room in real time and performs cleaning, A means to receive feedback after cleaning is complete and to improve the artificial intelligence model, A means including an application for an information terminal to notify the user of the optimal cleaning timing, A system that includes this.

2. The system according to claim 1, which sets initial cleaning requirements based on input from residents and optimizes cleaning according to the residents' lifestyles.

3. The system according to claim 1, which allows residents to operate the system and send feedback using an information terminal application or voice control function, and to instruct the next cleaning.

Images