System

Abstract

A system is provided.SOLUTION: A system, comprising: means for detecting a degree of dirt of clothes by a sensor mounted on a washing machine; means for analyzing the detected degree of dirt by using a AI model generated by a server to determine whether the clothes need to be washed; means for notifying a panel of the washing machine and a user device of a result that the clothes do not need to be washed; and means for allowing the user to select whether to perform or stop washing.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern households, laundry is a major household chore that requires time and effort. Over-washing clothes, in particular, can waste water and increase the wear and tear on clothes, resulting in increased water bills, wasted resources, and negative impacts on the environment. Furthermore, the traditional method of visually checking the soiling of clothes to reduce unnecessary laundry is burdensome for users and inefficient. It is desirable to solve this problem, reduce the burden of household chores, and conserve water resources. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means: A means for detecting the degree of soiling of clothes using a sensor installed in the washing machine, and a means for analyzing the detected data using an AI model generated by a server to determine whether the clothes need to be washed. Furthermore, a means for notifying the washing machine panel and the user's device that the clothes do not need to be washed is provided. Based on this information, the user can choose whether to perform or cancel the wash. This makes it possible to reduce unnecessary laundry, lighten the burden of housework, and conserve water resources.

[0006] A "sensor" is a device installed in a washing machine that detects the degree and type of dirt on laundry.

[0007] The "AI model" is an artificial intelligence algorithm generated on the server that analyzes acquired sensor data and determines whether laundry needs to be washed.

[0008] A "server" is a central computing device that stores AI models, receives and analyzes data from sensors, and transmits the results.

[0009] A "terminal" is a device that has a user interface, such as a washing machine control panel or a user's smartphone, and receives and displays notification information from the server.

[0010] A "user" is a person who operates the washing machine, puts in laundry, and decides whether to start or stop washing based on the analysis results.

[0011] The "panel" is a display device that is installed on the washing machine and that displays information to the user and accepts operation inputs.

[0012] "Washing necessity" is an analysis result that indicates whether the laundry actually needs to be washed.

[0013] "Notification" refers to information displayed on a panel or terminal to inform the user of the analysis results.

[0014] "Analysis results" are information about the need for laundry that is generated after the server analyzes the data obtained from the sensors based on an AI model. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention is a system that automatically detects the degree of soiling of clothes in a washing machine and performs washing only when necessary, thereby reducing unnecessary washing. This system mainly consists of the following elements.

[0037] 1. Dirt detection by sensor

[0038] The sensors installed in the washing machine detect the degree of soiling, color, and type of laundry in real time. These sensors can be optical or chemical sensors, for example. When a user puts laundry into the washing machine, the sensors are automatically activated and start collecting data.

[0039] 2. Data transmission

[0040] The sensed data is collected by a terminal inside the washing machine and then transmitted over the internet to a server, which uses a communications protocol to package and efficiently transmit the data.

[0041] 3. Analysis using AI models

[0042] The server launches a generative AI model to analyze the received data. This AI model learns from past data and can accurately determine whether clothes need to be washed. The server uses this model to generate data analysis results. For example, it outputs classification results such as "lightly soiled," "moderately soiled," and "heavily soiled."

[0043] 4. Notification of Results

[0044] The analysis results are sent to the washing machine terminal and the user's smartphone app. The terminal displays messages such as "No washing necessary" or "Washing required" on the washing machine panel. The user can check the detailed analysis results through the smartphone app.

[0045] 5. User Choice

[0046] Based on the analysis results, the user can choose whether to run or cancel the wash. For example, if the user receives a "no washing required" notification, the wash will not run unless the user presses the "start washing" button on the smartphone app or the washing machine's control panel. On the other hand, if the user presses the "start washing" button, the normal wash cycle will begin.

[0047] Specific examples

[0048] Case 1: Light soiling

[0049] The user puts laundry into the washing machine.

[0050] The device activates a sensor to detect the degree of dirt on the clothes.

[0051] The device transmits the detected data to the server.

[0052] The server analyzes the data using the generated AI model and determines that the item is "slightly soiled and no washing is required."

[0053] The server notifies the device and the user's smartphone app of the analysis results.

[0054] The device will display "No washing required" on its panel, and a notification will also be sent to the user's smartphone app.

[0055] The user checks the notification on the smartphone app and chooses to cancel the wash.

[0056] The device will stop the wash and prompt the user to remove the clothes.

[0057] Case 2: Moderate to severe soiling

[0058] The user puts laundry into the washing machine.

[0059] The device activates a sensor to detect the degree of dirt on the clothes.

[0060] The device transmits the detected data to the server.

[0061] The server analyzes the data using the generated AI model and determines that the garment is "moderately soiled and requires washing."

[0062] The server notifies the device and the user's smartphone app of the analysis results.

[0063] The device will display "Washing required" on the panel, and a notification will also be sent to the user's smartphone app.

[0064] The user checks the notification on the smartphone app and selects to perform the laundry.

[0065] The device will begin a normal wash cycle.

[0066] As described above, the system of the present invention can improve the efficiency of laundry through the interaction between sensors, servers, AI models, terminals, and users. This system also reduces unnecessary laundry, contributes to saving water resources, and reduces household water bills, making it environmentally friendly.

[0067] The processing flow will be explained below.

[0068] Step 1:

[0069] The user loads laundry into the washing machine and closes the door, ready for the wash cycle to begin.

[0070] Step 2:

[0071] The device activates the sensors inside the washing machine, which detect the soiling level, color, and type of clothes in real time and collect data.

[0072] Step 3:

[0073] The device temporarily stores the data acquired from the sensor and then prepares it to be sent to a server via the Internet.

[0074] Step 4:

[0075] The device then sends the packaged data over the internet to a server, which includes details such as the degree of dirt, color, and type.

[0076] Step 5:

[0077] The server then launches a generative AI model to analyze the received data. Based on the learned data, the AI ​​model analyzes the degree of soiling of the clothes and determines whether they need to be washed.

[0078] Step 6:

[0079] The server generates the analysis results, which are displayed as classifications such as "lightly soiled," "moderately soiled," and "heavily soiled."

[0080] Step 7:

[0081] The analysis results generated by the server are sent to the device and the user's smartphone app.

[0082] Step 8:

[0083] The device displays the analysis results on the washing machine's control panel, such as a message like "No washing necessary" or "Washing required."

[0084] Step 9:

[0085] The user opens the smartphone app and checks the analysis results, which are displayed in detail.

[0086] Step 10:

[0087] The user can choose whether to start or stop the wash by pressing the "Start Wash" or "Cancel" button on the smartphone app or the washing machine's control panel.

[0088] Step 11:

[0089] The device acts based on the user's selection. If the user selects "Start Wash," the device starts a normal wash cycle. If the user selects "Cancel," the device stops the wash and prompts the user to remove the clothes.

[0090] Step 12:

[0091] The user removes the clothes. If it is determined that washing is not necessary, the user removes the clothes and stores them. If washing is performed, the user removes the clothes after the washing is completed.

[0092] These are the specific steps in the washing machine's process, from detecting dirt to starting or canceling the wash. This system reduces unnecessary washing, lightens the burden of housework, and conserves water resources.

[0093] Example 1

[0094] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0095] Conventional washing machines have difficulty accurately detecting the degree of soiling of laundry, resulting in unnecessary washing. This results in wasted water and electricity consumption, placing a heavy burden on the environment. Another issue is that users must check the degree of soiling of their clothes and select the appropriate washing method every time they wash, which is time-consuming.

[0096] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0097] In this invention, the server includes: means for detecting the degree of soiling of clothes using a sensor installed in the washing machine; means for transmitting the detected data to the server via the Internet via a terminal; means for analyzing the received data using an AI model generated by the server and determining whether the clothes need to be washed; means for notifying the user of the analysis results via the washing machine's operation panel and the user's terminal; and means for the user to select whether to perform or cancel the wash based on the analysis results. This reduces unnecessary washing and prevents the wasteful consumption of water resources and electricity. It also allows users to wash clothes efficiently without hassle, contributing to environmental protection.

[0098] "Sensor" refers to a device installed in a washing machine that detects the degree of soiling, color, type, etc. of clothes. This includes optical sensors and chemical sensors.

[0099] A "terminal" is a device that is placed inside the washing machine and is responsible for collecting data from the sensors and sending it to the server.

[0100] A "server" is a computer system that receives and analyzes data sent from a terminal via the Internet.

[0101] The "AI model" is a machine learning model that runs on a server and is used to analyze received data and determine whether clothes need to be washed.

[0102] The "operation panel" is an interface device that is provided in the washing machine and that displays the analysis results to the user.

[0103] The "means of notifying the terminal" is a communication means for transmitting the analysis results from the server to the terminal. It uses the Internet Protocol.

[0104] A "user's device" is a communication-enabled device such as a smartphone or tablet held by a user, and is responsible for receiving and displaying notifications from the server.

[0105] "Means by which the user can choose whether to start or stop washing based on the analysis results" refers to an interface that allows the user to check the analysis results through the operation panel or the user's terminal and choose whether to start or stop washing.

[0106] The present invention is a system that automatically detects the degree of soiling of clothes in a washing machine and performs washing as necessary, thereby reducing unnecessary washing. This system is mainly composed of sensors, terminals, a server, a generative AI model, and a user smartphone app. Specific embodiments of the system are described below.

[0107] 1. Sensor activation and data collection

[0108] When a user loads laundry into the washing machine, the device inside the washing machine automatically activates sensors as soon as the lid is closed. The sensors use optical and chemical sensors to detect data such as the soiling level, color, and type of clothes. Optical sensors scan the surface of the clothes and measure the intensity and wavelength of reflected light. Chemical sensors detect specific pollutants, such as volatile organic compounds (VOCs) and specific stains. The detected data is collected and stored locally on the device.

[0109] 2. Data transmission

[0110] The device packages the collected data and sends it to a server via the Internet. The data is packaged in a specified format (for example, JSON format) and includes information such as the degree of dirt, color, and type. HTTP / HTTPS is generally used as the transmission protocol. For example, the endpoint URL is "https: / / example.com / api / wash-data".

[0111] 3. Data Analysis

[0112] The server launches a generative AI model to analyze the received data. The AI ​​model is implemented using frameworks such as TensorFlow and PyTorch. This AI model learns from past data and can accurately determine whether clothes need to be washed. The generative AI model analyzes the data and outputs classification results, such as "lightly soiled," "moderately soiled," or "heavily soiled." The analysis results are stored on the server.

[0113] 4. Result notification

[0114] The server notifies the device inside the washing machine and the user's smartphone app of the analysis results. Notifications are sent via push notifications and REST API calls. The smartphone app receives a notification such as "Lightly soiled, no need to wash," and the washing machine's control panel displays a message based on the analysis results.

[0115] 5. User Choice

[0116] Based on the analysis results, the user can choose whether to run or cancel the wash. They can receive a notification via the smartphone app and check the details. The user decides whether to run the wash by choosing whether to press the "Start Wash" button. If "washing is not necessary," the wash will not be run if the user does nothing. On the other hand, if it is determined that "washing is necessary," the device will start a normal wash cycle when the user presses the "Start Wash" button. The user will be notified by a voice message or other means when the cycle starts.

[0117] Specific examples

[0118] Case 1: Light soiling

[0119] The user puts laundry into the washing machine.

[0120] The device activates a sensor to detect the degree of dirt on the clothes.

[0121] The device transmits the detected data to the server.

[0122] The server analyzes the data using the generated AI model and determines that the item is "slightly soiled and no washing is required."

[0123] The server notifies the device and the user's smartphone app of the analysis results.

[0124] The device will display "No washing required" on the operation panel, and a notification will also be sent to the user's smartphone app.

[0125] The user checks the notification on the smartphone app and chooses to cancel the wash.

[0126] The device will stop the wash and prompt the user to remove the clothes.

[0127] Case 2: Moderate to severe soiling

[0128] The user puts laundry into the washing machine.

[0129] The device activates a sensor to detect the degree of dirt on the clothes.

[0130] The device transmits the detected data to the server.

[0131] The server analyzes the data using the generated AI model and determines that the garment is "moderately soiled and requires washing."

[0132] The server notifies the device and the user's smartphone app of the analysis results.

[0133] The device will display "Washing required" on the operation panel, and a notification will also be sent to the user's smartphone app.

[0134] The user checks the notification on the smartphone app and selects to perform the laundry.

[0135] The device will begin a normal wash cycle.

[0136] This reduces unnecessary washing and prevents the wasteful consumption of water and electricity. It also allows users to wash their clothes efficiently and hassle-free, contributing to environmental protection.

[0137] Prompt Sentence Examples

[0138] "Please create a prompt for a system that detects the degree of soiling of laundry and performs the washing if necessary."

[0139] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0140] Step 1:

[0141] Sensor activation and data collection

[0142] A user loads laundry into the washing machine and closes the lid. This action automatically activates sensors on a device inside the washing machine. The sensors use optical and chemical sensors to detect the soiling level of the clothes. Specifically, optical sensors scan the surface of the clothes and measure the intensity and wavelength of reflected light. Chemical sensors detect volatile organic compounds (VOCs) and specific stains. The collected data is then sent directly to the device and stored locally.

[0143] Input: Load laundry and close lid

[0144] Output: Detection data such as dirt level, color, type, etc.

[0145] Step 2:

[0146] Sending data

[0147] The device packages the data collected from the sensor. The data is packaged in a specified format such as JSON. For example, the data may include information such as the degree of dirt, color, and type. The device then transmits the data to a server via the Internet. HTTP / HTTPS is generally used as the transmission protocol.

[0148] Input: Detection data from sensors

[0149] Output: Packaged data sent to the server

[0150] Step 3:

[0151] Data analysis

[0152] The server launches a generative AI model to analyze the received data. The server then analyzes the data using an AI model implemented using frameworks such as TensorFlow or PyTorch. The generative AI model evaluates the degree of soiling of the clothes based on the received data and outputs a classification result such as "lightly soiled," "moderately soiled," or "heavily soiled." The analysis results are stored on the server.

[0153] Input: Data package sent from the terminal

[0154] Output: Analysis results (classified data: mild, mild, moderate, severe)

[0155] Step 4:

[0156] Result notification

[0157] The server notifies the washing machine device and the user's smartphone app of the analysis results. Notification methods include push notifications and REST API calls. This allows the user's smartphone app to receive specific notifications, such as "Lightly soiled, no need to wash." The results are also displayed on the washing machine's control panel.

[0158] Input: Analysis results of the AI ​​model

[0159] Output: Notifications to the device and the user's smartphone app

[0160] Step 5:

[0161] User Selection

[0162] The user checks the analysis results through the smartphone app or the washing machine's operation panel, and can then choose whether to start or stop the wash. If the notification is "lightly soiled, no washing necessary," the wash will not be carried out unless the user presses the "start washing" button. On the other hand, if the notification is "moderately soiled, washing required," the wash will begin if the user presses the "start washing" button on the smartphone app or operation panel.

[0163] Input: User selection via smartphone app or operation panel

[0164] Output: Decision to do or not do laundry

[0165] Step 6:

[0166] Starting the wash cycle

[0167] If the user decides to wash the clothes, the device will start the washing machine's normal wash cycle, and the user will be notified when the wash cycle starts, for example, through a voice message or a display on the operation panel.

[0168] Input: User's laundry run selection

[0169] Output: Start and run a normal wash cycle

[0170] (Application example 1)

[0171] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0172] The present invention relates to a system for reducing unnecessary laundry and false alarms in homes and other environments. Specifically, the system detects the degree of soiling of clothes and abnormalities in the home with high accuracy, and performs laundry and issues alarms only when necessary, thereby preventing waste of resources and false alarms.

[0173] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0174] In this invention, the server includes a means for detecting the degree of soiling of clothes using a sensor installed in the washing machine, a means for analyzing the detected data using an AI model generated by the server and determining whether the clothes need to be washed, and a means for detecting abnormalities in the home using cameras and chemical sensors installed in the home. This enables the system to automatically detect the degree of soiling of clothes and abnormal conditions in the home and notify the user, thereby reducing unnecessary washing and warnings.

[0175] A "sensor" is a device that senses physical or chemical quantities, converts them into electrical signals, and outputs them.

[0176] An "AI model" is an artificial intelligence algorithm designed to learn from past data and analyze new data to make decisions.

[0177] A "server" refers to equipment or software used to store, process, and analyze data on a network.

[0178] A "washing machine" is a mechanical device for washing clothes.

[0179] A "display device" is a device for visually displaying information, and includes, for example, a display or a panel.

[0180] An "information communication device" is a device for sending and receiving data and information over a network, and includes, for example, smartphones and tablets.

[0181] A "camera" is a device that takes pictures and videos and records them as data.

[0182] A "chemical sensor" is a device that detects specific chemical substances and outputs their presence and concentration as an electrical signal.

[0183] "Abnormal" refers to an event or situation that is different from normal, such as suspicious movements or sounds, or gas leaks.

[0184] An "alarm" refers to a system or function that uses sound, light, or other means to draw attention when an abnormality occurs.

[0185] This invention is a system for efficient home laundry and security management. The system consists of a washing machine, sensors, an AI model, a display device, an information and communication device, a camera, and a chemical sensor.

[0186] First, sensors installed in the washing machine detect the degree of soiling of the clothes put in. Specifically, optical sensors or chemical sensors are used. The data collected by the sensors is temporarily stored on a terminal inside the washing machine and then sent to a server via the Internet.

[0187] The server launches a generative AI model to analyze the received data. This AI model learns from past data and accurately determines whether clothes need to be washed. For example, it can classify clothes into categories such as "lightly soiled," "moderately soiled," and "heavily soiled." The analysis results are notified to the washing machine's display and the user's information and communication device.

[0188] Based on these notifications, the user can choose to either run or cancel the laundry. If the analysis results indicate that washing is not necessary, the user can cancel the laundry and, if necessary, run the laundry manually.

[0189] In addition, cameras and chemical sensors installed in the home will detect abnormalities, such as suspicious movements, sounds, and gas leaks. If these sensors detect an abnormality, the data will also be sent to the server. The generated AI model will analyze the severity of the abnormality and output a classification result such as "mild abnormality," "moderate abnormality," or "severe abnormality." This result will also be notified to the display device in the home and the user's information and communication device.

[0190] The user can choose to issue or cancel an alert based on this notification. If the analysis result is judged to be a "serious abnormality," the user is advised to take immediate action to address the issue.

[0191] For example, if a home camera detects abnormal activity, the data is sent to a server and analyzed by an AI model. If the analysis results in a "severe abnormality," a notification is sent to the user's smartphone stating, "Suspicious activity has been detected!" The user can then choose to activate or cancel the alarm via the smartphone app.

[0192] An example of a specific prompt is:

[0193] An abnormality was detected on a home camera at 16:45. After analyzing the video, it was determined that a "serious abnormality" had occurred. Please check immediately.

[0194] There are messages like this.

[0195] The hardware used includes washing machines, cameras, sensors, displays, and information and communication devices. The software includes OpenCV, Requests, and a generative AI model. The generative AI model is implemented using Python's TensorFlow and PyTorch.

[0196] With this configuration, the present invention can reduce wasted laundry and the occurrence of false alarms, contributing to improved efficiency and security within the home.

[0197] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0198] Step 1:

[0199] The device activates sensors installed in the washing machine to detect the degree of soiling of the clothes placed in it. Optical and chemical sensors are used for this purpose. The input is the physical data of the clothes, and the output is numerical data on the degree of soiling.

[0200] Step 2:

[0201] The device collects data on the level of dirt detected by the sensor and sends it to a server via the Internet. The input is numerical data on the level of dirt, and the output is packetized data.

[0202] Step 3:

[0203] The server launches a generative AI model to analyze the received soiling data. Because this model is trained based on a past database, it can analyze new data with high accuracy. The input is numerical data on the soiling level, and the output is classification data such as "light soiling," "moderate soiling," or "heavy soiling."

[0204] Step 4:

[0205] The server notifies the analysis results to the display device of the washing machine and the user's information communication device. The input is the classification data, and the output is a notification message.

[0206] Step 5:

[0207] The user checks the notification on the information communication device and selects whether to run or cancel the laundry. The input is the notification message, and the output is the user's selected action ("run" or "cancel").

[0208] Step 6:

[0209] Cameras and chemical sensors installed in the home detect abnormalities, such as suspicious movements, sounds, gas leaks, etc. The input is environmental data within the home, and the output is numerical data on abnormalities.

[0210] Step 7:

[0211] Data obtained from cameras and sensors is also collected by the terminal and sent to the server via the Internet. The input is numerical data of anomalies, and the output is packetized data.

[0212] Step 8:

[0213] The server then launches the generative AI model again to analyze the received anomaly data. The input is the numerical data of the anomaly, and the output is classification data such as "mild anomaly," "moderate anomaly," or "severe anomaly."

[0214] Step 9:

[0215] The server notifies the user of the results of the anomaly analysis to the display device in the home and the user's information and communication device. The input is the classification data, and the output is an alarm notification message.

[0216] Step 10:

[0217] The user checks the alarm notification on the information communication device and selects whether to activate or cancel the alarm. The input is the alarm notification message, and the output is the user's selected action ("activate" or "cancel").

[0218] In this way, the system of the present invention achieves improved efficiency and security within the home through collaboration between sensors, cameras, generative AI models, display devices, information and communication devices, and servers.

[0219] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0220] The present invention is a system that automatically detects the degree of soiling of clothes in a washing machine and performs washing only when necessary, thereby reducing unnecessary washing and suggesting washing actions taking into account the user's emotions. This system mainly consists of the following elements.

[0221] 1. Dirt detection by sensor

[0222] The sensors installed inside the washing machine have the ability to detect the degree of dirt, color, and type of laundry in real time. When a user puts laundry into the washing machine, the sensors are automatically activated and start collecting data.

[0223] 2. Data transmission

[0224] The detected data is collected by a terminal inside the washing machine and then transmitted to a server over the internet, where the terminal uses a communication protocol to package and transmit the data efficiently.

[0225] 3. Analysis using AI models

[0226] The server launches a generative AI model to analyze the received data. Based on the learned data, the AI ​​model analyzes the degree of soiling of the clothes and determines whether they need to be washed. The server uses this model to generate the results of the data analysis. For example, it outputs classification results such as "lightly soiled," "moderately soiled," and "heavily soiled."

[0227] 4. Recognition of user emotions using an emotion engine

[0228] The emotion engine has the ability to recognize the user's emotions based on voice or image data acquired from the user's device. For example, the emotion engine analyzes data provided by the user through the smartphone's camera or microphone to determine the user's emotional state (stress, joy, calm, etc.).

[0229] 5. Notification of Results

[0230] The analysis results are sent to the washing machine terminal and the user's smartphone app. The terminal displays messages such as "No washing necessary" or "Washing required" on the washing machine panel. The user can check the detailed analysis results through the smartphone app.

[0231] 6. User Choice

[0232] Based on the analysis results, the user can choose whether to run or cancel the wash. For example, if the user receives a "no washing required" notification, the wash will not run unless the user presses the "start washing" button on the smartphone app or the washing machine's control panel. On the other hand, if the user presses the "start washing" button, the normal wash cycle will begin.

[0233] 7. Emotion-based recommendations

[0234] If the emotion engine recognizes that the user is in a stressful state, the system will suggest or automatically start doing laundry, regardless of the result of the laundry necessity judgment. This reduces the user's mental burden.

[0235] Specific examples

[0236] Case 1: Light soiling

[0237] The user places the laundry in the washing machine and closes the door.

[0238] The device activates a sensor to detect the degree of dirt on the clothes.

[0239] The device transmits the detected data to the server.

[0240] The server analyzes the data using the generated AI model and determines that the item is "slightly soiled and no washing is required."

[0241] The server notifies the device and the user's smartphone app of the analysis results.

[0242] The device will display "No washing required" on its panel, and a notification will also be sent to the user's smartphone app.

[0243] When the user recognizes through the emotion engine that they are in a stressful state, the system will suggest or automatically start doing laundry.

[0244] The device will start the wash and notify the user when it is finished.

[0245] Case 2: Moderate to severe soiling

[0246] The user places the laundry in the washing machine and closes the door.

[0247] The device activates a sensor to detect the degree of dirt on the clothes.

[0248] The device transmits the detected data to the server.

[0249] The server analyzes the data using the generated AI model and determines that the garment is "moderately soiled and requires washing."

[0250] The server notifies the device and the user's smartphone app of the analysis results.

[0251] The device will display "Washing required" on the panel, and a notification will also be sent to the user's smartphone app.

[0252] The user checks the notification on the smartphone app and selects to perform the laundry.

[0253] The device will begin a normal wash cycle.

[0254] As described above, the system of the present invention can improve the efficiency of laundry through interactions between sensors, servers, AI models, emotion engines, terminals, and users. This system is also environmentally friendly, as it reduces unnecessary laundry, contributes to saving water resources, and reduces household water bills. Furthermore, by suggesting laundry actions based on the user's emotional state, it also contributes to reducing mental burden.

[0255] The processing flow will be explained below.

[0256] Step 1:

[0257] The user loads laundry into the washing machine and closes the door, ready for the wash cycle to begin.

[0258] Step 2:

[0259] The device activates the sensors inside the washing machine, which detect the soiling level, color, and type of clothes in real time and collect data.

[0260] Step 3:

[0261] The device temporarily stores the data acquired from the sensor and then prepares it to be sent to a server via the Internet.

[0262] Step 4:

[0263] The device then sends the packaged data over the internet to a server, which includes details such as the soiling level, color, and type of clothing.

[0264] Step 5:

[0265] The server then launches a generative AI model to analyze the received data. Based on past learning data, the AI ​​model analyzes the degree of soiling of the clothes and determines whether they need to be washed.

[0266] Step 6:

[0267] The server generates the analysis results, which are displayed as classifications such as "lightly soiled," "moderately soiled," and "heavily soiled."

[0268] Step 7:

[0269] The analysis results generated by the server are sent to the device and the user's smartphone app.

[0270] Step 8:

[0271] The device displays the analysis results on the washing machine's control panel, such as a message like "No washing necessary" or "Washing required."

[0272] Step 9:

[0273] The user opens the smartphone app and checks the analysis results, which are displayed in detail.

[0274] Step 10:

[0275] To recognize the user's emotions, the emotion engine acquires voice or image data from the user's device and determines the user's emotional state (stress, joy, calm, etc.) based on this data.

[0276] Step 11:

[0277] If the user's emotion recognized by the emotion engine is a stress state, the system will suggest to the user to do laundry or automatically start laundry regardless of the result of the judgment on the necessity of laundry.

[0278] Step 12:

[0279] The user can choose whether to start or stop the wash by pressing the "Start Wash" or "Cancel" button on the smartphone app or the washing machine's control panel.

[0280] Step 13:

[0281] The device acts based on the user's selection. If the user selects "Start Wash," the device starts a normal wash cycle. If the user selects "Cancel," the device stops the wash and prompts the user to remove the clothes.

[0282] Step 14:

[0283] The user removes the clothes. If it is determined that washing is not necessary, the user removes the clothes and stores them. If washing is performed, the user removes the clothes after the washing is completed.

[0284] These are the specific processing steps from detecting dirt in a washing machine to starting or canceling the wash. This system can reduce unnecessary laundry, lighten the burden of housework, and conserve water resources. Furthermore, the emotion engine can take the user's emotions into consideration when suggesting laundry actions, thereby helping to reduce the user's mental burden.

[0285] Example 2

[0286] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0287] Conventional washing machine systems wash clothes without considering the degree of dirt or the user's emotional state, which has led to problems such as increased wasted laundry and mental stress on the user.In addition, there is a lack of systems that accurately determine the need for laundry, resulting in wasted water resources and energy.

[0288] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0289] In this invention, the server includes a means for detecting the degree of soiling of clothes using a sensor installed in the washing machine, a means for analyzing the detected data using an AI model generated by the server and determining whether the clothes need to be washed, and a means for recognizing the user's emotional state using an emotion engine and suggesting or automatically starting a washing action based on that. This reduces unnecessary laundry, eases the user's mental burden, and enables the saving of water resources and energy.

[0290] A "sensor" refers to a device installed in a washing machine that detects the degree of dirt, color, and type of clothes in real time.

[0291] "Server" refers to the central computer that uses the generated AI model to analyze the detected data and determine whether clothes need to be washed.

[0292] An "AI model" refers to an algorithm that is generated on a server, analyzes learned data, and determines the degree of dirtiness of clothing.

[0293] An "emotion engine" refers to software that analyzes a user's voice data or image data and recognizes the user's emotional state.

[0294] The "washing machine panel" refers to the display device attached to the washing machine itself, which notifies the user of the analysis results and system status.

[0295] "User's terminal" refers to a mobile device such as a smartphone or tablet used by the user, which can be used to check notifications and results from the system and perform operations.

[0296] "Analysis results" refers to the judgment results on the degree of dirt and the need for washing that are generated after the server analyzes the data using an AI model.

[0297] "Laundry behavior" refers to a series of actions that a washing machine actually takes to wash clothes based on the analysis results and the user's emotional state.

[0298] This invention is a system that automatically detects soiling of clothes in a washing machine and performs washing only when necessary, thereby reducing unnecessary washing and suggesting washing actions taking into account the user's emotions. This system is realized using sensors installed in the washing machine, a server, a generative AI model, an emotion engine, a user's device, and communication means via the Internet.

[0299] First, the user puts laundry into the washing machine and closes the door. At this time, the device activates the sensors inside the washing machine. The sensors detect the degree of dirt, color, and type of the clothes in real time and collect that data. Specific examples of sensors include optical sensors for detecting dirt and color identification sensors.

[0300] The detected data is collected by the device and then transmitted to a server via the Internet. The device uses a communication protocol to efficiently transmit the data, such as HTTP or MQTT.

[0301] The server launches a generative AI model based on the received data. The generative AI model includes an algorithm that analyzes the degree of soiling of the clothes based on the learned data and determines whether they need to be washed. The analysis results are output as classifications such as "lightly soiled," "moderately soiled," or "heavily soiled."

[0302] At the same time, the voice or image data provided by the user using the smartphone's camera or microphone is analyzed by the emotion engine on the server, which determines the user's emotional state (stress, joy, calm, etc.).

[0303] The analysis results are then sent back to the device and the user's smartphone app. The device displays a message such as "No washing necessary" or "Washing necessary" on the washing machine's panel, and a notification is also sent to the user's smartphone app. For example, if "Washing unnecessary" is displayed, the user must press the "Start washing" button for the washing to begin.

[0304] Furthermore, if the emotion engine recognizes the user's emotion as stressed, the server will send the user a notification suggesting that they do laundry, regardless of the result of the judgment on the necessity of doing laundry. Even if the user ignores the suggestion, the system will automatically start the laundry. In this way, the laundry is carried out, and when it is finished, the terminal will send a completion notification to the user.

[0305] Specific examples

[0306] Case 1: Light soiling

[0307] 1. The user loads laundry into the washing machine and closes the door.

[0308] 2. The device activates the sensor and detects the degree of dirt on the clothes.

[0309] 3. The device sends the detection data to the server.

[0310] 4. The server analyzes the data using the generated AI model and determines that the item is "slightly soiled and no washing is required."

[0311] 5. The server notifies the device and the user's smartphone app of the analysis results.

[0312] 6. The device will display "No washing required" on the panel, and a notification will also be sent to the user's smartphone app.

[0313] 7. When the user recognizes through the emotion engine that they are in a stressful state, the system will suggest or automatically start doing laundry.

[0314] 8. The device starts the wash and notifies the user when it is finished.

[0315] Case 2: Moderate to severe soiling

[0316] 1. The user loads laundry into the washing machine and closes the door.

[0317] 2. The device activates the sensor and detects the degree of dirt on the clothes.

[0318] 3. The device sends the detection data to the server.

[0319] 4. The server analyzes the data using the generated AI model and determines that the item is "moderately soiled and requires washing."

[0320] 5. The server notifies the device and the user's smartphone app of the analysis results.

[0321] 6. The device will display "Laundry required" on the panel, and a notification will also be sent to the user's smartphone app.

[0322] 7. The user checks the notification on the smartphone app and selects to perform the laundry.

[0323] 8. The device will begin a normal wash cycle.

[0324] The system leverages generative AI models and an emotion engine to provide an efficient and user-friendly laundry experience.

[0325] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0326] Step 1:

[0327] The user places laundry in the washing machine and closes the door. This action automatically activates the device's built-in sensors, which detect the laundry's soiling level, color, and type in real time and collect the data. The input is the laundry, and the output is data on soiling level, color, and type.

[0328] Step 2:

[0329] The terminal first records the collected data in its internal memory and then transmits it to a server via the Internet. The terminal uses communication protocols such as HTTP and MQTT to transmit data efficiently. The input is data from the sensor, and the output is packaged data.

[0330] Step 3:

[0331] The server stores the received data in a database. Next, it launches a generative AI model and performs analysis based on the stored data. The AI ​​model evaluates the level of dirt through shading analysis and color identification, and outputs a classification result such as "lightly dirty," "moderately dirty," or "heavily dirty." The input is packaged data, and the output is the classification result of the level of dirt.

[0332] Step 4:

[0333] The user provides voice or image data using the camera or microphone on their smartphone. This data is sent to a server via the Internet and analyzed by an emotion engine. The emotion engine determines the user's emotional state (stress, joy, calm, etc.) and sends the results to the server. The input is voice or image data, and the output is the determined emotional state.

[0334] Step 5:

[0335] The server sends the analysis results and the emotional state determination results to the washing machine device and the user's smartphone app. The device displays a message such as "No washing necessary" or "Laundry required" on the washing machine panel, and a notification is also sent to the user's smartphone app. The input is the analysis results and the emotional state determination results, and the output is a notification message.

[0336] Step 6:

[0337] The user selects whether to press the "Start Wash" button on the smartphone app or the washing machine's control panel based on the notification of the laundry need. If the user selects to run the laundry, the device starts a normal wash cycle. The input is the user's selection, and the output is running the laundry.

[0338] Step 7:

[0339] If the emotion engine recognizes the user's emotion as stressful, the server sends a notification to the user suggesting laundry actions regardless of the result of the judgment on the necessity of laundry. Even if the suggestion is ignored, the system will automatically start the laundry. The input is the emotion analysis result, and the output is the laundry action suggestion or automatic execution.

[0340] Step 8:

[0341] When the wash is finished, the device sends a completion notification to the user. The notification is displayed on the washing machine panel and on the user's smartphone app. The input is the end status of the wash cycle, and the output is the completion notification.

[0342] In this way, it is possible to reduce wasted laundry and suggest laundry actions that take into account the user's emotional state.

[0343] (Application example 2)

[0344] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0345] On factory production lines, contamination of products and parts has a significant impact on shipping quality, but manually detecting and cleaning contamination is time-consuming and laborious. Furthermore, depending on the emotional state of the operator, it can be difficult to make appropriate decisions. Therefore, a system is needed that automatically detects contamination on product lines and makes decisions about cleaning or halting shipments as necessary. Furthermore, a system is needed that reduces the mental burden by making appropriate suggestions and taking appropriate action, taking into account the emotional state of the operator.

[0346] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0347] In this invention, the server includes means for detecting the degree of soiling of clothes using a sensor installed in the washing machine, means for analyzing the detected data using an AI model generated by the server and determining whether the clothes need to be washed, means for notifying the washing machine panel and the user's terminal of the result that the clothes do not need to be washed, means for the user to choose whether to perform or cancel the washing, means for automatically detecting the contamination state of the product line and parts, means for determining whether to perform cleaning work or stop shipping based on the contamination state, and means for recognizing the user's emotional state and suggesting cleaning or shipping. This makes it possible to automatically detect soiling and take necessary measures, while also reducing the burden on the operator.

[0348] A "sensor" is a device that measures physical or chemical quantities and outputs them as data. Optical sensors and image recognition sensors are particularly used.

[0349] A "server" is a computer system that provides services such as data storage, processing, and distribution over a network.

[0350] An "AI model" is an algorithm that has been trained to perform a specific task using artificial intelligence techniques, particularly frameworks such as TensorFlow and PyTorch.

[0351] A "panel" is an interface for displaying information. It is installed in washing machines and other appliances.

[0352] A "user's terminal" is a device used by a user, such as a computer or smartphone.

[0353] A "product line" refers to the equipment and facilities used to continuously produce products within a factory.

[0354] "Stain level" refers to the degree of foreign matter adhering to the surface of an object and the resulting discoloration.

[0355] "Data analysis" is the process of analyzing collected data and extracting meaningful information.

[0356] A "cleaning operation" is a manual or automated process for removing dirt or foreign matter.

[0357] "Shipment suspension" means stopping the shipment of a product, and is done to ensure quality.

[0358] "Emotional state" refers to the current psychological state of the user or operator.

[0359] The present invention is a system that uses a robot installed on a factory production line to automatically detect dirt on products and parts, and based on the results, decides whether to perform cleaning work or stop product shipments. This system is composed of the following elements.

[0360] 1. Dirt detection by sensor

[0361] The robot is equipped with optical sensors and image recognition sensors that constantly monitor the production line and detect the degree of dirt on products and parts. The data detected by the sensors is temporarily collected on a terminal inside the robot.

[0362] 2. Data transmission

[0363] The collected data is sent to a cloud server via Wi-Fi or a wired connection, and the device uses a communication protocol to package and efficiently transmit the data.

[0364] 3. Analysis using AI models

[0365] The server uses a generative AI model to analyze the data it receives, specifically an AI model based on frameworks such as TensorFlow and PyTorch, to analyze the data and classify the level of dirt into categories such as "lightly dirty," "moderately dirty," and "heavily dirty."

[0366] 4. Recognition of user emotions using an emotion engine

[0367] An emotion engine is used to recognize the emotional state of the operator through a smartphone app or computer. If the operator is under stress or overload, the emotion engine will detect this.

[0368] 5. Notification of Results

[0369] The analysis results are sent to the robot's terminal and the operator's smartphone or computer. The terminal displays messages such as "Cleaning required" or "Shipping suspended" on the robot's panel, and the same information is also sent to the operator's terminal.

[0370] 6. User Selection and Autorun

[0371] Based on the analysis results, the operator can choose to carry out cleaning work, cancel the work, or stop shipping. If the operator's emotional state is stressed, the system will automatically carry out cleaning work or stop shipping.

[0372] This system will automatically detect contamination and take necessary measures, and is expected to reduce the burden and mental stress on operators.

[0373] As a concrete example, you can use the following prompt to have an AI model perform an analysis:

[0374] It judges the degree of soiling of the product line based on images, and starts cleaning if the soiling score exceeds the previous threshold, or skips cleaning if it does not. It also starts cleaning if the operator's emotional state is stressed or overloaded.

[0375] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0376] Step 1:

[0377] Dirt detection by sensors

[0378] Optical sensors and image recognition sensors installed on the robot detect the degree of dirt on products and parts on the production line. The input is the physical condition of the product or part, and the output is a measurement value of the degree of dirt as digital data. This measurement value is temporarily stored on a terminal inside the robot.

[0379] Step 2:

[0380] Sending data

[0381] The device transmits the collected soiling data to a cloud server via Wi-Fi or a wired connection. The input is the measurement data stored in the device, and the output is the data transferred to the cloud server. The device packages the data for efficient transmission.

[0382] Step 3:

[0383] Analysis using AI models

[0384] The server analyzes the received data using a generative AI model. Specifically, it uses a pre-trained model based on TensorFlow and PyTorch to classify the degree of dirt into categories such as "lightly dirty," "moderately dirty," and "heavily dirty." The input is the digital data of the dirt level sent to the cloud server, and the output is the classification result.

[0385] Step 4:

[0386] Recognizing user emotions with an emotion engine

[0387] The emotion engine recognizes the operator's emotional state through the user's smartphone app or computer. The input is voice data and image data provided by the operator, and the output is the judgment result of the operator's emotional state. The emotion engine uses technologies such as OpenVINO.

[0388] Step 5:

[0389] Notification of analysis results and emotional state

[0390] The server notifies the robot's terminal and the operator's smartphone or computer of the analysis results and emotional state. The input is the classification results of the dirt level analyzed by the server and the judgment results of the emotion engine, and the output is a notification message. The robot's panel displays messages such as "Cleaning required" or "Shipping suspended."

[0391] Step 6:

[0392] User selection and automatic execution

[0393] Based on the notified analysis results, the operator can choose whether to carry out cleaning work or stop shipments. The input is the notification message and the operator's choice, and the output is the start of cleaning work or the stop of shipments. If the operator's emotional state is stressed, cleaning work or shipments will be automatically stopped.

[0394] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0395] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0396] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0397] [Second embodiment]

[0398] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0399] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0400] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0401] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0402] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0403] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0404] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0405] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0406] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0407] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0408] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0409] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0410] The present invention is a system that automatically detects the degree of soiling of clothes in a washing machine and performs washing only when necessary, thereby reducing unnecessary washing. This system mainly consists of the following elements.

[0411] 1. Dirt detection by sensor

[0412] The sensors installed in the washing machine detect the degree of soiling, color, and type of laundry in real time. These sensors can be optical or chemical sensors, for example. When a user puts laundry into the washing machine, the sensors are automatically activated and start collecting data.

[0413] 2. Data transmission

[0414] The sensed data is collected by a terminal inside the washing machine and then transmitted over the internet to a server, which uses a communications protocol to package and efficiently transmit the data.

[0415] 3. Analysis using AI models

[0416] The server launches a generative AI model to analyze the received data. This AI model learns from past data and can accurately determine whether clothes need to be washed. The server uses this model to generate data analysis results. For example, it outputs classification results such as "lightly soiled," "moderately soiled," and "heavily soiled."

[0417] 4. Notification of Results

[0418] The analysis results are sent to the washing machine terminal and the user's smartphone app. The terminal displays messages such as "No washing necessary" or "Washing required" on the washing machine panel. The user can check the detailed analysis results through the smartphone app.

[0419] 5. User Choice

[0420] Based on the analysis results, the user can choose whether to run or cancel the wash. For example, if the user receives a "no washing required" notification, the wash will not run unless the user presses the "start washing" button on the smartphone app or the washing machine's control panel. On the other hand, if the user presses the "start washing" button, the normal wash cycle will begin.

[0421] Specific examples

[0422] Case 1: Light soiling

[0423] The user puts laundry into the washing machine.

[0424] The device activates a sensor to detect the degree of dirt on the clothes.

[0425] The device transmits the detected data to the server.

[0426] The server analyzes the data using the generated AI model and determines that the item is "slightly soiled and no washing is required."

[0427] The server notifies the device and the user's smartphone app of the analysis results.

[0428] The device will display "No washing required" on its panel, and a notification will also be sent to the user's smartphone app.

[0429] The user checks the notification on the smartphone app and chooses to cancel the wash.

[0430] The device will stop the wash and prompt the user to remove the clothes.

[0431] Case 2: Moderate to severe soiling

[0432] The user puts laundry into the washing machine.

[0433] The device activates a sensor to detect the degree of dirt on the clothes.

[0434] The device transmits the detected data to the server.

[0435] The server analyzes the data using the generated AI model and determines that the garment is "moderately soiled and requires washing."

[0436] The server notifies the device and the user's smartphone app of the analysis results.

[0437] The device will display "Washing required" on the panel, and a notification will also be sent to the user's smartphone app.

[0438] The user checks the notification on the smartphone app and selects to perform the laundry.

[0439] The device will begin a normal wash cycle.

[0440] As described above, the system of the present invention can improve the efficiency of laundry through the interaction between sensors, servers, AI models, terminals, and users. This system also reduces unnecessary laundry, contributes to saving water resources, and reduces household water bills, making it environmentally friendly.

[0441] The processing flow will be explained below.

[0442] Step 1:

[0443] The user loads laundry into the washing machine and closes the door, ready for the wash cycle to begin.

[0444] Step 2:

[0445] The device activates the sensors inside the washing machine, which detect the soiling level, color, and type of clothes in real time and collect data.

[0446] Step 3:

[0447] The device temporarily stores the data acquired from the sensor and then prepares it to be sent to a server via the Internet.

[0448] Step 4:

[0449] The device then sends the packaged data over the internet to a server, which includes details such as the degree of dirt, color, and type.

[0450] Step 5:

[0451] The server then launches a generative AI model to analyze the received data. Based on the learned data, the AI ​​model analyzes the degree of soiling of the clothes and determines whether they need to be washed.

[0452] Step 6:

[0453] The server generates the analysis results, which are displayed as classifications such as "lightly soiled," "moderately soiled," and "heavily soiled."

[0454] Step 7:

[0455] The analysis results generated by the server are sent to the device and the user's smartphone app.

[0456] Step 8:

[0457] The device displays the analysis results on the washing machine's control panel, such as a message like "No washing necessary" or "Washing required."

[0458] Step 9:

[0459] The user opens the smartphone app and checks the analysis results, which are displayed in detail.

[0460] Step 10:

[0461] The user can choose whether to start or stop the wash by pressing the "Start Wash" or "Cancel" button on the smartphone app or the washing machine's control panel.

[0462] Step 11:

[0463] The device acts based on the user's selection. If the user selects "Start Wash," the device starts a normal wash cycle. If the user selects "Cancel," the device stops the wash and prompts the user to remove the clothes.

[0464] Step 12:

[0465] The user removes the clothes. If it is determined that washing is not necessary, the user removes the clothes and stores them. If washing is performed, the user removes the clothes after the washing is completed.

[0466] These are the specific steps in the washing machine's process, from detecting dirt to starting or canceling the wash. This system reduces unnecessary washing, lightens the burden of housework, and conserves water resources.

[0467] Example 1

[0468] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0469] Conventional washing machines have difficulty accurately detecting the degree of soiling of laundry, resulting in unnecessary washing. This results in wasted water and electricity consumption, placing a heavy burden on the environment. Another issue is that users must check the degree of soiling of their clothes and select the appropriate washing method every time they wash, which is time-consuming.

[0470] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0471] In this invention, the server includes: means for detecting the degree of soiling of clothes using a sensor installed in the washing machine; means for transmitting the detected data to the server via the Internet via a terminal; means for analyzing the received data using an AI model generated by the server and determining whether the clothes need to be washed; means for notifying the user of the analysis results via the washing machine's operation panel and the user's terminal; and means for the user to select whether to perform or cancel the wash based on the analysis results. This reduces unnecessary washing and prevents the wasteful consumption of water resources and electricity. It also allows users to wash clothes efficiently without hassle, contributing to environmental protection.

[0472] "Sensor" refers to a device installed in a washing machine that detects the degree of soiling, color, type, etc. of clothes. This includes optical sensors and chemical sensors.

[0473] A "terminal" is a device that is placed inside the washing machine and is responsible for collecting data from the sensors and sending it to the server.

[0474] A "server" is a computer system that receives and analyzes data sent from a terminal via the Internet.

[0475] The "AI model" is a machine learning model that runs on a server and is used to analyze received data and determine whether clothes need to be washed.

[0476] The "operation panel" is an interface device that is provided in the washing machine and that displays the analysis results to the user.

[0477] The "means of notifying the terminal" is a communication means for transmitting the analysis results from the server to the terminal. It uses the Internet Protocol.

[0478] A "user's device" is a communication-enabled device such as a smartphone or tablet held by a user, and is responsible for receiving and displaying notifications from the server.

[0479] "Means by which the user can choose whether to start or stop washing based on the analysis results" refers to an interface that allows the user to check the analysis results through the operation panel or the user's terminal and choose whether to start or stop washing.

[0480] The present invention is a system that automatically detects the degree of soiling of clothes in a washing machine and performs washing as necessary, thereby reducing unnecessary washing. This system is mainly composed of sensors, terminals, a server, a generative AI model, and a user smartphone app. Specific embodiments of the system are described below.

[0481] 1. Sensor activation and data collection

[0482] When a user loads laundry into the washing machine, the device inside the washing machine automatically activates sensors as soon as the lid is closed. The sensors use optical and chemical sensors to detect data such as the soiling level, color, and type of clothes. Optical sensors scan the surface of the clothes and measure the intensity and wavelength of reflected light. Chemical sensors detect specific pollutants, such as volatile organic compounds (VOCs) and specific stains. The detected data is collected and stored locally on the device.

[0483] 2. Data transmission

[0484] The device packages the collected data and sends it to a server via the Internet. The data is packaged in a specified format (for example, JSON format) and includes information such as the degree of dirt, color, and type. HTTP / HTTPS is generally used as the transmission protocol. For example, the endpoint URL is "https: / / example.com / api / wash-data".

[0485] 3. Data Analysis

[0486] The server launches a generative AI model to analyze the received data. The AI ​​model is implemented using frameworks such as TensorFlow and PyTorch. This AI model learns from past data and can accurately determine whether clothes need to be washed. The generative AI model analyzes the data and outputs classification results, such as "lightly soiled," "moderately soiled," or "heavily soiled." The analysis results are stored on the server.

[0487] 4. Result notification

[0488] The server notifies the device inside the washing machine and the user's smartphone app of the analysis results. Notifications are sent via push notifications and REST API calls. The smartphone app receives a notification such as "Lightly soiled, no need to wash," and the washing machine's control panel displays a message based on the analysis results.

[0489] 5. User Choice

[0490] Based on the analysis results, the user can choose whether to run or cancel the wash. They can receive a notification via the smartphone app and check the details. The user decides whether to run the wash by choosing whether to press the "Start Wash" button. If "washing is not necessary," the wash will not be run if the user does nothing. On the other hand, if it is determined that "washing is necessary," the device will start a normal wash cycle when the user presses the "Start Wash" button. The user will be notified by a voice message or other means when the cycle starts.

[0491] Specific examples

[0492] Case 1: Light soiling

[0493] The user puts laundry into the washing machine.

[0494] The device activates a sensor to detect the degree of dirt on the clothes.

[0495] The device transmits the detected data to the server.

[0496] The server analyzes the data using the generated AI model and determines that the item is "slightly soiled and no washing is required."

[0497] The server notifies the device and the user's smartphone app of the analysis results.

[0498] The device will display "No washing required" on the operation panel, and a notification will also be sent to the user's smartphone app.

[0499] The user checks the notification on the smartphone app and chooses to cancel the wash.

[0500] The device will stop the wash and prompt the user to remove the clothes.

[0501] Case 2: Moderate to severe soiling

[0502] The user puts laundry into the washing machine.

[0503] The device activates a sensor to detect the degree of dirt on the clothes.

[0504] The device transmits the detected data to the server.

[0505] The server analyzes the data using the generated AI model and determines that the garment is "moderately soiled and requires washing."

[0506] The server notifies the device and the user's smartphone app of the analysis results.

[0507] The device will display "Washing required" on the operation panel, and a notification will also be sent to the user's smartphone app.

[0508] The user checks the notification on the smartphone app and selects to perform the laundry.

[0509] The device will begin a normal wash cycle.

[0510] This reduces unnecessary washing and prevents the wasteful consumption of water and electricity. It also allows users to wash their clothes efficiently and hassle-free, contributing to environmental protection.

[0511] Prompt Sentence Examples

[0512] "Please create a prompt for a system that detects the degree of soiling of laundry and performs the washing if necessary."

[0513] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0514] Step 1:

[0515] Sensor activation and data collection

[0516] A user loads laundry into the washing machine and closes the lid. This action automatically activates sensors on a device inside the washing machine. The sensors use optical and chemical sensors to detect the soiling level of the clothes. Specifically, optical sensors scan the surface of the clothes and measure the intensity and wavelength of reflected light. Chemical sensors detect volatile organic compounds (VOCs) and specific stains. The collected data is then sent directly to the device and stored locally.

[0517] Input: Load laundry and close lid

[0518] Output: Detection data such as dirt level, color, type, etc.

[0519] Step 2:

[0520] Sending data

[0521] The device packages the data collected from the sensor. The data is packaged in a specified format such as JSON. For example, the data may include information such as the degree of dirt, color, and type. The device then transmits the data to a server via the Internet. HTTP / HTTPS is generally used as the transmission protocol.

[0522] Input: Detection data from sensors

[0523] Output: Packaged data sent to the server

[0524] Step 3:

[0525] Data analysis

[0526] The server launches a generative AI model to analyze the received data. The server then analyzes the data using an AI model implemented using frameworks such as TensorFlow or PyTorch. The generative AI model evaluates the degree of soiling of the clothes based on the received data and outputs a classification result such as "lightly soiled," "moderately soiled," or "heavily soiled." The analysis results are stored on the server.

[0527] Input: Data package sent from the terminal

[0528] Output: Analysis results (classified data: mild, mild, moderate, severe)

[0529] Step 4:

[0530] Result notification

[0531] The server notifies the washing machine device and the user's smartphone app of the analysis results. Notification methods include push notifications and REST API calls. This allows the user's smartphone app to receive specific notifications, such as "Lightly soiled, no need to wash." The results are also displayed on the washing machine's control panel.

[0532] Input: Analysis results of the AI ​​model

[0533] Output: Notifications to the device and the user's smartphone app

[0534] Step 5:

[0535] User Selection

[0536] The user checks the analysis results through the smartphone app or the washing machine's operation panel, and can then choose whether to start or stop the wash. If the notification is "lightly soiled, no washing necessary," the wash will not be carried out unless the user presses the "start washing" button. On the other hand, if the notification is "moderately soiled, washing required," the wash will begin if the user presses the "start washing" button on the smartphone app or operation panel.

[0537] Input: User selection via smartphone app or operation panel

[0538] Output: Decision to do or not do laundry

[0539] Step 6:

[0540] Starting the wash cycle

[0541] If the user decides to wash the clothes, the device will start the washing machine's normal wash cycle, and the user will be notified when the wash cycle starts, for example, through a voice message or a display on the operation panel.

[0542] Input: User's laundry run selection

[0543] Output: Start and run a normal wash cycle

[0544] (Application example 1)

[0545] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0546] The present invention relates to a system for reducing unnecessary laundry and false alarms in homes and other environments. Specifically, the system detects the degree of soiling of clothes and abnormalities in the home with high accuracy, and performs laundry and issues alarms only when necessary, thereby preventing waste of resources and false alarms.

[0547] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0548] In this invention, the server includes a means for detecting the degree of soiling of clothes using a sensor installed in the washing machine, a means for analyzing the detected data using an AI model generated by the server and determining whether the clothes need to be washed, and a means for detecting abnormalities in the home using cameras and chemical sensors installed in the home. This enables the system to automatically detect the degree of soiling of clothes and abnormal conditions in the home and notify the user, thereby reducing unnecessary washing and warnings.

[0549] A "sensor" is a device that senses physical or chemical quantities, converts them into electrical signals, and outputs them.

[0550] An "AI model" is an artificial intelligence algorithm designed to learn from past data and analyze new data to make decisions.

[0551] A "server" refers to equipment or software used to store, process, and analyze data on a network.

[0552] A "washing machine" is a mechanical device for washing clothes.

[0553] A "display device" is a device for visually displaying information, and includes, for example, a display or a panel.

[0554] An "information communication device" is a device for sending and receiving data and information over a network, and includes, for example, smartphones and tablets.

[0555] A "camera" is a device that takes pictures and videos and records them as data.

[0556] A "chemical sensor" is a device that detects specific chemical substances and outputs their presence and concentration as an electrical signal.

[0557] "Abnormal" refers to an event or situation that is different from normal, such as suspicious movements or sounds, or gas leaks.

[0558] An "alarm" refers to a system or function that uses sound, light, or other means to draw attention when an abnormality occurs.

[0559] This invention is a system for efficient home laundry and security management. The system consists of a washing machine, sensors, an AI model, a display device, an information and communication device, a camera, and a chemical sensor.

[0560] First, sensors installed in the washing machine detect the degree of soiling of the clothes put in. Specifically, optical sensors or chemical sensors are used. The data collected by the sensors is temporarily stored on a terminal inside the washing machine and then sent to a server via the Internet.

[0561] The server launches a generative AI model to analyze the received data. This AI model learns from past data and accurately determines whether clothes need to be washed. For example, it can classify clothes into categories such as "lightly soiled," "moderately soiled," and "heavily soiled." The analysis results are notified to the washing machine's display and the user's information and communication device.

[0562] Based on these notifications, the user can choose to either run or cancel the laundry. If the analysis results indicate that washing is not necessary, the user can cancel the laundry and, if necessary, run the laundry manually.

[0563] In addition, cameras and chemical sensors installed in the home will detect abnormalities, such as suspicious movements, sounds, and gas leaks. If these sensors detect an abnormality, the data will also be sent to the server. The generated AI model will analyze the severity of the abnormality and output a classification result such as "mild abnormality," "moderate abnormality," or "severe abnormality." This result will also be notified to the display device in the home and the user's information and communication device.

[0564] The user can choose to issue or cancel an alert based on this notification. If the analysis result is judged to be a "serious abnormality," the user is advised to take immediate action to address the issue.

[0565] For example, if a home camera detects abnormal activity, the data is sent to a server and analyzed by an AI model. If the analysis results in a "severe abnormality," a notification is sent to the user's smartphone stating, "Suspicious activity has been detected!" The user can then choose to activate or cancel the alarm via the smartphone app.

[0566] An example of a specific prompt is:

[0567] An abnormality was detected on a home camera at 16:45. After analyzing the video, it was determined that a "serious abnormality" had occurred. Please check immediately.

[0568] There are messages like this.

[0569] The hardware used includes washing machines, cameras, sensors, displays, and information and communication devices. The software includes OpenCV, Requests, and a generative AI model. The generative AI model is implemented using Python's TensorFlow and PyTorch.

[0570] With this configuration, the present invention can reduce wasted laundry and the occurrence of false alarms, contributing to improved efficiency and security within the home.

[0571] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0572] Step 1:

[0573] The device activates sensors installed in the washing machine to detect the degree of soiling of the clothes placed in it. Optical and chemical sensors are used for this purpose. The input is the physical data of the clothes, and the output is numerical data on the degree of soiling.

[0574] Step 2:

[0575] The device collects data on the level of dirt detected by the sensor and sends it to a server via the Internet. The input is numerical data on the level of dirt, and the output is packetized data.

[0576] Step 3:

[0577] The server launches a generative AI model to analyze the received soiling data. Because this model is trained based on a past database, it can analyze new data with high accuracy. The input is numerical data on the soiling level, and the output is classification data such as "light soiling," "moderate soiling," or "heavy soiling."

[0578] Step 4:

[0579] The server notifies the analysis results to the display device of the washing machine and the user's information communication device. The input is the classification data, and the output is a notification message.

[0580] Step 5:

[0581] The user checks the notification on the information communication device and selects whether to run or cancel the laundry. The input is the notification message, and the output is the user's selected action ("run" or "cancel").

[0582] Step 6:

[0583] Cameras and chemical sensors installed in the home detect abnormalities, such as suspicious movements, sounds, gas leaks, etc. The input is environmental data within the home, and the output is numerical data on abnormalities.

[0584] Step 7:

[0585] Data obtained from cameras and sensors is also collected by the terminal and sent to the server via the Internet. The input is numerical data of anomalies, and the output is packetized data.

[0586] Step 8:

[0587] The server then launches the generative AI model again to analyze the received anomaly data. The input is the numerical data of the anomaly, and the output is classification data such as "mild anomaly," "moderate anomaly," or "severe anomaly."

[0588] Step 9:

[0589] The server notifies the user of the results of the anomaly analysis to the display device in the home and the user's information and communication device. The input is the classification data, and the output is an alarm notification message.

[0590] Step 10:

[0591] The user checks the alarm notification on the information communication device and selects whether to activate or cancel the alarm. The input is the alarm notification message, and the output is the user's selected action ("activate" or "cancel").

[0592] In this way, the system of the present invention achieves improved efficiency and security within the home through collaboration between sensors, cameras, generative AI models, display devices, information and communication devices, and servers.

[0593] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0594] The present invention is a system that automatically detects the degree of soiling of clothes in a washing machine and performs washing only when necessary, thereby reducing unnecessary washing and suggesting washing actions taking into account the user's emotions. This system mainly consists of the following elements.

[0595] 1. Dirt detection by sensor

[0596] The sensors installed inside the washing machine have the ability to detect the degree of dirt, color, and type of laundry in real time. When a user puts laundry into the washing machine, the sensors are automatically activated and start collecting data.

[0597] 2. Data transmission

[0598] The detected data is collected by a terminal inside the washing machine and then transmitted to a server over the internet, where the terminal uses a communication protocol to package and transmit the data efficiently.

[0599] 3. Analysis using AI models

[0600] The server launches a generative AI model to analyze the received data. Based on the learned data, the AI ​​model analyzes the degree of soiling of the clothes and determines whether they need to be washed. The server uses this model to generate the results of the data analysis. For example, it outputs classification results such as "lightly soiled," "moderately soiled," and "heavily soiled."

[0601] 4. Recognition of user emotions using an emotion engine

[0602] The emotion engine has the ability to recognize the user's emotions based on voice or image data acquired from the user's device. For example, the emotion engine analyzes data provided by the user through the smartphone's camera or microphone to determine the user's emotional state (stress, joy, calm, etc.).

[0603] 5. Notification of Results

[0604] The analysis results are sent to the washing machine terminal and the user's smartphone app. The terminal displays messages such as "No washing necessary" or "Washing required" on the washing machine panel. The user can check the detailed analysis results through the smartphone app.

[0605] 6. User Choice

[0606] Based on the analysis results, the user can choose whether to run or cancel the wash. For example, if the user receives a "no washing required" notification, the wash will not run unless the user presses the "start washing" button on the smartphone app or the washing machine's control panel. On the other hand, if the user presses the "start washing" button, the normal wash cycle will begin.

[0607] 7. Emotion-based recommendations

[0608] If the emotion engine recognizes that the user is in a stressful state, the system will suggest or automatically start doing laundry, regardless of the result of the laundry necessity judgment. This reduces the user's mental burden.

[0609] Specific examples

[0610] Case 1: Light soiling

[0611] The user places the laundry in the washing machine and closes the door.

[0612] The device activates a sensor to detect the degree of dirt on the clothes.

[0613] The device transmits the detected data to the server.

[0614] The server analyzes the data using the generated AI model and determines that the item is "slightly soiled and no washing is required."

[0615] The server notifies the device and the user's smartphone app of the analysis results.

[0616] The device will display "No washing required" on its panel, and a notification will also be sent to the user's smartphone app.

[0617] When the user recognizes through the emotion engine that they are in a stressful state, the system will suggest or automatically start doing laundry.

[0618] The device will start the wash and notify the user when it is finished.

[0619] Case 2: Moderate to severe soiling

[0620] The user places the laundry in the washing machine and closes the door.

[0621] The device activates a sensor to detect the degree of dirt on the clothes.

[0622] The device transmits the detected data to the server.

[0623] The server analyzes the data using the generated AI model and determines that the garment is "moderately soiled and requires washing."

[0624] The server notifies the device and the user's smartphone app of the analysis results.

[0625] The device will display "Washing required" on the panel, and a notification will also be sent to the user's smartphone app.

[0626] The user checks the notification on the smartphone app and selects to perform the laundry.

[0627] The device will begin a normal wash cycle.

[0628] As described above, the system of the present invention can improve the efficiency of laundry through interactions between sensors, servers, AI models, emotion engines, terminals, and users. This system is also environmentally friendly, as it reduces unnecessary laundry, contributes to saving water resources, and reduces household water bills. Furthermore, by suggesting laundry actions based on the user's emotional state, it also contributes to reducing mental burden.

[0629] The processing flow will be explained below.

[0630] Step 1:

[0631] The user loads laundry into the washing machine and closes the door, ready for the wash cycle to begin.

[0632] Step 2:

[0633] The device activates the sensors inside the washing machine, which detect the soiling level, color, and type of clothes in real time and collect data.

[0634] Step 3:

[0635] The device temporarily stores the data acquired from the sensor and then prepares it to be sent to a server via the Internet.

[0636] Step 4:

[0637] The device then sends the packaged data over the internet to a server, which includes details such as the soiling level, color, and type of clothing.

[0638] Step 5:

[0639] The server then launches a generative AI model to analyze the received data. Based on past learning data, the AI ​​model analyzes the degree of soiling of the clothes and determines whether they need to be washed.

[0640] Step 6:

[0641] The server generates the analysis results, which are displayed as classifications such as "lightly soiled," "moderately soiled," and "heavily soiled."

[0642] Step 7:

[0643] The analysis results generated by the server are sent to the device and the user's smartphone app.

[0644] Step 8:

[0645] The device displays the analysis results on the washing machine's control panel, such as a message like "No washing necessary" or "Washing required."

[0646] Step 9:

[0647] The user opens the smartphone app and checks the analysis results, which are displayed in detail.

[0648] Step 10:

[0649] To recognize the user's emotions, the emotion engine acquires voice or image data from the user's device and determines the user's emotional state (stress, joy, calm, etc.) based on this data.

[0650] Step 11:

[0651] If the user's emotion recognized by the emotion engine is a stress state, the system will suggest to the user to do laundry or automatically start laundry regardless of the result of the judgment on the necessity of laundry.

[0652] Step 12:

[0653] The user can choose whether to start or stop the wash by pressing the "Start Wash" or "Cancel" button on the smartphone app or the washing machine's control panel.

[0654] Step 13:

[0655] The device acts based on the user's selection. If the user selects "Start Wash," the device starts a normal wash cycle. If the user selects "Cancel," the device stops the wash and prompts the user to remove the clothes.

[0656] Step 14:

[0657] The user removes the clothes. If it is determined that washing is not necessary, the user removes the clothes and stores them. If washing is performed, the user removes the clothes after the washing is completed.

[0658] These are the specific processing steps from detecting dirt in a washing machine to starting or canceling the wash. This system can reduce unnecessary laundry, lighten the burden of housework, and conserve water resources. Furthermore, the emotion engine can take the user's emotions into consideration when suggesting laundry actions, thereby helping to reduce the user's mental burden.

[0659] Example 2

[0660] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0661] Conventional washing machine systems wash clothes without considering the degree of dirt or the user's emotional state, which has led to problems such as increased wasted laundry and mental stress on the user.In addition, there is a lack of systems that accurately determine the need for laundry, resulting in wasted water resources and energy.

[0662] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0663] In this invention, the server includes a means for detecting the degree of soiling of clothes using a sensor installed in the washing machine, a means for analyzing the detected data using an AI model generated by the server and determining whether the clothes need to be washed, and a means for recognizing the user's emotional state using an emotion engine and suggesting or automatically starting a washing action based on that. This reduces unnecessary laundry, eases the user's mental burden, and enables the saving of water resources and energy.

[0664] A "sensor" refers to a device installed in a washing machine that detects the degree of dirt, color, and type of clothes in real time.

[0665] "Server" refers to the central computer that uses the generated AI model to analyze the detected data and determine whether clothes need to be washed.

[0666] An "AI model" refers to an algorithm that is generated on a server, analyzes learned data, and determines the degree of dirtiness of clothing.

[0667] An "emotion engine" refers to software that analyzes a user's voice data or image data and recognizes the user's emotional state.

[0668] The "washing machine panel" refers to the display device attached to the washing machine itself, which notifies the user of the analysis results and system status.

[0669] "User's terminal" refers to a mobile device such as a smartphone or tablet used by the user, which can be used to check notifications and results from the system and perform operations.

[0670] "Analysis results" refers to the judgment results on the degree of dirt and the need for washing that are generated after the server analyzes the data using an AI model.

[0671] "Laundry behavior" refers to a series of actions that a washing machine actually takes to wash clothes based on the analysis results and the user's emotional state.

[0672] This invention is a system that automatically detects soiling of clothes in a washing machine and performs washing only when necessary, thereby reducing unnecessary washing and suggesting washing actions taking into account the user's emotions. This system is realized using sensors installed in the washing machine, a server, a generative AI model, an emotion engine, a user's device, and communication means via the Internet.

[0673] First, the user puts laundry into the washing machine and closes the door. At this time, the device activates the sensors inside the washing machine. The sensors detect the degree of dirt, color, and type of the clothes in real time and collect that data. Specific examples of sensors include optical sensors for detecting dirt and color identification sensors.

[0674] The detected data is collected by the device and then transmitted to a server via the Internet. The device uses a communication protocol to efficiently transmit the data, such as HTTP or MQTT.

[0675] The server launches a generative AI model based on the received data. The generative AI model includes an algorithm that analyzes the degree of soiling of the clothes based on the learned data and determines whether they need to be washed. The analysis results are output as classifications such as "lightly soiled," "moderately soiled," or "heavily soiled."

[0676] At the same time, the voice or image data provided by the user using the smartphone's camera or microphone is analyzed by the emotion engine on the server, which determines the user's emotional state (stress, joy, calm, etc.).

[0677] The analysis results are then sent back to the device and the user's smartphone app. The device displays a message such as "No washing necessary" or "Washing necessary" on the washing machine's panel, and a notification is also sent to the user's smartphone app. For example, if "Washing unnecessary" is displayed, the user must press the "Start washing" button for the washing to begin.

[0678] Furthermore, if the emotion engine recognizes the user's emotion as stressed, the server will send the user a notification suggesting that they do laundry, regardless of the result of the judgment on the necessity of doing laundry. Even if the user ignores the suggestion, the system will automatically start the laundry. In this way, the laundry is carried out, and when it is finished, the terminal will send a completion notification to the user.

[0679] Specific examples

[0680] Case 1: Light soiling

[0681] 1. The user loads laundry into the washing machine and closes the door.

[0682] 2. The device activates the sensor and detects the degree of dirt on the clothes.

[0683] 3. The device sends the detection data to the server.

[0684] 4. The server analyzes the data using the generated AI model and determines that the item is "slightly soiled and no washing is required."

[0685] 5. The server notifies the device and the user's smartphone app of the analysis results.

[0686] 6. The device will display "No washing required" on the panel, and a notification will also be sent to the user's smartphone app.

[0687] 7. When the user recognizes through the emotion engine that they are in a stressful state, the system will suggest or automatically start doing laundry.

[0688] 8. The device starts the wash and notifies the user when it is finished.

[0689] Case 2: Moderate to severe soiling

[0690] 1. The user loads laundry into the washing machine and closes the door.

[0691] 2. The device activates the sensor and detects the degree of dirt on the clothes.

[0692] 3. The device sends the detection data to the server.

[0693] 4. The server analyzes the data using the generated AI model and determines that the item is "moderately soiled and requires washing."

[0694] 5. The server notifies the device and the user's smartphone app of the analysis results.

[0695] 6. The device will display "Laundry required" on the panel, and a notification will also be sent to the user's smartphone app.

[0696] 7. The user checks the notification on the smartphone app and selects to perform the laundry.

[0697] 8. The device will begin a normal wash cycle.

[0698] The system leverages generative AI models and an emotion engine to provide an efficient and user-friendly laundry experience.

[0699] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0700] Step 1:

[0701] The user places laundry in the washing machine and closes the door. This action automatically activates the device's built-in sensors, which detect the laundry's soiling level, color, and type in real time and collect the data. The input is the laundry, and the output is data on soiling level, color, and type.

[0702] Step 2:

[0703] The terminal first records the collected data in its internal memory and then transmits it to a server via the Internet. The terminal uses communication protocols such as HTTP and MQTT to transmit data efficiently. The input is data from the sensor, and the output is packaged data.

[0704] Step 3:

[0705] The server stores the received data in a database. Next, it launches a generative AI model and performs analysis based on the stored data. The AI ​​model evaluates the level of dirt through shading analysis and color identification, and outputs a classification result such as "lightly dirty," "moderately dirty," or "heavily dirty." The input is packaged data, and the output is the classification result of the level of dirt.

[0706] Step 4:

[0707] The user provides voice or image data using the camera or microphone on their smartphone. This data is sent to a server via the Internet and analyzed by an emotion engine. The emotion engine determines the user's emotional state (stress, joy, calm, etc.) and sends the results to the server. The input is voice or image data, and the output is the determined emotional state.

[0708] Step 5:

[0709] The server sends the analysis results and the emotional state determination results to the washing machine device and the user's smartphone app. The device displays a message such as "No washing necessary" or "Laundry required" on the washing machine panel, and a notification is also sent to the user's smartphone app. The input is the analysis results and the emotional state determination results, and the output is a notification message.

[0710] Step 6:

[0711] The user selects whether to press the "Start Wash" button on the smartphone app or the washing machine's control panel based on the notification of the laundry need. If the user selects to run the laundry, the device starts a normal wash cycle. The input is the user's selection, and the output is running the laundry.

[0712] Step 7:

[0713] If the emotion engine recognizes the user's emotion as stressful, the server sends a notification to the user suggesting laundry actions regardless of the result of the judgment on the necessity of laundry. Even if the suggestion is ignored, the system will automatically start the laundry. The input is the emotion analysis result, and the output is the laundry action suggestion or automatic execution.

[0714] Step 8:

[0715] When the wash is finished, the device sends a completion notification to the user. The notification is displayed on the washing machine panel and on the user's smartphone app. The input is the end status of the wash cycle, and the output is the completion notification.

[0716] In this way, it is possible to reduce wasted laundry and suggest laundry actions that take into account the user's emotional state.

[0717] (Application example 2)

[0718] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0719] On factory production lines, contamination of products and parts has a significant impact on shipping quality, but manually detecting and cleaning contamination is time-consuming and laborious. Furthermore, depending on the emotional state of the operator, it can be difficult to make appropriate decisions. Therefore, a system is needed that automatically detects contamination on product lines and makes decisions about cleaning or halting shipments as necessary. Furthermore, a system is needed that reduces the mental burden by making appropriate suggestions and taking appropriate action, taking into account the emotional state of the operator.

[0720] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0721] In this invention, the server includes means for detecting the degree of soiling of clothes using a sensor installed in the washing machine, means for analyzing the detected data using an AI model generated by the server and determining whether the clothes need to be washed, means for notifying the washing machine panel and the user's terminal of the result that the clothes do not need to be washed, means for the user to choose whether to perform or cancel the washing, means for automatically detecting the contamination state of the product line and parts, means for determining whether to perform cleaning work or stop shipping based on the contamination state, and means for recognizing the user's emotional state and suggesting cleaning or shipping. This makes it possible to automatically detect soiling and take necessary measures, while also reducing the burden on the operator.

[0722] A "sensor" is a device that measures physical or chemical quantities and outputs them as data. Optical sensors and image recognition sensors are particularly used.

[0723] A "server" is a computer system that provides services such as data storage, processing, and distribution over a network.

[0724] An "AI model" is an algorithm that has been trained to perform a specific task using artificial intelligence techniques, particularly frameworks such as TensorFlow and PyTorch.

[0725] A "panel" is an interface for displaying information. It is installed in washing machines and other appliances.

[0726] A "user's terminal" is a device used by a user, such as a computer or smartphone.

[0727] A "product line" refers to the equipment and facilities used to continuously produce products within a factory.

[0728] "Stain level" refers to the degree of foreign matter adhering to the surface of an object and the resulting discoloration.

[0729] "Data analysis" is the process of analyzing collected data and extracting meaningful information.

[0730] A "cleaning operation" is a manual or automated process for removing dirt or foreign matter.

[0731] "Shipment suspension" means stopping the shipment of a product, and is done to ensure quality.

[0732] "Emotional state" refers to the current psychological state of the user or operator.

[0733] The present invention is a system that uses a robot installed on a factory production line to automatically detect dirt on products and parts, and based on the results, decides whether to perform cleaning work or stop product shipments. This system is composed of the following elements.

[0734] 1. Dirt detection by sensor

[0735] The robot is equipped with optical sensors and image recognition sensors that constantly monitor the production line and detect the degree of dirt on products and parts. The data detected by the sensors is temporarily collected on a terminal inside the robot.

[0736] 2. Data transmission

[0737] The collected data is sent to a cloud server via Wi-Fi or a wired connection, and the device uses a communication protocol to package and efficiently transmit the data.

[0738] 3. Analysis using AI models

[0739] The server uses a generative AI model to analyze the data it receives, specifically an AI model based on frameworks such as TensorFlow and PyTorch, to analyze the data and classify the level of dirt into categories such as "lightly dirty," "moderately dirty," and "heavily dirty."

[0740] 4. Recognition of user emotions using an emotion engine

[0741] An emotion engine is used to recognize the emotional state of the operator through a smartphone app or computer. If the operator is under stress or overload, the emotion engine will detect this.

[0742] 5. Notification of Results

[0743] The analysis results are sent to the robot's terminal and the operator's smartphone or computer. The terminal displays messages such as "Cleaning required" or "Shipping suspended" on the robot's panel, and the same information is also sent to the operator's terminal.

[0744] 6. User Selection and Autorun

[0745] Based on the analysis results, the operator can choose to carry out cleaning work, cancel the work, or stop shipping. If the operator's emotional state is stressed, the system will automatically carry out cleaning work or stop shipping.

[0746] This system will automatically detect contamination and take necessary measures, and is expected to reduce the burden and mental stress on operators.

[0747] As a concrete example, you can use the following prompt to have an AI model perform an analysis:

[0748] It judges the degree of soiling of the product line based on images, and starts cleaning if the soiling score exceeds the previous threshold, or skips cleaning if it does not. It also starts cleaning if the operator's emotional state is stressed or overloaded.

[0749] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0750] Step 1:

[0751] Dirt detection by sensors

[0752] Optical sensors and image recognition sensors installed on the robot detect the degree of dirt on products and parts on the production line. The input is the physical condition of the product or part, and the output is a measurement value of the degree of dirt as digital data. This measurement value is temporarily stored on a terminal inside the robot.

[0753] Step 2:

[0754] Sending data

[0755] The device transmits the collected soiling data to a cloud server via Wi-Fi or a wired connection. The input is the measurement data stored in the device, and the output is the data transferred to the cloud server. The device packages the data for efficient transmission.

[0756] Step 3:

[0757] Analysis using AI models

[0758] The server analyzes the received data using a generative AI model. Specifically, it uses a pre-trained model based on TensorFlow and PyTorch to classify the degree of dirt into categories such as "lightly dirty," "moderately dirty," and "heavily dirty." The input is the digital data of the dirt level sent to the cloud server, and the output is the classification result.

[0759] Step 4:

[0760] Recognizing user emotions with an emotion engine

[0761] The emotion engine recognizes the operator's emotional state through the user's smartphone app or computer. The input is voice data and image data provided by the operator, and the output is the judgment result of the operator's emotional state. The emotion engine uses technologies such as OpenVINO.

[0762] Step 5:

[0763] Notification of analysis results and emotional state

[0764] The server notifies the robot's terminal and the operator's smartphone or computer of the analysis results and emotional state. The input is the classification results of the dirt level analyzed by the server and the judgment results of the emotion engine, and the output is a notification message. The robot's panel displays messages such as "Cleaning required" or "Shipping suspended."

[0765] Step 6:

[0766] User selection and automatic execution

[0767] Based on the notified analysis results, the operator can choose whether to carry out cleaning work or stop shipments. The input is the notification message and the operator's choice, and the output is the start of cleaning work or the stop of shipments. If the operator's emotional state is stressed, cleaning work or shipments will be automatically stopped.

[0768] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0769] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0770] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0771] [Third embodiment]

[0772] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0773] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0774] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0775] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0776] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0777] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0778] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0779] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0780] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0781] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0782] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0783] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0784] The present invention is a system that automatically detects the degree of soiling of clothes in a washing machine and performs washing only when necessary, thereby reducing unnecessary washing. This system mainly consists of the following elements.

[0785] 1. Dirt detection by sensor

[0786] The sensors installed in the washing machine detect the degree of soiling, color, and type of laundry in real time. These sensors can be optical or chemical sensors, for example. When a user puts laundry into the washing machine, the sensors are automatically activated and start collecting data.

[0787] 2. Data transmission

[0788] The sensed data is collected by a terminal inside the washing machine and then transmitted over the internet to a server, which uses a communications protocol to package and efficiently transmit the data.

[0789] 3. Analysis using AI models

[0790] The server launches a generative AI model to analyze the received data. This AI model learns from past data and can accurately determine whether clothes need to be washed. The server uses this model to generate data analysis results. For example, it outputs classification results such as "lightly soiled," "moderately soiled," and "heavily soiled."

[0791] 4. Notification of Results

[0792] The analysis results are sent to the washing machine terminal and the user's smartphone app. The terminal displays messages such as "No washing necessary" or "Washing required" on the washing machine panel. The user can check the detailed analysis results through the smartphone app.

[0793] 5. User Choice

[0794] Based on the analysis results, the user can choose whether to run or cancel the wash. For example, if the user receives a "no washing required" notification, the wash will not run unless the user presses the "start washing" button on the smartphone app or the washing machine's control panel. On the other hand, if the user presses the "start washing" button, the normal wash cycle will begin.

[0795] Specific examples

[0796] Case 1: Light soiling

[0797] The user puts laundry into the washing machine.

[0798] The device activates a sensor to detect the degree of dirt on the clothes.

[0799] The device transmits the detected data to the server.

[0800] The server analyzes the data using the generated AI model and determines that the item is "slightly soiled and no washing is required."

[0801] The server notifies the device and the user's smartphone app of the analysis results.

[0802] The device will display "No washing required" on its panel, and a notification will also be sent to the user's smartphone app.

[0803] The user checks the notification on the smartphone app and chooses to cancel the wash.

[0804] The device will stop the wash and prompt the user to remove the clothes.

[0805] Case 2: Moderate to severe soiling

[0806] The user puts laundry into the washing machine.

[0807] The device activates a sensor to detect the degree of dirt on the clothes.

[0808] The device transmits the detected data to the server.

[0809] The server analyzes the data using the generated AI model and determines that the garment is "moderately soiled and requires washing."

[0810] The server notifies the device and the user's smartphone app of the analysis results.

[0811] The device will display "Washing required" on the panel, and a notification will also be sent to the user's smartphone app.

[0812] The user checks the notification on the smartphone app and selects to perform the laundry.

[0813] The device will begin a normal wash cycle.

[0814] As described above, the system of the present invention can improve the efficiency of laundry through the interaction between sensors, servers, AI models, terminals, and users. This system also reduces unnecessary laundry, contributes to saving water resources, and reduces household water bills, making it environmentally friendly.

[0815] The processing flow will be explained below.

[0816] Step 1:

[0817] The user loads laundry into the washing machine and closes the door, ready for the wash cycle to begin.

[0818] Step 2:

[0819] The device activates the sensors inside the washing machine, which detect the soiling level, color, and type of clothes in real time and collect data.

[0820] Step 3:

[0821] The device temporarily stores the data acquired from the sensor and then prepares it to be sent to a server via the Internet.

[0822] Step 4:

[0823] The device then sends the packaged data over the internet to a server, which includes details such as the degree of dirt, color, and type.

[0824] Step 5:

[0825] The server then launches a generative AI model to analyze the received data. Based on the learned data, the AI ​​model analyzes the degree of soiling of the clothes and determines whether they need to be washed.

[0826] Step 6:

[0827] The server generates the analysis results, which are displayed as classifications such as "lightly soiled," "moderately soiled," and "heavily soiled."

[0828] Step 7:

[0829] The analysis results generated by the server are sent to the device and the user's smartphone app.

[0830] Step 8:

[0831] The device displays the analysis results on the washing machine's control panel, such as a message like "No washing necessary" or "Washing required."

[0832] Step 9:

[0833] The user opens the smartphone app and checks the analysis results, which are displayed in detail.

[0834] Step 10:

[0835] The user can choose whether to start or stop the wash by pressing the "Start Wash" or "Cancel" button on the smartphone app or the washing machine's control panel.

[0836] Step 11:

[0837] The device acts based on the user's selection. If the user selects "Start Wash," the device starts a normal wash cycle. If the user selects "Cancel," the device stops the wash and prompts the user to remove the clothes.

[0838] Step 12:

[0839] The user removes the clothes. If it is determined that washing is not necessary, the user removes the clothes and stores them. If washing is performed, the user removes the clothes after the washing is completed.

[0840] These are the specific steps in the washing machine's process, from detecting dirt to starting or canceling the wash. This system reduces unnecessary washing, lightens the burden of housework, and conserves water resources.

[0841] Example 1

[0842] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0843] Conventional washing machines have difficulty accurately detecting the degree of soiling of laundry, resulting in unnecessary washing. This results in wasted water and electricity consumption, placing a heavy burden on the environment. Another issue is that users must check the degree of soiling of their clothes and select the appropriate washing method every time they wash, which is time-consuming.

[0844] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0845] In this invention, the server includes: means for detecting the degree of soiling of clothes using a sensor installed in the washing machine; means for transmitting the detected data to the server via the Internet via a terminal; means for analyzing the received data using an AI model generated by the server and determining whether the clothes need to be washed; means for notifying the user of the analysis results via the washing machine's operation panel and the user's terminal; and means for the user to select whether to perform or cancel the wash based on the analysis results. This reduces unnecessary washing and prevents the wasteful consumption of water resources and electricity. It also allows users to wash clothes efficiently without hassle, contributing to environmental protection.

[0846] "Sensor" refers to a device installed in a washing machine that detects the degree of soiling, color, type, etc. of clothes. This includes optical sensors and chemical sensors.

[0847] A "terminal" is a device that is placed inside the washing machine and is responsible for collecting data from the sensors and sending it to the server.

[0848] A "server" is a computer system that receives and analyzes data sent from a terminal via the Internet.

[0849] The "AI model" is a machine learning model that runs on a server and is used to analyze received data and determine whether clothes need to be washed.

[0850] The "operation panel" is an interface device that is provided in the washing machine and that displays the analysis results to the user.

[0851] The "means of notifying the terminal" is a communication means for transmitting the analysis results from the server to the terminal. It uses the Internet Protocol.

[0852] A "user's device" is a communication-enabled device such as a smartphone or tablet held by a user, and is responsible for receiving and displaying notifications from the server.

[0853] "Means by which the user can choose whether to start or stop washing based on the analysis results" refers to an interface that allows the user to check the analysis results through the operation panel or the user's terminal and choose whether to start or stop washing.

[0854] The present invention is a system that automatically detects the degree of soiling of clothes in a washing machine and performs washing as necessary, thereby reducing unnecessary washing. This system is mainly composed of sensors, terminals, a server, a generative AI model, and a user smartphone app. Specific embodiments of the system are described below.

[0855] 1. Sensor activation and data collection

[0856] When a user loads laundry into the washing machine, the device inside the washing machine automatically activates sensors as soon as the lid is closed. The sensors use optical and chemical sensors to detect data such as the soiling level, color, and type of clothes. Optical sensors scan the surface of the clothes and measure the intensity and wavelength of reflected light. Chemical sensors detect specific pollutants, such as volatile organic compounds (VOCs) and specific stains. The detected data is collected and stored locally on the device.

[0857] 2. Data transmission

[0858] The device packages the collected data and sends it to a server via the Internet. The data is packaged in a specified format (for example, JSON format) and includes information such as the degree of dirt, color, and type. HTTP / HTTPS is generally used as the transmission protocol. For example, the endpoint URL is "https: / / example.com / api / wash-data".

[0859] 3. Data Analysis

[0860] The server launches a generative AI model to analyze the received data. The AI ​​model is implemented using frameworks such as TensorFlow and PyTorch. This AI model learns from past data and can accurately determine whether clothes need to be washed. The generative AI model analyzes the data and outputs classification results, such as "lightly soiled," "moderately soiled," or "heavily soiled." The analysis results are stored on the server.

[0861] 4. Result notification

[0862] The server notifies the device inside the washing machine and the user's smartphone app of the analysis results. Notifications are sent via push notifications and REST API calls. The smartphone app receives a notification such as "Lightly soiled, no need to wash," and the washing machine's control panel displays a message based on the analysis results.

[0863] 5. User Choice

[0864] Based on the analysis results, the user can choose whether to run or cancel the wash. They can receive a notification via the smartphone app and check the details. The user decides whether to run the wash by choosing whether to press the "Start Wash" button. If "washing is not necessary," the wash will not be run if the user does nothing. On the other hand, if it is determined that "washing is necessary," the device will start a normal wash cycle when the user presses the "Start Wash" button. The user will be notified by a voice message or other means when the cycle starts.

[0865] Specific examples

[0866] Case 1: Light soiling

[0867] The user puts laundry into the washing machine.

[0868] The device activates a sensor to detect the degree of dirt on the clothes.

[0869] The device transmits the detected data to the server.

[0870] The server analyzes the data using the generated AI model and determines that the item is "slightly soiled and no washing is required."

[0871] The server notifies the device and the user's smartphone app of the analysis results.

[0872] The device will display "No washing required" on the operation panel, and a notification will also be sent to the user's smartphone app.

[0873] The user checks the notification on the smartphone app and chooses to cancel the wash.

[0874] The device will stop the wash and prompt the user to remove the clothes.

[0875] Case 2: Moderate to severe soiling

[0876] The user puts laundry into the washing machine.

[0877] The device activates a sensor to detect the degree of dirt on the clothes.

[0878] The device transmits the detected data to the server.

[0879] The server analyzes the data using the generated AI model and determines that the garment is "moderately soiled and requires washing."

[0880] The server notifies the device and the user's smartphone app of the analysis results.

[0881] The device will display "Washing required" on the operation panel, and a notification will also be sent to the user's smartphone app.

[0882] The user checks the notification on the smartphone app and selects to perform the laundry.

[0883] The device will begin a normal wash cycle.

[0884] This reduces unnecessary washing and prevents the wasteful consumption of water and electricity. It also allows users to wash their clothes efficiently and hassle-free, contributing to environmental protection.

[0885] Prompt Sentence Examples

[0886] "Please create a prompt for a system that detects the degree of soiling of laundry and performs the washing if necessary."

[0887] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0888] Step 1:

[0889] Sensor activation and data collection

[0890] A user loads laundry into the washing machine and closes the lid. This action automatically activates sensors on a device inside the washing machine. The sensors use optical and chemical sensors to detect the soiling level of the clothes. Specifically, optical sensors scan the surface of the clothes and measure the intensity and wavelength of reflected light. Chemical sensors detect volatile organic compounds (VOCs) and specific stains. The collected data is then sent directly to the device and stored locally.

[0891] Input: Load laundry and close lid

[0892] Output: Detection data such as dirt level, color, type, etc.

[0893] Step 2:

[0894] Sending data

[0895] The device packages the data collected from the sensor. The data is packaged in a specified format such as JSON. For example, the data may include information such as the degree of dirt, color, and type. The device then transmits the data to a server via the Internet. HTTP / HTTPS is generally used as the transmission protocol.

[0896] Input: Detection data from sensors

[0897] Output: Packaged data sent to the server

[0898] Step 3:

[0899] Data analysis

[0900] The server launches a generative AI model to analyze the received data. The server then analyzes the data using an AI model implemented using frameworks such as TensorFlow or PyTorch. The generative AI model evaluates the degree of soiling of the clothes based on the received data and outputs a classification result such as "lightly soiled," "moderately soiled," or "heavily soiled." The analysis results are stored on the server.

[0901] Input: Data package sent from the terminal

[0902] Output: Analysis results (classified data: mild, mild, moderate, severe)

[0903] Step 4:

[0904] Result notification

[0905] The server notifies the washing machine device and the user's smartphone app of the analysis results. Notification methods include push notifications and REST API calls. This allows the user's smartphone app to receive specific notifications, such as "Lightly soiled, no need to wash." The results are also displayed on the washing machine's control panel.

[0906] Input: Analysis results of the AI ​​model

[0907] Output: Notifications to the device and the user's smartphone app

[0908] Step 5:

[0909] User Selection

[0910] The user checks the analysis results through the smartphone app or the washing machine's operation panel, and can then choose whether to start or stop the wash. If the notification is "lightly soiled, no washing necessary," the wash will not be carried out unless the user presses the "start washing" button. On the other hand, if the notification is "moderately soiled, washing required," the wash will begin if the user presses the "start washing" button on the smartphone app or operation panel.

[0911] Input: User selection via smartphone app or operation panel

[0912] Output: Decision to do or not do laundry

[0913] Step 6:

[0914] Starting the wash cycle

[0915] If the user decides to wash the clothes, the device will start the washing machine's normal wash cycle, and the user will be notified when the wash cycle starts, for example, through a voice message or a display on the operation panel.

[0916] Input: User's laundry run selection

[0917] Output: Start and run a normal wash cycle

[0918] (Application example 1)

[0919] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0920] The present invention relates to a system for reducing unnecessary laundry and false alarms in homes and other environments. Specifically, the system detects the degree of soiling of clothes and abnormalities in the home with high accuracy, and performs laundry and issues alarms only when necessary, thereby preventing waste of resources and false alarms.

[0921] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0922] In this invention, the server includes a means for detecting the degree of soiling of clothes using a sensor installed in the washing machine, a means for analyzing the detected data using an AI model generated by the server and determining whether the clothes need to be washed, and a means for detecting abnormalities in the home using cameras and chemical sensors installed in the home. This enables the system to automatically detect the degree of soiling of clothes and abnormal conditions in the home and notify the user, thereby reducing unnecessary washing and warnings.

[0923] A "sensor" is a device that senses physical or chemical quantities, converts them into electrical signals, and outputs them.

[0924] An "AI model" is an artificial intelligence algorithm designed to learn from past data and analyze new data to make decisions.

[0925] A "server" refers to equipment or software used to store, process, and analyze data on a network.

[0926] A "washing machine" is a mechanical device for washing clothes.

[0927] A "display device" is a device for visually displaying information, and includes, for example, a display or a panel.

[0928] An "information communication device" is a device for sending and receiving data and information over a network, and includes, for example, smartphones and tablets.

[0929] A "camera" is a device that takes pictures and videos and records them as data.

[0930] A "chemical sensor" is a device that detects specific chemical substances and outputs their presence and concentration as an electrical signal.

[0931] "Abnormal" refers to an event or situation that is different from normal, such as suspicious movements or sounds, or gas leaks.

[0932] An "alarm" refers to a system or function that uses sound, light, or other means to draw attention when an abnormality occurs.

[0933] This invention is a system for efficient home laundry and security management. The system consists of a washing machine, sensors, an AI model, a display device, an information and communication device, a camera, and a chemical sensor.

[0934] First, sensors installed in the washing machine detect the degree of soiling of the clothes put in. Specifically, optical sensors or chemical sensors are used. The data collected by the sensors is temporarily stored on a terminal inside the washing machine and then sent to a server via the Internet.

[0935] The server launches a generative AI model to analyze the received data. This AI model learns from past data and accurately determines whether clothes need to be washed. For example, it can classify clothes into categories such as "lightly soiled," "moderately soiled," and "heavily soiled." The analysis results are notified to the washing machine's display and the user's information and communication device.

[0936] Based on these notifications, the user can choose to either run or cancel the laundry. If the analysis results indicate that washing is not necessary, the user can cancel the laundry and, if necessary, run the laundry manually.

[0937] In addition, cameras and chemical sensors installed in the home will detect abnormalities, such as suspicious movements, sounds, and gas leaks. If these sensors detect an abnormality, the data will also be sent to the server. The generated AI model will analyze the severity of the abnormality and output a classification result such as "mild abnormality," "moderate abnormality," or "severe abnormality." This result will also be notified to the display device in the home and the user's information and communication device.

[0938] The user can choose to issue or cancel an alert based on this notification. If the analysis result is judged to be a "serious abnormality," the user is advised to take immediate action to address the issue.

[0939] For example, if a home camera detects abnormal activity, the data is sent to a server and analyzed by an AI model. If the analysis results in a "severe abnormality," a notification is sent to the user's smartphone stating, "Suspicious activity has been detected!" The user can then choose to activate or cancel the alarm via the smartphone app.

[0940] An example of a specific prompt is:

[0941] An abnormality was detected on a home camera at 16:45. After analyzing the video, it was determined that a "serious abnormality" had occurred. Please check immediately.

[0942] There are messages like this.

[0943] The hardware used includes washing machines, cameras, sensors, displays, and information and communication devices. The software includes OpenCV, Requests, and a generative AI model. The generative AI model is implemented using Python's TensorFlow and PyTorch.

[0944] With this configuration, the present invention can reduce wasted laundry and the occurrence of false alarms, contributing to improved efficiency and security within the home.

[0945] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0946] Step 1:

[0947] The device activates sensors installed in the washing machine to detect the degree of soiling of the clothes placed in it. Optical and chemical sensors are used for this purpose. The input is the physical data of the clothes, and the output is numerical data on the degree of soiling.

[0948] Step 2:

[0949] The device collects data on the level of dirt detected by the sensor and sends it to a server via the Internet. The input is numerical data on the level of dirt, and the output is packetized data.

[0950] Step 3:

[0951] The server launches a generative AI model to analyze the received soiling data. Because this model is trained based on a past database, it can analyze new data with high accuracy. The input is numerical data on the soiling level, and the output is classification data such as "light soiling," "moderate soiling," or "heavy soiling."

[0952] Step 4:

[0953] The server notifies the analysis results to the display device of the washing machine and the user's information communication device. The input is the classification data, and the output is a notification message.

[0954] Step 5:

[0955] The user checks the notification on the information communication device and selects whether to run or cancel the laundry. The input is the notification message, and the output is the user's selected action ("run" or "cancel").

[0956] Step 6:

[0957] Cameras and chemical sensors installed in the home detect abnormalities, such as suspicious movements, sounds, gas leaks, etc. The input is environmental data within the home, and the output is numerical data on abnormalities.

[0958] Step 7:

[0959] Data obtained from cameras and sensors is also collected by the terminal and sent to the server via the Internet. The input is numerical data of anomalies, and the output is packetized data.

[0960] Step 8:

[0961] The server then launches the generative AI model again to analyze the received anomaly data. The input is the numerical data of the anomaly, and the output is classification data such as "mild anomaly," "moderate anomaly," or "severe anomaly."

[0962] Step 9:

[0963] The server notifies the user of the results of the anomaly analysis to the display device in the home and the user's information and communication device. The input is the classification data, and the output is an alarm notification message.

[0964] Step 10:

[0965] The user checks the alarm notification on the information communication device and selects whether to activate or cancel the alarm. The input is the alarm notification message, and the output is the user's selected action ("activate" or "cancel").

[0966] In this way, the system of the present invention achieves improved efficiency and security within the home through collaboration between sensors, cameras, generative AI models, display devices, information and communication devices, and servers.

[0967] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0968] The present invention is a system that automatically detects the degree of soiling of clothes in a washing machine and performs washing only when necessary, thereby reducing unnecessary washing and suggesting washing actions taking into account the user's emotions. This system mainly consists of the following elements.

[0969] 1. Dirt detection by sensor

[0970] The sensors installed inside the washing machine have the ability to detect the degree of dirt, color, and type of laundry in real time. When a user puts laundry into the washing machine, the sensors are automatically activated and start collecting data.

[0971] 2. Data transmission

[0972] The detected data is collected by a terminal inside the washing machine and then transmitted to a server over the internet, where the terminal uses a communication protocol to package and transmit the data efficiently.

[0973] 3. Analysis using AI models

[0974] The server launches a generative AI model to analyze the received data. Based on the learned data, the AI ​​model analyzes the degree of soiling of the clothes and determines whether they need to be washed. The server uses this model to generate the results of the data analysis. For example, it outputs classification results such as "lightly soiled," "moderately soiled," and "heavily soiled."

[0975] 4. Recognition of user emotions using an emotion engine

[0976] The emotion engine has the ability to recognize the user's emotions based on voice or image data acquired from the user's device. For example, the emotion engine analyzes data provided by the user through the smartphone's camera or microphone to determine the user's emotional state (stress, joy, calm, etc.).

[0977] 5. Notification of Results

[0978] The analysis results are sent to the washing machine terminal and the user's smartphone app. The terminal displays messages such as "No washing necessary" or "Washing required" on the washing machine panel. The user can check the detailed analysis results through the smartphone app.

[0979] 6. User Choice

[0980] Based on the analysis results, the user can choose whether to run or cancel the wash. For example, if the user receives a "no washing required" notification, the wash will not run unless the user presses the "start washing" button on the smartphone app or the washing machine's control panel. On the other hand, if the user presses the "start washing" button, the normal wash cycle will begin.

[0981] 7. Emotion-based recommendations

[0982] If the emotion engine recognizes that the user is in a stressful state, the system will suggest or automatically start doing laundry, regardless of the result of the laundry necessity judgment. This reduces the user's mental burden.

[0983] Specific examples

[0984] Case 1: Light soiling

[0985] The user places the laundry in the washing machine and closes the door.

[0986] The device activates a sensor to detect the degree of dirt on the clothes.

[0987] The device transmits the detected data to the server.

[0988] The server analyzes the data using the generated AI model and determines that the item is "slightly soiled and no washing is required."

[0989] The server notifies the device and the user's smartphone app of the analysis results.

[0990] The device will display "No washing required" on its panel, and a notification will also be sent to the user's smartphone app.

[0991] When the user recognizes through the emotion engine that they are in a stressful state, the system will suggest or automatically start doing laundry.

[0992] The device will start the wash and notify the user when it is finished.

[0993] Case 2: Moderate to severe soiling

[0994] The user places the laundry in the washing machine and closes the door.

[0995] The device activates a sensor to detect the degree of dirt on the clothes.

[0996] The device transmits the detected data to the server.

[0997] The server analyzes the data using the generated AI model and determines that the garment is "moderately soiled and requires washing."

[0998] The server notifies the device and the user's smartphone app of the analysis results.

[0999] The device will display "Washing required" on the panel, and a notification will also be sent to the user's smartphone app.

[1000] The user checks the notification on the smartphone app and selects to perform the laundry.

[1001] The device will begin a normal wash cycle.

[1002] As described above, the system of the present invention can improve the efficiency of laundry through interactions between sensors, servers, AI models, emotion engines, terminals, and users. This system is also environmentally friendly, as it reduces unnecessary laundry, contributes to saving water resources, and reduces household water bills. Furthermore, by suggesting laundry actions based on the user's emotional state, it also contributes to reducing mental burden.

[1003] The processing flow will be explained below.

[1004] Step 1:

[1005] The user loads laundry into the washing machine and closes the door, ready for the wash cycle to begin.

[1006] Step 2:

[1007] The device activates the sensors inside the washing machine, which detect the soiling level, color, and type of clothes in real time and collect data.

[1008] Step 3:

[1009] The device temporarily stores the data acquired from the sensor and then prepares it to be sent to a server via the Internet.

[1010] Step 4:

[1011] The device then sends the packaged data over the internet to a server, which includes details such as the soiling level, color, and type of clothing.

[1012] Step 5:

[1013] The server then launches a generative AI model to analyze the received data. Based on past learning data, the AI ​​model analyzes the degree of soiling of the clothes and determines whether they need to be washed.

[1014] Step 6:

[1015] The server generates the analysis results, which are displayed as classifications such as "lightly soiled," "moderately soiled," and "heavily soiled."

[1016] Step 7:

[1017] The analysis results generated by the server are sent to the device and the user's smartphone app.

[1018] Step 8:

[1019] The device displays the analysis results on the washing machine's control panel, such as a message like "No washing necessary" or "Washing required."

[1020] Step 9:

[1021] The user opens the smartphone app and checks the analysis results, which are displayed in detail.

[1022] Step 10:

[1023] To recognize the user's emotions, the emotion engine acquires voice or image data from the user's device and determines the user's emotional state (stress, joy, calm, etc.) based on this data.

[1024] Step 11:

[1025] If the user's emotion recognized by the emotion engine is a stress state, the system will suggest to the user to do laundry or automatically start laundry regardless of the result of the judgment on the necessity of laundry.

[1026] Step 12:

[1027] The user can choose whether to start or stop the wash by pressing the "Start Wash" or "Cancel" button on the smartphone app or the washing machine's control panel.

[1028] Step 13:

[1029] The device acts based on the user's selection. If the user selects "Start Wash," the device starts a normal wash cycle. If the user selects "Cancel," the device stops the wash and prompts the user to remove the clothes.

[1030] Step 14:

[1031] The user removes the clothes. If it is determined that washing is not necessary, the user removes the clothes and stores them. If washing is performed, the user removes the clothes after the washing is completed.

[1032] These are the specific processing steps from detecting dirt in a washing machine to starting or canceling the wash. This system can reduce unnecessary laundry, lighten the burden of housework, and conserve water resources. Furthermore, the emotion engine can take the user's emotions into consideration when suggesting laundry actions, thereby helping to reduce the user's mental burden.

[1033] Example 2

[1034] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1035] Conventional washing machine systems wash clothes without considering the degree of dirt or the user's emotional state, which has led to problems such as increased wasted laundry and mental stress on the user.In addition, there is a lack of systems that accurately determine the need for laundry, resulting in wasted water resources and energy.

[1036] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1037] In this invention, the server includes a means for detecting the degree of soiling of clothes using a sensor installed in the washing machine, a means for analyzing the detected data using an AI model generated by the server and determining whether the clothes need to be washed, and a means for recognizing the user's emotional state using an emotion engine and suggesting or automatically starting a washing action based on that. This reduces unnecessary laundry, eases the user's mental burden, and enables the saving of water resources and energy.

[1038] A "sensor" refers to a device installed in a washing machine that detects the degree of dirt, color, and type of clothes in real time.

[1039] "Server" refers to the central computer that uses the generated AI model to analyze the detected data and determine whether clothes need to be washed.

[1040] An "AI model" refers to an algorithm that is generated on a server, analyzes learned data, and determines the degree of dirtiness of clothing.

[1041] An "emotion engine" refers to software that analyzes a user's voice data or image data and recognizes the user's emotional state.

[1042] The "washing machine panel" refers to the display device attached to the washing machine itself, which notifies the user of the analysis results and system status.

[1043] "User's terminal" refers to a mobile device such as a smartphone or tablet used by the user, which can be used to check notifications and results from the system and perform operations.

[1044] "Analysis results" refers to the judgment results on the degree of dirt and the need for washing that are generated after the server analyzes the data using an AI model.

[1045] "Laundry behavior" refers to a series of actions that a washing machine actually takes to wash clothes based on the analysis results and the user's emotional state.

[1046] This invention is a system that automatically detects soiling of clothes in a washing machine and performs washing only when necessary, thereby reducing unnecessary washing and suggesting washing actions taking into account the user's emotions. This system is realized using sensors installed in the washing machine, a server, a generative AI model, an emotion engine, a user's device, and communication means via the Internet.

[1047] First, the user puts laundry into the washing machine and closes the door. At this time, the device activates the sensors inside the washing machine. The sensors detect the degree of dirt, color, and type of the clothes in real time and collect that data. Specific examples of sensors include optical sensors for detecting dirt and color identification sensors.

[1048] The detected data is collected by the device and then transmitted to a server via the Internet. The device uses a communication protocol to efficiently transmit the data, such as HTTP or MQTT.

[1049] The server launches a generative AI model based on the received data. The generative AI model includes an algorithm that analyzes the degree of soiling of the clothes based on the learned data and determines whether they need to be washed. The analysis results are output as classifications such as "lightly soiled," "moderately soiled," or "heavily soiled."

[1050] At the same time, the voice or image data provided by the user using the smartphone's camera or microphone is analyzed by the emotion engine on the server, which determines the user's emotional state (stress, joy, calm, etc.).

[1051] The analysis results are then sent back to the device and the user's smartphone app. The device displays a message such as "No washing necessary" or "Washing necessary" on the washing machine's panel, and a notification is also sent to the user's smartphone app. For example, if "Washing unnecessary" is displayed, the user must press the "Start washing" button for the washing to begin.

[1052] Furthermore, if the emotion engine recognizes the user's emotion as stressed, the server will send the user a notification suggesting that they do laundry, regardless of the result of the judgment on the necessity of doing laundry. Even if the user ignores the suggestion, the system will automatically start the laundry. In this way, the laundry is carried out, and when it is finished, the terminal will send a completion notification to the user.

[1053] Specific examples

[1054] Case 1: Light soiling

[1055] 1. The user loads laundry into the washing machine and closes the door.

[1056] 2. The device activates the sensor and detects the degree of dirt on the clothes.

[1057] 3. The device sends the detection data to the server.

[1058] 4. The server analyzes the data using the generated AI model and determines that the item is "slightly soiled and no washing is required."

[1059] 5. The server notifies the device and the user's smartphone app of the analysis results.

[1060] 6. The device will display "No washing required" on the panel, and a notification will also be sent to the user's smartphone app.

[1061] 7. When the user recognizes through the emotion engine that they are in a stressful state, the system will suggest or automatically start doing laundry.

[1062] 8. The device starts the wash and notifies the user when it is finished.

[1063] Case 2: Moderate to severe soiling

[1064] 1. The user loads laundry into the washing machine and closes the door.

[1065] 2. The device activates the sensor and detects the degree of dirt on the clothes.

[1066] 3. The device sends the detection data to the server.

[1067] 4. The server analyzes the data using the generated AI model and determines that the item is "moderately soiled and requires washing."

[1068] 5. The server notifies the device and the user's smartphone app of the analysis results.

[1069] 6. The device will display "Laundry required" on the panel, and a notification will also be sent to the user's smartphone app.

[1070] 7. The user checks the notification on the smartphone app and selects to perform the laundry.

[1071] 8. The device will begin a normal wash cycle.

[1072] The system leverages generative AI models and an emotion engine to provide an efficient and user-friendly laundry experience.

[1073] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1074] Step 1:

[1075] The user places laundry in the washing machine and closes the door. This action automatically activates the device's built-in sensors, which detect the laundry's soiling level, color, and type in real time and collect the data. The input is the laundry, and the output is data on soiling level, color, and type.

[1076] Step 2:

[1077] The terminal first records the collected data in its internal memory and then transmits it to a server via the Internet. The terminal uses communication protocols such as HTTP and MQTT to transmit data efficiently. The input is data from the sensor, and the output is packaged data.

[1078] Step 3:

[1079] The server stores the received data in a database. Next, it launches a generative AI model and performs analysis based on the stored data. The AI ​​model evaluates the level of dirt through shading analysis and color identification, and outputs a classification result such as "lightly dirty," "moderately dirty," or "heavily dirty." The input is packaged data, and the output is the classification result of the level of dirt.

[1080] Step 4:

[1081] The user provides voice or image data using the camera or microphone on their smartphone. This data is sent to a server via the Internet and analyzed by an emotion engine. The emotion engine determines the user's emotional state (stress, joy, calm, etc.) and sends the results to the server. The input is voice or image data, and the output is the determined emotional state.

[1082] Step 5:

[1083] The server sends the analysis results and the emotional state determination results to the washing machine device and the user's smartphone app. The device displays a message such as "No washing necessary" or "Laundry required" on the washing machine panel, and a notification is also sent to the user's smartphone app. The input is the analysis results and the emotional state determination results, and the output is a notification message.

[1084] Step 6:

[1085] The user selects whether to press the "Start Wash" button on the smartphone app or the washing machine's control panel based on the notification of the laundry need. If the user selects to run the laundry, the device starts a normal wash cycle. The input is the user's selection, and the output is running the laundry.

[1086] Step 7:

[1087] If the emotion engine recognizes the user's emotion as stressful, the server sends a notification to the user suggesting laundry actions regardless of the result of the judgment on the necessity of laundry. Even if the suggestion is ignored, the system will automatically start the laundry. The input is the emotion analysis result, and the output is the laundry action suggestion or automatic execution.

[1088] Step 8:

[1089] When the wash is finished, the device sends a completion notification to the user. The notification is displayed on the washing machine panel and on the user's smartphone app. The input is the end status of the wash cycle, and the output is the completion notification.

[1090] In this way, it is possible to reduce wasted laundry and suggest laundry actions that take into account the user's emotional state.

[1091] (Application example 2)

[1092] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1093] On factory production lines, contamination of products and parts has a significant impact on shipping quality, but manually detecting and cleaning contamination is time-consuming and laborious. Furthermore, depending on the emotional state of the operator, it can be difficult to make appropriate decisions. Therefore, a system is needed that automatically detects contamination on product lines and makes decisions about cleaning or halting shipments as necessary. Furthermore, a system is needed that reduces the mental burden by making appropriate suggestions and taking appropriate action, taking into account the emotional state of the operator.

[1094] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1095] In this invention, the server includes means for detecting the degree of soiling of clothes using a sensor installed in the washing machine, means for analyzing the detected data using an AI model generated by the server and determining whether the clothes need to be washed, means for notifying the washing machine panel and the user's terminal of the result that the clothes do not need to be washed, means for the user to choose whether to perform or cancel the washing, means for automatically detecting the contamination state of the product line and parts, means for determining whether to perform cleaning work or stop shipping based on the contamination state, and means for recognizing the user's emotional state and suggesting cleaning or shipping. This makes it possible to automatically detect soiling and take necessary measures, while also reducing the burden on the operator.

[1096] A "sensor" is a device that measures physical or chemical quantities and outputs them as data. Optical sensors and image recognition sensors are particularly used.

[1097] A "server" is a computer system that provides services such as data storage, processing, and distribution over a network.

[1098] An "AI model" is an algorithm that has been trained to perform a specific task using artificial intelligence techniques, particularly frameworks such as TensorFlow and PyTorch.

[1099] A "panel" is an interface for displaying information. It is installed in washing machines and other appliances.

[1100] A "user's terminal" is a device used by a user, such as a computer or smartphone.

[1101] A "product line" refers to the equipment and facilities used to continuously produce products within a factory.

[1102] "Stain level" refers to the degree of foreign matter adhering to the surface of an object and the resulting discoloration.

[1103] "Data analysis" is the process of analyzing collected data and extracting meaningful information.

[1104] A "cleaning operation" is a manual or automated process for removing dirt or foreign matter.

[1105] "Shipment suspension" means stopping the shipment of a product, and is done to ensure quality.

[1106] "Emotional state" refers to the current psychological state of the user or operator.

[1107] The present invention is a system that uses a robot installed on a factory production line to automatically detect dirt on products and parts, and based on the results, decides whether to perform cleaning work or stop product shipments. This system is composed of the following elements.

[1108] 1. Dirt detection by sensor

[1109] The robot is equipped with optical sensors and image recognition sensors that constantly monitor the production line and detect the degree of dirt on products and parts. The data detected by the sensors is temporarily collected on a terminal inside the robot.

[1110] 2. Data transmission

[1111] The collected data is sent to a cloud server via Wi-Fi or a wired connection, and the device uses a communication protocol to package and efficiently transmit the data.

[1112] 3. Analysis using AI models

[1113] The server uses a generative AI model to analyze the data it receives, specifically an AI model based on frameworks such as TensorFlow and PyTorch, to analyze the data and classify the level of dirt into categories such as "lightly dirty," "moderately dirty," and "heavily dirty."

[1114] 4. Recognition of user emotions using an emotion engine

[1115] An emotion engine is used to recognize the emotional state of the operator through a smartphone app or computer. If the operator is under stress or overload, the emotion engine will detect this.

[1116] 5. Notification of Results

[1117] The analysis results are sent to the robot's terminal and the operator's smartphone or computer. The terminal displays messages such as "Cleaning required" or "Shipping suspended" on the robot's panel, and the same information is also sent to the operator's terminal.

[1118] 6. User Selection and Autorun

[1119] Based on the analysis results, the operator can choose to carry out cleaning work, cancel the work, or stop shipping. If the operator's emotional state is stressed, the system will automatically carry out cleaning work or stop shipping.

[1120] This system will automatically detect contamination and take necessary measures, and is expected to reduce the burden and mental stress on operators.

[1121] As a concrete example, you can use the following prompt to have an AI model perform an analysis:

[1122] It judges the degree of soiling of the product line based on images, and starts cleaning if the soiling score exceeds the previous threshold, or skips cleaning if it does not. It also starts cleaning if the operator's emotional state is stressed or overloaded.

[1123] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1124] Step 1:

[1125] Dirt detection by sensors

[1126] Optical sensors and image recognition sensors installed on the robot detect the degree of dirt on products and parts on the production line. The input is the physical condition of the product or part, and the output is a measurement value of the degree of dirt as digital data. This measurement value is temporarily stored on a terminal inside the robot.

[1127] Step 2:

[1128] Sending data

[1129] The device transmits the collected soiling data to a cloud server via Wi-Fi or a wired connection. The input is the measurement data stored in the device, and the output is the data transferred to the cloud server. The device packages the data for efficient transmission.

[1130] Step 3:

[1131] Analysis using AI models

[1132] The server analyzes the received data using a generative AI model. Specifically, it uses a pre-trained model based on TensorFlow and PyTorch to classify the degree of dirt into categories such as "lightly dirty," "moderately dirty," and "heavily dirty." The input is the digital data of the dirt level sent to the cloud server, and the output is the classification result.

[1133] Step 4:

[1134] Recognizing user emotions with an emotion engine

[1135] The emotion engine recognizes the operator's emotional state through the user's smartphone app or computer. The input is voice data and image data provided by the operator, and the output is the judgment result of the operator's emotional state. The emotion engine uses technologies such as OpenVINO.

[1136] Step 5:

[1137] Notification of analysis results and emotional state

[1138] The server notifies the robot's terminal and the operator's smartphone or computer of the analysis results and emotional state. The input is the classification results of the dirt level analyzed by the server and the judgment results of the emotion engine, and the output is a notification message. The robot's panel displays messages such as "Cleaning required" or "Shipping suspended."

[1139] Step 6:

[1140] User selection and automatic execution

[1141] Based on the notified analysis results, the operator can choose whether to carry out cleaning work or stop shipments. The input is the notification message and the operator's choice, and the output is the start of cleaning work or the stop of shipments. If the operator's emotional state is stressed, cleaning work or shipments will be automatically stopped.

[1142] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1143] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1144] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1145] [Fourth embodiment]

[1146] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1147] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1148] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1149] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1150] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1151] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1152] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1153] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1154] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1155] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1156] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1157] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1158] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1159] The present invention is a system that automatically detects the degree of soiling of clothes in a washing machine and performs washing only when necessary, thereby reducing unnecessary washing. This system mainly consists of the following elements.

[1160] 1. Dirt detection by sensor

[1161] The sensors installed in the washing machine detect the degree of soiling, color, and type of laundry in real time. These sensors can be optical or chemical sensors, for example. When a user puts laundry into the washing machine, the sensors are automatically activated and start collecting data.

[1162] 2. Data transmission

[1163] The sensed data is collected by a terminal inside the washing machine and then transmitted over the internet to a server, which uses a communications protocol to package and efficiently transmit the data.

[1164] 3. Analysis using AI models

[1165] The server launches a generative AI model to analyze the received data. This AI model learns from past data and can accurately determine whether clothes need to be washed. The server uses this model to generate data analysis results. For example, it outputs classification results such as "lightly soiled," "moderately soiled," and "heavily soiled."

[1166] 4. Notification of Results

[1167] The analysis results are sent to the washing machine terminal and the user's smartphone app. The terminal displays messages such as "No washing necessary" or "Washing required" on the washing machine panel. The user can check the detailed analysis results through the smartphone app.

[1168] 5. User Choice

[1169] Based on the analysis results, the user can choose whether to run or cancel the wash. For example, if the user receives a "no washing required" notification, the wash will not run unless the user presses the "start washing" button on the smartphone app or the washing machine's control panel. On the other hand, if the user presses the "start washing" button, the normal wash cycle will begin.

[1170] Specific examples

[1171] Case 1: Light soiling

[1172] The user puts laundry into the washing machine.

[1173] The device activates a sensor to detect the degree of dirt on the clothes.

[1174] The device transmits the detected data to the server.

[1175] The server analyzes the data using the generated AI model and determines that the item is "slightly soiled and no washing is required."

[1176] The server notifies the device and the user's smartphone app of the analysis results.

[1177] The device will display "No washing required" on its panel, and a notification will also be sent to the user's smartphone app.

[1178] The user checks the notification on the smartphone app and chooses to cancel the wash.

[1179] The device will stop the wash and prompt the user to remove the clothes.

[1180] Case 2: Moderate to severe soiling

[1181] The user puts laundry into the washing machine.

[1182] The device activates a sensor to detect the degree of dirt on the clothes.

[1183] The device transmits the detected data to the server.

[1184] The server analyzes the data using the generated AI model and determines that the garment is "moderately soiled and requires washing."

[1185] The server notifies the device and the user's smartphone app of the analysis results.

[1186] The device will display "Washing required" on the panel, and a notification will also be sent to the user's smartphone app.

[1187] The user checks the notification on the smartphone app and selects to perform the laundry.

[1188] The device will begin a normal wash cycle.

[1189] As described above, the system of the present invention can improve the efficiency of laundry through the interaction between sensors, servers, AI models, terminals, and users. This system also reduces unnecessary laundry, contributes to saving water resources, and reduces household water bills, making it environmentally friendly.

[1190] The processing flow will be explained below.

[1191] Step 1:

[1192] The user loads laundry into the washing machine and closes the door, ready for the wash cycle to begin.

[1193] Step 2:

[1194] The device activates the sensors inside the washing machine, which detect the soiling level, color, and type of clothes in real time and collect data.

[1195] Step 3:

[1196] The device temporarily stores the data acquired from the sensor and then prepares it to be sent to a server via the Internet.

[1197] Step 4:

[1198] The device then sends the packaged data over the internet to a server, which includes details such as the degree of dirt, color, and type.

[1199] Step 5:

[1200] The server then launches a generative AI model to analyze the received data. Based on the learned data, the AI ​​model analyzes the degree of soiling of the clothes and determines whether they need to be washed.

[1201] Step 6:

[1202] The server generates the analysis results, which are displayed as classifications such as "lightly soiled," "moderately soiled," and "heavily soiled."

[1203] Step 7:

[1204] The analysis results generated by the server are sent to the device and the user's smartphone app.

[1205] Step 8:

[1206] The device displays the analysis results on the washing machine's control panel, such as a message like "No washing necessary" or "Washing required."

[1207] Step 9:

[1208] The user opens the smartphone app and checks the analysis results, which are displayed in detail.

[1209] Step 10:

[1210] The user can choose whether to start or stop the wash by pressing the "Start Wash" or "Cancel" button on the smartphone app or the washing machine's control panel.

[1211] Step 11:

[1212] The device acts based on the user's selection. If the user selects "Start Wash," the device starts a normal wash cycle. If the user selects "Cancel," the device stops the wash and prompts the user to remove the clothes.

[1213] Step 12:

[1214] The user removes the clothes. If it is determined that washing is not necessary, the user removes the clothes and stores them. If washing is performed, the user removes the clothes after the washing is completed.

[1215] These are the specific steps in the washing machine's process, from detecting dirt to starting or canceling the wash. This system reduces unnecessary washing, lightens the burden of housework, and conserves water resources.

[1216] Example 1

[1217] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1218] Conventional washing machines have difficulty accurately detecting the degree of soiling of laundry, resulting in unnecessary washing. This results in wasted water and electricity consumption, placing a heavy burden on the environment. Another issue is that users must check the degree of soiling of their clothes and select the appropriate washing method every time they wash, which is time-consuming.

[1219] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1220] In this invention, the server includes: means for detecting the degree of soiling of clothes using a sensor installed in the washing machine; means for transmitting the detected data to the server via the Internet via a terminal; means for analyzing the received data using an AI model generated by the server and determining whether the clothes need to be washed; means for notifying the user of the analysis results via the washing machine's operation panel and the user's terminal; and means for the user to select whether to perform or cancel the wash based on the analysis results. This reduces unnecessary washing and prevents the wasteful consumption of water resources and electricity. It also allows users to wash clothes efficiently without hassle, contributing to environmental protection.

[1221] "Sensor" refers to a device installed in a washing machine that detects the degree of soiling, color, type, etc. of clothes. This includes optical sensors and chemical sensors.

[1222] A "terminal" is a device that is placed inside the washing machine and is responsible for collecting data from the sensors and sending it to the server.

[1223] A "server" is a computer system that receives and analyzes data sent from a terminal via the Internet.

[1224] The "AI model" is a machine learning model that runs on a server and is used to analyze received data and determine whether clothes need to be washed.

[1225] The "operation panel" is an interface device that is provided in the washing machine and that displays the analysis results to the user.

[1226] The "means of notifying the terminal" is a communication means for transmitting the analysis results from the server to the terminal. It uses the Internet Protocol.

[1227] A "user's device" is a communication-enabled device such as a smartphone or tablet held by a user, and is responsible for receiving and displaying notifications from the server.

[1228] "Means by which the user can choose whether to start or stop washing based on the analysis results" refers to an interface that allows the user to check the analysis results through the operation panel or the user's terminal and choose whether to start or stop washing.

[1229] The present invention is a system that automatically detects the degree of soiling of clothes in a washing machine and performs washing as necessary, thereby reducing unnecessary washing. This system is mainly composed of sensors, terminals, a server, a generative AI model, and a user smartphone app. Specific embodiments of the system are described below.

[1230] 1. Sensor activation and data collection

[1231] When a user loads laundry into the washing machine, the device inside the washing machine automatically activates sensors as soon as the lid is closed. The sensors use optical and chemical sensors to detect data such as the soiling level, color, and type of clothes. Optical sensors scan the surface of the clothes and measure the intensity and wavelength of reflected light. Chemical sensors detect specific pollutants, such as volatile organic compounds (VOCs) and specific stains. The detected data is collected and stored locally on the device.

[1232] 2. Data transmission

[1233] The device packages the collected data and sends it to a server via the Internet. The data is packaged in a specified format (for example, JSON format) and includes information such as the degree of dirt, color, and type. HTTP / HTTPS is generally used as the transmission protocol. For example, the endpoint URL is "https: / / example.com / api / wash-data".

[1234] 3. Data Analysis

[1235] The server launches a generative AI model to analyze the received data. The AI ​​model is implemented using frameworks such as TensorFlow and PyTorch. This AI model learns from past data and can accurately determine whether clothes need to be washed. The generative AI model analyzes the data and outputs classification results, such as "lightly soiled," "moderately soiled," or "heavily soiled." The analysis results are stored on the server.

[1236] 4. Result notification

[1237] The server notifies the device inside the washing machine and the user's smartphone app of the analysis results. Notifications are sent via push notifications and REST API calls. The smartphone app receives a notification such as "Lightly soiled, no need to wash," and the washing machine's control panel displays a message based on the analysis results.

[1238] 5. User Choice

[1239] Based on the analysis results, the user can choose whether to run or cancel the wash. They can receive a notification via the smartphone app and check the details. The user decides whether to run the wash by choosing whether to press the "Start Wash" button. If "washing is not necessary," the wash will not be run if the user does nothing. On the other hand, if it is determined that "washing is necessary," the device will start a normal wash cycle when the user presses the "Start Wash" button. The user will be notified by a voice message or other means when the cycle starts.

[1240] Specific examples

[1241] Case 1: Light soiling

[1242] The user puts laundry into the washing machine.

[1243] The device activates a sensor to detect the degree of dirt on the clothes.

[1244] The device transmits the detected data to the server.

[1245] The server analyzes the data using the generated AI model and determines that the item is "slightly soiled and no washing is required."

[1246] The server notifies the device and the user's smartphone app of the analysis results.

[1247] The device will display "No washing required" on the operation panel, and a notification will also be sent to the user's smartphone app.

[1248] The user checks the notification on the smartphone app and chooses to cancel the wash.

[1249] The device will stop the wash and prompt the user to remove the clothes.

[1250] Case 2: Moderate to severe soiling

[1251] The user puts laundry into the washing machine.

[1252] The device activates a sensor to detect the degree of dirt on the clothes.

[1253] The device transmits the detected data to the server.

[1254] The server analyzes the data using the generated AI model and determines that the garment is "moderately soiled and requires washing."

[1255] The server notifies the device and the user's smartphone app of the analysis results.

[1256] The device will display "Washing required" on the operation panel, and a notification will also be sent to the user's smartphone app.

[1257] The user checks the notification on the smartphone app and selects to perform the laundry.

[1258] The device will begin a normal wash cycle.

[1259] This reduces unnecessary washing and prevents the wasteful consumption of water and electricity. It also allows users to wash their clothes efficiently and hassle-free, contributing to environmental protection.

[1260] Prompt Sentence Examples

[1261] "Please create a prompt for a system that detects the degree of soiling of laundry and performs the washing if necessary."

[1262] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1263] Step 1:

[1264] Sensor activation and data collection

[1265] A user loads laundry into the washing machine and closes the lid. This action automatically activates sensors on a device inside the washing machine. The sensors use optical and chemical sensors to detect the soiling level of the clothes. Specifically, optical sensors scan the surface of the clothes and measure the intensity and wavelength of reflected light. Chemical sensors detect volatile organic compounds (VOCs) and specific stains. The collected data is then sent directly to the device and stored locally.

[1266] Input: Load laundry and close lid

[1267] Output: Detection data such as dirt level, color, type, etc.

[1268] Step 2:

[1269] Sending data

[1270] The device packages the data collected from the sensor. The data is packaged in a specified format such as JSON. For example, the data may include information such as the degree of dirt, color, and type. The device then transmits the data to a server via the Internet. HTTP / HTTPS is generally used as the transmission protocol.

[1271] Input: Detection data from sensors

[1272] Output: Packaged data sent to the server

[1273] Step 3:

[1274] Data analysis

[1275] The server launches a generative AI model to analyze the received data. The server then analyzes the data using an AI model implemented using frameworks such as TensorFlow or PyTorch. The generative AI model evaluates the degree of soiling of the clothes based on the received data and outputs a classification result such as "lightly soiled," "moderately soiled," or "heavily soiled." The analysis results are stored on the server.

[1276] Input: Data package sent from the terminal

[1277] Output: Analysis results (classified data: mild, mild, moderate, severe)

[1278] Step 4:

[1279] Result notification

[1280] The server notifies the washing machine device and the user's smartphone app of the analysis results. Notification methods include push notifications and REST API calls. This allows the user's smartphone app to receive specific notifications, such as "Lightly soiled, no need to wash." The results are also displayed on the washing machine's control panel.

[1281] Input: Analysis results of the AI ​​model

[1282] Output: Notifications to the device and the user's smartphone app

[1283] Step 5:

[1284] User Selection

[1285] The user checks the analysis results through the smartphone app or the washing machine's operation panel, and can then choose whether to start or stop the wash. If the notification is "lightly soiled, no washing necessary," the wash will not be carried out unless the user presses the "start washing" button. On the other hand, if the notification is "moderately soiled, washing required," the wash will begin if the user presses the "start washing" button on the smartphone app or operation panel.

[1286] Input: User selection via smartphone app or operation panel

[1287] Output: Decision to do or not do laundry

[1288] Step 6:

[1289] Starting the wash cycle

[1290] If the user decides to wash the clothes, the device will start the washing machine's normal wash cycle, and the user will be notified when the wash cycle starts, for example, through a voice message or a display on the operation panel.

[1291] Input: User's laundry run selection

[1292] Output: Start and run a normal wash cycle

[1293] (Application example 1)

[1294] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1295] The present invention relates to a system for reducing unnecessary laundry and false alarms in homes and other environments. Specifically, the system detects the degree of soiling of clothes and abnormalities in the home with high accuracy, and performs laundry and issues alarms only when necessary, thereby preventing waste of resources and false alarms.

[1296] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1297] In this invention, the server includes a means for detecting the degree of soiling of clothes using a sensor installed in the washing machine, a means for analyzing the detected data using an AI model generated by the server and determining whether the clothes need to be washed, and a means for detecting abnormalities in the home using cameras and chemical sensors installed in the home. This enables the system to automatically detect the degree of soiling of clothes and abnormal conditions in the home and notify the user, thereby reducing unnecessary washing and warnings.

[1298] A "sensor" is a device that senses physical or chemical quantities, converts them into electrical signals, and outputs them.

[1299] An "AI model" is an artificial intelligence algorithm designed to learn from past data and analyze new data to make decisions.

[1300] A "server" refers to equipment or software used to store, process, and analyze data on a network.

[1301] A "washing machine" is a mechanical device for washing clothes.

[1302] A "display device" is a device for visually displaying information, and includes, for example, a display or a panel.

[1303] An "information communication device" is a device for sending and receiving data and information over a network, and includes, for example, smartphones and tablets.

[1304] A "camera" is a device that takes pictures and videos and records them as data.

[1305] A "chemical sensor" is a device that detects specific chemical substances and outputs their presence and concentration as an electrical signal.

[1306] "Abnormal" refers to an event or situation that is different from normal, such as suspicious movements or sounds, or gas leaks.

[1307] An "alarm" refers to a system or function that uses sound, light, or other means to draw attention when an abnormality occurs.

[1308] This invention is a system for efficient home laundry and security management. The system consists of a washing machine, sensors, an AI model, a display device, an information and communication device, a camera, and a chemical sensor.

[1309] First, sensors installed in the washing machine detect the degree of soiling of the clothes put in. Specifically, optical sensors or chemical sensors are used. The data collected by the sensors is temporarily stored on a terminal inside the washing machine and then sent to a server via the Internet.

[1310] The server launches a generative AI model to analyze the received data. This AI model learns from past data and accurately determines whether clothes need to be washed. For example, it can classify clothes into categories such as "lightly soiled," "moderately soiled," and "heavily soiled." The analysis results are notified to the washing machine's display and the user's information and communication device.

[1311] Based on these notifications, the user can choose to either run or cancel the laundry. If the analysis results indicate that washing is not necessary, the user can cancel the laundry and, if necessary, run the laundry manually.

[1312] In addition, cameras and chemical sensors installed in the home will detect abnormalities, such as suspicious movements, sounds, and gas leaks. If these sensors detect an abnormality, the data will also be sent to the server. The generated AI model will analyze the severity of the abnormality and output a classification result such as "mild abnormality," "moderate abnormality," or "severe abnormality." This result will also be notified to the display device in the home and the user's information and communication device.

[1313] The user can choose to issue or cancel an alert based on this notification. If the analysis result is judged to be a "serious abnormality," the user is advised to take immediate action to address the issue.

[1314] For example, if a home camera detects abnormal activity, the data is sent to a server and analyzed by an AI model. If the analysis results in a "severe abnormality," a notification is sent to the user's smartphone stating, "Suspicious activity has been detected!" The user can then choose to activate or cancel the alarm via the smartphone app.

[1315] An example of a specific prompt is:

[1316] An abnormality was detected on a home camera at 16:45. After analyzing the video, it was determined that a "serious abnormality" had occurred. Please check immediately.

[1317] There are messages like this.

[1318] The hardware used includes washing machines, cameras, sensors, displays, and information and communication devices. The software includes OpenCV, Requests, and a generative AI model. The generative AI model is implemented using Python's TensorFlow and PyTorch.

[1319] With this configuration, the present invention can reduce wasted laundry and the occurrence of false alarms, contributing to improved efficiency and security within the home.

[1320] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1321] Step 1:

[1322] The device activates sensors installed in the washing machine to detect the degree of soiling of the clothes placed in it. Optical and chemical sensors are used for this purpose. The input is the physical data of the clothes, and the output is numerical data on the degree of soiling.

[1323] Step 2:

[1324] The device collects data on the level of dirt detected by the sensor and sends it to a server via the Internet. The input is numerical data on the level of dirt, and the output is packetized data.

[1325] Step 3:

[1326] The server launches a generative AI model to analyze the received soiling data. Because this model is trained based on a past database, it can analyze new data with high accuracy. The input is numerical data on the soiling level, and the output is classification data such as "light soiling," "moderate soiling," or "heavy soiling."

[1327] Step 4:

[1328] The server notifies the analysis results to the display device of the washing machine and the user's information communication device. The input is the classification data, and the output is a notification message.

[1329] Step 5:

[1330] The user checks the notification on the information communication device and selects whether to run or cancel the laundry. The input is the notification message, and the output is the user's selected action ("run" or "cancel").

[1331] Step 6:

[1332] Cameras and chemical sensors installed in the home detect abnormalities, such as suspicious movements, sounds, gas leaks, etc. The input is environmental data within the home, and the output is numerical data on abnormalities.

[1333] Step 7:

[1334] Data obtained from cameras and sensors is also collected by the terminal and sent to the server via the Internet. The input is numerical data of anomalies, and the output is packetized data.

[1335] Step 8:

[1336] The server then launches the generative AI model again to analyze the received anomaly data. The input is the numerical data of the anomaly, and the output is classification data such as "mild anomaly," "moderate anomaly," or "severe anomaly."

[1337] Step 9:

[1338] The server notifies the user of the results of the anomaly analysis to the display device in the home and the user's information and communication device. The input is the classification data, and the output is an alarm notification message.

[1339] Step 10:

[1340] The user checks the alarm notification on the information communication device and selects whether to activate or cancel the alarm. The input is the alarm notification message, and the output is the user's selected action ("activate" or "cancel").

[1341] In this way, the system of the present invention achieves improved efficiency and security within the home through collaboration between sensors, cameras, generative AI models, display devices, information and communication devices, and servers.

[1342] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1343] The present invention is a system that automatically detects the degree of soiling of clothes in a washing machine and performs washing only when necessary, thereby reducing unnecessary washing and suggesting washing actions taking into account the user's emotions. This system mainly consists of the following elements.

[1344] 1. Dirt detection by sensor

[1345] The sensors installed inside the washing machine have the ability to detect the degree of dirt, color, and type of laundry in real time. When a user puts laundry into the washing machine, the sensors are automatically activated and start collecting data.

[1346] 2. Data transmission

[1347] The detected data is collected by a terminal inside the washing machine and then transmitted to a server over the internet, where the terminal uses a communication protocol to package and transmit the data efficiently.

[1348] 3. Analysis using AI models

[1349] The server launches a generative AI model to analyze the received data. Based on the learned data, the AI ​​model analyzes the degree of soiling of the clothes and determines whether they need to be washed. The server uses this model to generate the results of the data analysis. For example, it outputs classification results such as "lightly soiled," "moderately soiled," and "heavily soiled."

[1350] 4. Recognition of user emotions using an emotion engine

[1351] The emotion engine has the ability to recognize the user's emotions based on voice or image data acquired from the user's device. For example, the emotion engine analyzes data provided by the user through the smartphone's camera or microphone to determine the user's emotional state (stress, joy, calm, etc.).

[1352] 5. Notification of Results

[1353] The analysis results are sent to the washing machine terminal and the user's smartphone app. The terminal displays messages such as "No washing necessary" or "Washing required" on the washing machine panel. The user can check the detailed analysis results through the smartphone app.

[1354] 6. User Choice

[1355] Based on the analysis results, the user can choose whether to run or cancel the wash. For example, if the user receives a "no washing required" notification, the wash will not run unless the user presses the "start washing" button on the smartphone app or the washing machine's control panel. On the other hand, if the user presses the "start washing" button, the normal wash cycle will begin.

[1356] 7. Emotion-based recommendations

[1357] If the emotion engine recognizes that the user is in a stressful state, the system will suggest or automatically start doing laundry, regardless of the result of the laundry necessity judgment. This reduces the user's mental burden.

[1358] Specific examples

[1359] Case 1: Light soiling

[1360] The user places the laundry in the washing machine and closes the door.

[1361] The device activates a sensor to detect the degree of dirt on the clothes.

[1362] The device transmits the detected data to the server.

[1363] The server analyzes the data using the generated AI model and determines that the item is "slightly soiled and no washing is required."

[1364] The server notifies the device and the user's smartphone app of the analysis results.

[1365] The device will display "No washing required" on its panel, and a notification will also be sent to the user's smartphone app.

[1366] When the user recognizes through the emotion engine that they are in a stressful state, the system will suggest or automatically start doing laundry.

[1367] The device will start the wash and notify the user when it is finished.

[1368] Case 2: Moderate to severe soiling

[1369] The user places the laundry in the washing machine and closes the door.

[1370] The device activates a sensor to detect the degree of dirt on the clothes.

[1371] The device transmits the detected data to the server.

[1372] The server analyzes the data using the generated AI model and determines that the garment is "moderately soiled and requires washing."

[1373] The server notifies the device and the user's smartphone app of the analysis results.

[1374] The device will display "Washing required" on the panel, and a notification will also be sent to the user's smartphone app.

[1375] The user checks the notification on the smartphone app and selects to perform the laundry.

[1376] The device will begin a normal wash cycle.

[1377] As described above, the system of the present invention can improve the efficiency of laundry through interactions between sensors, servers, AI models, emotion engines, terminals, and users. This system is also environmentally friendly, as it reduces unnecessary laundry, contributes to saving water resources, and reduces household water bills. Furthermore, by suggesting laundry actions based on the user's emotional state, it also contributes to reducing mental burden.

[1378] The processing flow will be explained below.

[1379] Step 1:

[1380] The user loads laundry into the washing machine and closes the door, ready for the wash cycle to begin.

[1381] Step 2:

[1382] The device activates the sensors inside the washing machine, which detect the soiling level, color, and type of clothes in real time and collect data.

[1383] Step 3:

[1384] The device temporarily stores the data acquired from the sensor and then prepares it to be sent to a server via the Internet.

[1385] Step 4:

[1386] The device then sends the packaged data over the internet to a server, which includes details such as the soiling level, color, and type of clothing.

[1387] Step 5:

[1388] The server then launches a generative AI model to analyze the received data. Based on past learning data, the AI ​​model analyzes the degree of soiling of the clothes and determines whether they need to be washed.

[1389] Step 6:

[1390] The server generates the analysis results, which are displayed as classifications such as "lightly soiled," "moderately soiled," and "heavily soiled."

[1391] Step 7:

[1392] The analysis results generated by the server are sent to the device and the user's smartphone app.

[1393] Step 8:

[1394] The device displays the analysis results on the washing machine's control panel, such as a message like "No washing necessary" or "Washing required."

[1395] Step 9:

[1396] The user opens the smartphone app and checks the analysis results, which are displayed in detail.

[1397] Step 10:

[1398] To recognize the user's emotions, the emotion engine acquires voice or image data from the user's device and determines the user's emotional state (stress, joy, calm, etc.) based on this data.

[1399] Step 11:

[1400] If the user's emotion recognized by the emotion engine is a stress state, the system will suggest to the user to do laundry or automatically start laundry regardless of the result of the judgment on the necessity of laundry.

[1401] Step 12:

[1402] The user can choose whether to start or stop the wash by pressing the "Start Wash" or "Cancel" button on the smartphone app or the washing machine's control panel.

[1403] Step 13:

[1404] The device acts based on the user's selection. If the user selects "Start Wash," the device starts a normal wash cycle. If the user selects "Cancel," the device stops the wash and prompts the user to remove the clothes.

[1405] Step 14:

[1406] The user removes the clothes. If it is determined that washing is not necessary, the user removes the clothes and stores them. If washing is performed, the user removes the clothes after the washing is completed.

[1407] These are the specific processing steps from detecting dirt in a washing machine to starting or canceling the wash. This system can reduce unnecessary laundry, lighten the burden of housework, and conserve water resources. Furthermore, the emotion engine can take the user's emotions into consideration when suggesting laundry actions, thereby helping to reduce the user's mental burden.

[1408] Example 2

[1409] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1410] Conventional washing machine systems wash clothes without considering the degree of dirt or the user's emotional state, which has led to problems such as increased wasted laundry and mental stress on the user.In addition, there is a lack of systems that accurately determine the need for laundry, resulting in wasted water resources and energy.

[1411] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1412] In this invention, the server includes a means for detecting the degree of soiling of clothes using a sensor installed in the washing machine, a means for analyzing the detected data using an AI model generated by the server and determining whether the clothes need to be washed, and a means for recognizing the user's emotional state using an emotion engine and suggesting or automatically starting a washing action based on that. This reduces unnecessary laundry, eases the user's mental burden, and enables the saving of water resources and energy.

[1413] A "sensor" refers to a device installed in a washing machine that detects the degree of dirt, color, and type of clothes in real time.

[1414] "Server" refers to the central computer that uses the generated AI model to analyze the detected data and determine whether clothes need to be washed.

[1415] An "AI model" refers to an algorithm that is generated on a server, analyzes learned data, and determines the degree of dirtiness of clothing.

[1416] An "emotion engine" refers to software that analyzes a user's voice data or image data and recognizes the user's emotional state.

[1417] The "washing machine panel" refers to the display device attached to the washing machine itself, which notifies the user of the analysis results and system status.

[1418] "User's terminal" refers to a mobile device such as a smartphone or tablet used by the user, which can be used to check notifications and results from the system and perform operations.

[1419] "Analysis results" refers to the judgment results on the degree of dirt and the need for washing that are generated after the server analyzes the data using an AI model.

[1420] "Laundry behavior" refers to a series of actions that a washing machine actually takes to wash clothes based on the analysis results and the user's emotional state.

[1421] This invention is a system that automatically detects soiling of clothes in a washing machine and performs washing only when necessary, thereby reducing unnecessary washing and suggesting washing actions taking into account the user's emotions. This system is realized using sensors installed in the washing machine, a server, a generative AI model, an emotion engine, a user's device, and communication means via the Internet.

[1422] First, the user puts laundry into the washing machine and closes the door. At this time, the device activates the sensors inside the washing machine. The sensors detect the degree of dirt, color, and type of the clothes in real time and collect that data. Specific examples of sensors include optical sensors for detecting dirt and color identification sensors.

[1423] The detected data is collected by the device and then transmitted to a server via the Internet. The device uses a communication protocol to efficiently transmit the data, such as HTTP or MQTT.

[1424] The server launches a generative AI model based on the received data. The generative AI model includes an algorithm that analyzes the degree of soiling of the clothes based on the learned data and determines whether they need to be washed. The analysis results are output as classifications such as "lightly soiled," "moderately soiled," or "heavily soiled."

[1425] At the same time, the voice or image data provided by the user using the smartphone's camera or microphone is analyzed by the emotion engine on the server, which determines the user's emotional state (stress, joy, calm, etc.).

[1426] The analysis results are then sent back to the device and the user's smartphone app. The device displays a message such as "No washing necessary" or "Washing necessary" on the washing machine's panel, and a notification is also sent to the user's smartphone app. For example, if "Washing unnecessary" is displayed, the user must press the "Start washing" button for the washing to begin.

[1427] Furthermore, if the emotion engine recognizes the user's emotion as stressed, the server will send the user a notification suggesting that they do laundry, regardless of the result of the judgment on the necessity of doing laundry. Even if the user ignores the suggestion, the system will automatically start the laundry. In this way, the laundry is carried out, and when it is finished, the terminal will send a completion notification to the user.

[1428] Specific examples

[1429] Case 1: Light soiling

[1430] 1. The user loads laundry into the washing machine and closes the door.

[1431] 2. The device activates the sensor and detects the degree of dirt on the clothes.

[1432] 3. The device sends the detection data to the server.

[1433] 4. The server analyzes the data using the generated AI model and determines that the item is "slightly soiled and no washing is required."

[1434] 5. The server notifies the device and the user's smartphone app of the analysis results.

[1435] 6. The device will display "No washing required" on the panel, and a notification will also be sent to the user's smartphone app.

[1436] 7. When the user recognizes through the emotion engine that they are in a stressful state, the system will suggest or automatically start doing laundry.

[1437] 8. The device starts the wash and notifies the user when it is finished.

[1438] Case 2: Moderate to severe soiling

[1439] 1. The user loads laundry into the washing machine and closes the door.

[1440] 2. The device activates the sensor and detects the degree of dirt on the clothes.

[1441] 3. The device sends the detection data to the server.

[1442] 4. The server analyzes the data using the generated AI model and determines that the item is "moderately soiled and requires washing."

[1443] 5. The server notifies the device and the user's smartphone app of the analysis results.

[1444] 6. The device will display "Laundry required" on the panel, and a notification will also be sent to the user's smartphone app.

[1445] 7. The user checks the notification on the smartphone app and selects to perform the laundry.

[1446] 8. The device will begin a normal wash cycle.

[1447] The system leverages generative AI models and an emotion engine to provide an efficient and user-friendly laundry experience.

[1448] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1449] Step 1:

[1450] The user places laundry in the washing machine and closes the door. This action automatically activates the device's built-in sensors, which detect the laundry's soiling level, color, and type in real time and collect the data. The input is the laundry, and the output is data on soiling level, color, and type.

[1451] Step 2:

[1452] The terminal first records the collected data in its internal memory and then transmits it to a server via the Internet. The terminal uses communication protocols such as HTTP and MQTT to transmit data efficiently. The input is data from the sensor, and the output is packaged data.

[1453] Step 3:

[1454] The server stores the received data in a database. Next, it launches a generative AI model and performs analysis based on the stored data. The AI ​​model evaluates the level of dirt through shading analysis and color identification, and outputs a classification result such as "lightly dirty," "moderately dirty," or "heavily dirty." The input is packaged data, and the output is the classification result of the level of dirt.

[1455] Step 4:

[1456] The user provides voice or image data using the camera or microphone on their smartphone. This data is sent to a server via the Internet and analyzed by an emotion engine. The emotion engine determines the user's emotional state (stress, joy, calm, etc.) and sends the results to the server. The input is voice or image data, and the output is the determined emotional state.

[1457] Step 5:

[1458] The server sends the analysis results and the emotional state determination results to the washing machine device and the user's smartphone app. The device displays a message such as "No washing necessary" or "Laundry required" on the washing machine panel, and a notification is also sent to the user's smartphone app. The input is the analysis results and the emotional state determination results, and the output is a notification message.

[1459] Step 6:

[1460] The user selects whether to press the "Start Wash" button on the smartphone app or the washing machine's control panel based on the notification of the laundry need. If the user selects to run the laundry, the device starts a normal wash cycle. The input is the user's selection, and the output is running the laundry.

[1461] Step 7:

[1462] If the emotion engine recognizes the user's emotion as stressful, the server sends a notification to the user suggesting laundry actions regardless of the result of the judgment on the necessity of laundry. Even if the suggestion is ignored, the system will automatically start the laundry. The input is the emotion analysis result, and the output is the laundry action suggestion or automatic execution.

[1463] Step 8:

[1464] When the wash is finished, the device sends a completion notification to the user. The notification is displayed on the washing machine panel and on the user's smartphone app. The input is the end status of the wash cycle, and the output is the completion notification.

[1465] In this way, it is possible to reduce wasted laundry and suggest laundry actions that take into account the user's emotional state.

[1466] (Application example 2)

[1467] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1468] On factory production lines, contamination of products and parts has a significant impact on shipping quality, but manually detecting and cleaning contamination is time-consuming and laborious. Furthermore, depending on the emotional state of the operator, it can be difficult to make appropriate decisions. Therefore, a system is needed that automatically detects contamination on product lines and makes decisions about cleaning or halting shipments as necessary. Furthermore, a system is needed that reduces the mental burden by making appropriate suggestions and taking appropriate action, taking into account the emotional state of the operator.

[1469] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1470] In this invention, the server includes means for detecting the degree of soiling of clothes using a sensor installed in the washing machine, means for analyzing the detected data using an AI model generated by the server and determining whether the clothes need to be washed, means for notifying the washing machine panel and the user's terminal of the result that the clothes do not need to be washed, means for the user to choose whether to perform or cancel the washing, means for automatically detecting the contamination state of the product line and parts, means for determining whether to perform cleaning work or stop shipping based on the contamination state, and means for recognizing the user's emotional state and suggesting cleaning or shipping. This makes it possible to automatically detect soiling and take necessary measures, while also reducing the burden on the operator.

[1471] A "sensor" is a device that measures physical or chemical quantities and outputs them as data. Optical sensors and image recognition sensors are particularly used.

[1472] A "server" is a computer system that provides services such as data storage, processing, and distribution over a network.

[1473] An "AI model" is an algorithm that has been trained to perform a specific task using artificial intelligence techniques, particularly frameworks such as TensorFlow and PyTorch.

[1474] A "panel" is an interface for displaying information. It is installed in washing machines and other appliances.

[1475] A "user's terminal" is a device used by a user, such as a computer or smartphone.

[1476] A "product line" refers to the equipment and facilities used to continuously produce products within a factory.

[1477] "Stain level" refers to the degree of foreign matter adhering to the surface of an object and the resulting discoloration.

[1478] "Data analysis" is the process of analyzing collected data and extracting meaningful information.

[1479] A "cleaning operation" is a manual or automated process for removing dirt or foreign matter.

[1480] "Shipment suspension" means stopping the shipment of a product, and is done to ensure quality.

[1481] "Emotional state" refers to the current psychological state of the user or operator.

[1482] The present invention is a system that uses a robot installed on a factory production line to automatically detect dirt on products and parts, and based on the results, decides whether to perform cleaning work or stop product shipments. This system is composed of the following elements.

[1483] 1. Dirt detection by sensor

[1484] The robot is equipped with optical sensors and image recognition sensors that constantly monitor the production line and detect the degree of dirt on products and parts. The data detected by the sensors is temporarily collected on a terminal inside the robot.

[1485] 2. Data transmission

[1486] The collected data is sent to a cloud server via Wi-Fi or a wired connection, and the device uses a communication protocol to package and efficiently transmit the data.

[1487] 3. Analysis using AI models

[1488] The server uses a generative AI model to analyze the data it receives, specifically an AI model based on frameworks such as TensorFlow and PyTorch, to analyze the data and classify the level of dirt into categories such as "lightly dirty," "moderately dirty," and "heavily dirty."

[1489] 4. Recognition of user emotions using an emotion engine

[1490] An emotion engine is used to recognize the emotional state of the operator through a smartphone app or computer. If the operator is under stress or overload, the emotion engine will detect this.

[1491] 5. Notification of Results

[1492] The analysis results are sent to the robot's terminal and the operator's smartphone or computer. The terminal displays messages such as "Cleaning required" or "Shipping suspended" on the robot's panel, and the same information is also sent to the operator's terminal.

[1493] 6. User Selection and Autorun

[1494] Based on the analysis results, the operator can choose to carry out cleaning work, cancel the work, or stop shipping. If the operator's emotional state is stressed, the system will automatically carry out cleaning work or stop shipping.

[1495] This system will automatically detect contamination and take necessary measures, and is expected to reduce the burden and mental stress on operators.

[1496] As a concrete example, you can use the following prompt to have an AI model perform an analysis:

[1497] It judges the degree of soiling of the product line based on images, and starts cleaning if the soiling score exceeds the previous threshold, or skips cleaning if it does not. It also starts cleaning if the operator's emotional state is stressed or overloaded.

[1498] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1499] Step 1:

[1500] Dirt detection by sensors

[1501] Optical sensors and image recognition sensors installed on the robot detect the degree of dirt on products and parts on the production line. The input is the physical condition of the product or part, and the output is a measurement value of the degree of dirt as digital data. This measurement value is temporarily stored on a terminal inside the robot.

[1502] Step 2:

[1503] Sending data

[1504] The device transmits the collected soiling data to a cloud server via Wi-Fi or a wired connection. The input is the measurement data stored in the device, and the output is the data transferred to the cloud server. The device packages the data for efficient transmission.

[1505] Step 3:

[1506] Analysis using AI models

[1507] The server analyzes the received data using a generative AI model. Specifically, it uses a pre-trained model based on TensorFlow and PyTorch to classify the degree of dirt into categories such as "lightly dirty," "moderately dirty," and "heavily dirty." The input is the digital data of the dirt level sent to the cloud server, and the output is the classification result.

[1508] Step 4:

[1509] Recognizing user emotions with an emotion engine

[1510] The emotion engine recognizes the operator's emotional state through the user's smartphone app or computer. The input is voice data and image data provided by the operator, and the output is the judgment result of the operator's emotional state. The emotion engine uses technologies such as OpenVINO.

[1511] Step 5:

[1512] Notification of analysis results and emotional state

[1513] The server notifies the robot's terminal and the operator's smartphone or computer of the analysis results and emotional state. The input is the classification results of the dirt level analyzed by the server and the judgment results of the emotion engine, and the output is a notification message. The robot's panel displays messages such as "Cleaning required" or "Shipping suspended."

[1514] Step 6:

[1515] User selection and automatic execution

[1516] Based on the notified analysis results, the operator can choose whether to carry out cleaning work or stop shipments. The input is the notification message and the operator's choice, and the output is the start of cleaning work or the stop of shipments. If the operator's emotional state is stressed, cleaning work or shipments will be automatically stopped.

[1517] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1518] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1519] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1520] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1521] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1522] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1523] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1524] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1525] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1526] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1527] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1528] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1529] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1530] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1531] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1532] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1533] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1534] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1535] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1536] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1537] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1538] The following is further disclosed regarding the above embodiment.

[1539] (Claim 1)

[1540] A means to detect the degree of dirtiness of clothes using a sensor installed in the washing machine,

[1541] A means for analyzing the detected data using an AI model generated on the server and determining whether or not the clothes need to be washed;

[1542] means for notifying a panel of the washing machine and a user's terminal of the result that the clothes do not need to be washed;

[1543] A means for allowing a user to select whether to perform or stop washing;

[1544] A system including:

[1545] (Claim 2)

[1546] The system according to claim 1, further comprising displaying the analysis results on the washing machine panel and on the user's terminal.

[1547] (Claim 3)

[1548] The system according to claim 1, characterized in that the user can choose whether to carry out or stop washing based on the analysis results through the panel of the washing machine or a terminal.

[1549] "Example 1"

[1550] (Claim 1)

[1551] A means to detect the degree of dirtiness of clothes using a sensor installed in the washing machine,

[1552] means for transmitting the detected data via the terminal to a server via the Internet;

[1553] A means for analyzing the received data and determining whether or not the clothes need to be washed using an AI model generated on the server;

[1554] a means for notifying the analysis result to the operation panel of the washing machine and a user's terminal;

[1555] A means for allowing the user to select whether to perform or stop the washing based on the analysis results;

[1556] A system including:

[1557] (Claim 2)

[1558] The system according to claim 1, characterized in that the analysis results are displayed on the operation panel of the washing machine and on the user's terminal.

[1559] (Claim 3)

[1560] The system according to claim 1, characterized in that the user can select whether to carry out or cancel the washing based on the analysis results through the operation panel or terminal of the washing machine.

[1561] "Application Example 1"

[1562] (Claim 1)

[1563] A means to detect the degree of dirtiness of clothes using a sensor installed in the washing machine,

[1564] A means for analyzing the detected data using an AI model generated on the server and determining whether or not the clothes need to be washed;

[1565] means for notifying a display device of the washing machine and an information communication device of the user that the laundry does not need to be washed;

[1566] A means for allowing a user to select whether to perform or stop washing;

[1567] A means of detecting abnormalities within the home using cameras and chemical sensors installed in the home,

[1568] A means for analyzing abnormal conditions in the home and determining the degree of abnormality using an AI model generated on the server;

[1569] means for notifying a display device in the home and a user's information communication device of the result when an abnormality occurs in the home;

[1570] a means for allowing a user to select whether to activate or deactivate an alarm;

[1571] A system including:

[1572] (Claim 2)

[1573] 2. The system according to claim 1, further comprising a step of displaying the analysis results on a display device in the home and on the user's information communication device.

[1574] (Claim 3)

[1575] 2. The system according to claim 1, wherein a user can select whether to issue or cancel an alarm based on the analysis results through a display device or an information and communication device in the home.

[1576] "Example 2: Combining Emotion Engines"

[1577] (Claim 1)

[1578] A means to detect the degree of dirtiness of clothes using a sensor installed in the washing machine,

[1579] A means for analyzing the detected data using an AI model generated on the server and determining whether or not the clothes need to be washed;

[1580] means for recognizing a user's emotional state using an emotion engine and suggesting or automatically initiating laundry actions based thereon;

[1581] means for notifying a panel of the washing machine and a user's terminal of the result that the clothes do not need to be washed;

[1582] A means for allowing a user to select whether to perform or stop washing;

[1583] A system including:

[1584] (Claim 2)

[1585] The system according to claim 1, further comprising displaying the analysis results on the washing machine panel and on the user's terminal.

[1586] (Claim 3)

[1587] The system according to claim 1, characterized in that the user can choose whether to carry out or stop washing based on the analysis results through the panel of the washing machine or a terminal.

[1588] (Claim 4)

[1589] 2. The system of claim 1, wherein the emotion engine suggests or automatically initiates a laundry action when the user's emotional state is recognized as being stressed.

[1590] "Application example 2 when combining emotion engines"

[1591] (Claim 1)

[1592] A means to detect the degree of dirtiness of clothes using a sensor installed in the washing machine,

[1593] A means for analyzing the detected data using an AI model generated on the server and determining whether or not the clothes need to be washed;

[1594] means for notifying a panel of the washing machine and a user's terminal of the result that the clothes do not need to be washed;

[1595] A means for allowing a user to select whether to perform or stop washing;

[1596] A means of automatically detecting the contamination status of product lines and parts,

[1597] A method for determining whether to carry out cleaning work or stop shipments based on the contamination status;

[1598] A means for recognizing the user's emotional state and suggesting cleaning or shipping actions;

[1599] A system including:

[1600] (Claim 2)

[1601] The system according to claim 1, further comprising displaying the analysis results on the washing machine panel and on the user's terminal.

[1602] (Claim 3)

[1603] The system according to claim 1, characterized in that the user can choose whether to carry out or stop washing based on the analysis results through the panel of the washing machine or a terminal. [Explanation of symbols]

[1604] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means to detect the degree of dirtiness of clothes using a sensor installed in the washing machine, A means for analyzing the detected data using an AI model generated on the server and determining whether or not the clothes need to be washed; means for notifying a panel of the washing machine and a user's terminal of the result that the clothes do not need to be washed; A means for allowing a user to select whether to perform or stop washing; A system including:

2. The system according to claim 1, further comprising a function of displaying the analysis results on a panel of the washing machine and on a user's terminal.

3. 2. The system according to claim 1, wherein the user can select whether to carry out or stop washing based on the analysis results through a panel on the washing machine or a terminal.

Images