System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting data from a plurality of sensors installed in a living space; means for storing the data collected from the sensors in a database; means for analyzing the stored data and learning a behavior pattern and a preference of a user; means for generating a proposal for performing optimal environment setting on the basis of the analysis result; and means for notifying the user of the proposal and controlling a home appliance on the basis of the proposal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Smart home technologies have been gaining attention in recent years for their ability to improve the comfort of living spaces. However, it is difficult to provide accurate recommendations based on users' behavioral patterns and preferences. Furthermore, efficient control of home appliance usage is necessary to achieve an eco-conscious lifestyle. However, many existing systems do not fully meet these requirements. Specifically, they often lack sufficient analysis of data collected from sensors, or provide inappropriate recommendations to users. This invention aims to solve these issues and achieve both user comfort and an eco-conscious lifestyle. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for collecting data from multiple sensors installed in a living space, a means for storing the data collected from the sensors in a database, a means for analyzing the stored data and learning a user's behavioral patterns and preferences, a means for generating suggestions for optimal environmental settings based on the analysis results, and a means for notifying the user of the suggestions and controlling home appliances based on the suggestions. This system enables comfortable environmental settings tailored to the user's lifestyle and eco-friendly control of home appliances. The sensors detect temperature, humidity, illuminance, and motion, and the collected data is analyzed using AI technology. The analysis results are notified to the user via a mobile device, and the home appliances are automatically controlled to the optimal settings.

[0006] "Sensor" refers to a device that detects temperature, humidity, light, and motion and transmits that information to the system.

[0007] "Database" refers to a storage system that stores collected sensor data in an organized manner and makes it available for later analysis.

[0008] "Analysis" refers to the data processing and pattern recognition processes used to understand user behavior patterns and preferences using collected data.

[0009] "User" refers to the person who uses this smart home system within their living space.

[0010] "Behavioral patterns" refer to the user's daily movements and tendencies, and are detected through data analysis.

[0011] "Preferences" refer to particular preferences or usage patterns that a user finds particularly comfortable.

[0012] "Suggestion" refers to advice on optimal environment settings and actions presented to the user based on the analysis results.

[0013] "Notification" refers to a messaging mechanism to inform users of generated suggestions.

[0014] "Home appliances" refers to electrical equipment such as air conditioners, lighting, and heaters used in living spaces.

[0015] "Control" refers to an operation to automatically or manually change the settings of a home appliance.

[0016] "Living spaces" refers to places where sensors are installed, such as homes and offices.

[0017] "AI technology" refers to artificial intelligence techniques that perform pattern recognition and prediction through data analysis. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The system for implementing the present invention is composed of multiple sensors installed in a living space, a server that manages the data, an AI module that performs analysis, and a terminal that notifies the user.

[0040] First, multiple temperature, humidity, illuminance, and motion sensors are placed in the living space. These sensors continuously collect various environmental data and send it to a server.

[0041] sensor

[0042] The temperature sensor measures the room temperature in real time and sends the value to the server.

[0043] The humidity sensor measures the humidity level in the room and sends the data to a server.

[0044] The illuminance sensor detects the on / off status of the lights and the brightness level, and sends that information to the server.

[0045] The motion sensor detects the user's movement and reports to the server whether the user is in the room.

[0046] server

[0047] The server receives the data sent from each sensor in real time and stores it in a database, which accumulates a wealth of data in chronological order.

[0048] The received data is periodically analyzed by an AI module, which is a process that learns user behavior patterns and preferences based on the collected data.

[0049] AI Module

[0050] The AI ​​module analyzes past data and uses clustering techniques to understand user behavior patterns, such as air conditioning usage frequency and lighting preferences, and then derives comfortable environmental settings based on that information.

[0051] Based on the analysis results, suggestions are generated for optimal air conditioning temperatures, lighting levels, and even ways to increase eco-consciousness.

[0052] Terminal

[0053] The server then sends the generated suggestions to the user's device, such as a smartphone or tablet, including recommended air conditioning temperature settings and lighting adjustment levels.

[0054] Users can receive notifications and manually change preferences based on the suggestions provided, but the system also has the ability to automatically change settings on home appliances.

[0055] Specific examples

[0056] For example, if the temperature sensor detects 30°C in the summer and the humidity sensor indicates 50%, the AI ​​module will analyze this and generate a suggestion to set the air conditioner to 24°C, which the user finds comfortable. Furthermore, based on data from the light and motion sensors, it will suggest that the optimal lighting level is 70% when the user is confirmed to be in the room. These suggestions are sent to the user's device, and if the user accepts, the server will automatically change the air conditioner and lighting settings.

[0057] In this way, the present invention utilizes the abundant data obtained from sensors to improve both user comfort and environmental awareness.

[0058] The processing flow will be explained below.

[0059] Step 1:

[0060] Sensors periodically measure temperature, humidity, light, and motion within the living space, and transmit the measurements to a server in real time.

[0061] Step 2:

[0062] The server receives data (temperature, humidity, illuminance, motion) sent from each sensor and stores it in a database in chronological order. The stored data is organized by user.

[0063] Step 3:

[0064] The AI ​​module installed on the server periodically analyzes the data in the database, using clustering techniques and machine learning algorithms to learn user behavior patterns and preferences.

[0065] Step 4:

[0066] Based on the analysis results, the server identifies the environmental conditions that the user finds most comfortable (for example, the optimal air conditioning temperature or lighting brightness). This information is generated as a suggestion.

[0067] Step 5:

[0068] The server then sends the generated suggestions to the user's device, including specific suggestions such as "change the air conditioner temperature setting to 24°C" or "adjust the lighting brightness to 70%."

[0069] Step 6:

[0070] The user checks the notification and can either manually follow the suggestions or choose to have the system automatically adjust the settings for air conditioning and lighting according to the user's selection.

[0071] Step 7:

[0072] The server then stores the results of the configuration changes in the database and uses them for the next analysis and proposal generation. This process is repeated, continuously supporting the user's comfort and eco-conscious lifestyle.

[0073] Example 1

[0074] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0075] In modern residential environments, achieving efficient energy use while increasing user comfort is an important challenge. However, existing systems have difficulty automatically optimizing environmental settings based on user behavior patterns and preferences. Furthermore, environmental settings are often proposed and implemented manually, requiring cumbersome operations for users, making it difficult to maintain a continuously comfortable environment.

[0076] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0077] In this invention, the server includes means for collecting data from a plurality of sensors installed in the living space, means for storing the data collected from the sensors in a database in chronological order, means for analyzing the stored data using clustering technology to learn the user's behavioral patterns and preferences, means for generating suggestions for comfortable environmental settings based on the analysis results, means for notifying the user's terminal of the suggestions and automatically controlling the home appliances based on the suggestions, and means for re-collecting environmental data based on the suggested settings and continuously storing it in the database. This enables automatic and continuous optimization of environmental settings based on the data collected from the sensors, allowing the user to maintain a comfortable living space without having to perform complicated operations.

[0078] A "sensor" is a device installed in a living space to detect temperature, humidity, light, and motion.

[0079] A "server" is a device that receives data collected from multiple sensors, stores it in a database, and then analyzes and generates proposals.

[0080] A "database" is a storage device for storing data collected from sensors in chronological order.

[0081] "Clustering technology" is a method of analyzing stored data and grouping similar data together, and is used to learn user behavior patterns and preferences.

[0082] A "terminal" is a device such as a smartphone or tablet that receives proposals from the server and notifies the user.

[0083] A "suggestion" is a specific instruction for creating a comfortable environment, generated based on the analysis results.

[0084] "Home appliances" are electrical appliances in living spaces, such as air conditioners and lighting, that are controlled based on proposals.

[0085] The system consists of multiple sensors installed in living spaces, a server that manages the data, an AI module that performs analysis, and a terminal that notifies users.

[0086] First, multiple temperature sensors, humidity sensors, illuminance sensors, and motion sensors are placed in the living space. These sensors continuously collect various environmental data and send that data to a server. For example, if the temperature sensor sends data such as "The current temperature is 30°C," the humidity sensor may send data such as "The current humidity is 50%."

[0087] The server then receives this data in real time and stores it in a database in chronological order. For example, at 2:30 PM on October 5, 2023, the following data will be recorded: temperature 30°C, humidity 50%, lighting 100 lux, and user presence. This makes it possible to refer to past environmental data.

[0088] The stored data is analyzed by an AI module that uses clustering technology. This AI module uses past data to learn user behavior patterns and preferences. For example, the analysis may yield results such as "The optimal air conditioner temperature setting in summer is 24°C."

[0089] Based on the analysis results, the AI ​​module will generate specific suggestions for comfortable environmental settings. For example, it may suggest, "Today's weather is hot, so I recommend setting the air conditioner temperature to 24°C." It may also recommend, "It would be good to set the lighting to 70% when you are in the room."

[0090] This suggestion is sent from the server to the user's device, which can be a smartphone or tablet, and the notification includes specific instructions such as, "The current temperature is 30°C and the humidity is 50%. Please change the air conditioner's temperature setting to 24°C."

[0091] If the user approves the proposal, the server automatically changes the settings of the home appliance. For example, if the user approves the proposal to set the air conditioner to 24°C, the server will change the air conditioner's setting to 24°C. This allows the user to maintain a comfortable living space without having to perform complicated operations.

[0092] Furthermore, environmental data after the settings are changed is collected again from the sensors and saved in the database. This allows the AI ​​module to continuously analyze the new data. For example, the data that indicates the user's comfort level after setting the air conditioner to 24°C and humidity to 45% is saved.

[0093] A concrete example of a prompt is:

[0094] "Please suggest the optimal air conditioner setting based on the temperature data from the past week."

[0095] This text is input into the generative AI model.

[0096] This series of steps enables the system to automatically and continuously optimize environmental settings based on data obtained from sensors, providing users with a comfortable and energy-efficient living environment.

[0097] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0098] Step 1:

[0099] The server collects data from multiple sensors installed in the living space. Specifically, it receives data sent from temperature sensors, humidity sensors, illuminance sensors, and motion sensors in real time. The input is data from each sensor, and the output is the received sensor information. For example, it receives data such as temperature 30°C, humidity 50%, illuminance 100 lux, and whether the user is present in the room.

[0100] Step 2:

[0101] The server stores the received sensor data in a database in chronological order. The input is the sensor data received in step 1, and the output is the log-format data stored in the database. Specifically, the following is recorded in the database: "October 5, 2023, 14:30, temperature 30°C, humidity 50%, illuminance 100 lux, user present."

[0102] Step 3:

[0103] The server provides the stored data to the AI ​​module, which analyzes past data using clustering technology to learn the user's behavioral patterns and preferences. The input is environmental data stored in the database, and the output is analyzed patterns and preference information. Specifically, it extracts information such as air conditioner usage frequency and lighting preferences.

[0104] Step 4:

[0105] The AI ​​module generates recommendations for optimal environmental settings based on the analysis results. The input is the behavioral patterns and preference information obtained in step 3, and the output is specific recommendations. For example, a recommendation might be generated such as, "Since the current temperature is 30°C, we suggest setting the air conditioner to 24°C."

[0106] Step 5:

[0107] The server notifies the user's device of the generated suggestion. The input is the suggestion from the AI ​​module, and the output is a notification message displayed on the user's device. Specifically, a notification is sent to the smartphone saying, "Please change the air conditioner's temperature setting to 24°C."

[0108] Step 6:

[0109] The user accepts or rejects the proposal. The input is the proposal displayed on the terminal, and the output is the result of the operation: accept or reject. For example, the user accepts the proposal "change the air conditioner temperature setting to 24°C."

[0110] Step 7:

[0111] The server controls the home appliances based on the user's approval. The input is the user's approval result, and the output is the state of the home appliances after the settings are changed. Specifically, the air conditioner's set temperature is changed to 24°C.

[0112] Step 8:

[0113] The server collects the environmental data again after the settings are changed and stores it in the database. The input is the new data from the sensor, and the output is the log-format data added to the database. Specifically, the air conditioner settings of 24°C and 45% humidity, as well as feedback from the user about their comfort level, are also stored.

[0114] By repeating this process, the system automatically and continuously optimizes environmental settings based on sensor data, providing users with a comfortable and energy-efficient living environment.

[0115] (Application example 1)

[0116] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0117] In modern living spaces, environmental conditions such as temperature, humidity, and illuminance have a significant impact on people's comfort. However, it is difficult to automatically provide optimal environmental settings based on the behavior and preferences of individual users or customers. In particular, in commercial spaces, efficient environmental control while maintaining customer comfort is required. The present invention aims to solve these problems and provide optimal environmental settings in living and commercial spaces.

[0118] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0119] In this invention, the server includes means for collecting data from multiple sensors installed in the living space, means for storing the data collected from the sensors in a database, means for analyzing the stored data and learning user behavior patterns and preferences, means for generating suggestions for optimal environmental settings based on the analysis results, means for notifying the user of the suggestions and controlling electronic devices based on the suggestions, and means for analyzing customer traffic patterns in commercial spaces using data collected from the sensors and suggesting comfort settings. This makes it possible to analyze environmental data in living spaces and commercial spaces in real time and automatically provide optimal environmental settings that increase the comfort of users and customers.

[0120] "Living space" refers to the space in which people live their daily lives, and generally refers to the indoor environment of homes, offices, etc.

[0121] A "sensor" is a device that detects environmental data or conditions, such as temperature, humidity, light, or motion, and transmits that information electronically.

[0122] "Database" means a system or platform for storing and managing collected environmental data.

[0123] "Analysis" is the processing of collected data to extract meaningful information and patterns.

[0124] "Behavioral patterns" refer to the regularity and characteristics of a series of actions or behaviors of users or customers.

[0125] "Preferences" refers to the individual tastes and tendencies of users or customers to find particular environmental conditions or settings comfortable.

[0126] "Suggestions" are information or instructions that recommend optimal environmental settings or actions based on the analysis results.

[0127] "Notification" is the act or means of conveying information such as suggestions or warnings to users.

[0128] "Electronic devices" refers to all electrically operated devices, such as home appliances, computing devices, and lighting equipment.

[0129] "Commercial space" refers to a space used for business purposes, such as a store or office where customers visit.

[0130] "Customer traffic flow" refers to the route or path that customers take within a commercial space.

[0131] "Comfort settings" refers to adjusting environmental conditions to make users or customers feel most comfortable.

[0132] This invention is a system consisting of multiple sensors installed in living and commercial spaces, a server that manages and analyzes the data, and a terminal that notifies the results of the analysis. Each element that makes up this system operates as follows.

[0133] 1. Sensor installation and data collection

[0134] Living and commercial spaces are equipped with multiple temperature, humidity, light, and motion sensors that continuously collect environmental data and transmit it to a server.

[0135] Temperature sensor: Measures the room temperature in real time and sends the value to the server.

[0136] Humidity sensor: Measures the humidity level in the room and sends the data to the server.

[0137] Illuminance sensor: Detects the on / off status of the light and the brightness level, and sends that information to the server.

[0138] Motion sensor: Detects the user's movement and reports to the server whether the user is in the room.

[0139] 2. Saving to the database

[0140] The server receives the data sent from each sensor in real time and stores it in a database, which accumulates a wealth of data in chronological order.

[0141] 3. Analysis and learning by AI module

[0142] The received data is periodically analyzed by an AI module, which is a process that learns the behavioral patterns and preferences of users and customers based on the collected data.

[0143] AI module: Analyzes past data and learns things like air conditioner usage frequency and lighting preferences. To do this, it uses clustering technology. Based on the analysis results, it derives the optimal air conditioner temperature and lighting level.

[0144] 4. Proposal generation and notification

[0145] Based on the analysis results, suggestions for optimal environmental settings are generated, including adjusting the air conditioner temperature setting and lighting levels. The generated suggestions are then sent to the user's device.

[0146] Notifications: Suggestions include recommended air conditioning temperature settings and lighting adjustment levels. Users can receive notifications and manually change their environmental settings according to the suggestions provided, but the system also has the ability to automatically change the settings of their home appliances.

[0147] 5. Examples of applications in commercial spaces

[0148] In commercial spaces in particular, data collected from sensors is used to analyze customer movement patterns and propose comfortable settings.

[0149] Example: If the temperature inside a store is 27°C, humidity is 60%, and the illuminance is 200 lumens, and a motion sensor detects the presence of a customer, a suggestion will be generated to set the air conditioner temperature to 24°C and adjust the lighting to 500 lumens.

[0150] 6. Hardware and Software Used

[0151] Hardware: Temperature, humidity, light, motion sensors (e.g. Bosch BME280 sensor, PIR motion sensor)

[0152] Software: Python, Flask or Django, TensorFlow or PyTorch, SQLite or PostgreSQL

[0153] Example prompt sentence:

[0154] The temperature in your store is 27°C and the humidity is 60%. The light sensor detects 200 lumens and the motion sensor detects the presence of a customer. To maintain a comfortable environment, set the air conditioning to 24°C and adjust the lighting to 500 lumens.

[0155] According to this embodiment, the present invention can analyze environmental data in real time in living spaces and commercial spaces, and automatically provide optimal environmental settings that increase the comfort of users and customers.

[0156] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0157] Step 1:

[0158] Multiple sensors installed in living and commercial spaces collect temperature, humidity, illuminance, and motion data in real time. The input data from the sensors is sent to a server. Specifically, the temperature sensor detects room temperature, the humidity sensor detects humidity level, the illuminance sensor detects lighting brightness level, and the motion sensor detects movement. The output at this stage is raw data sent from each sensor to the server.

[0159] Step 2:

[0160] The server receives the data sent from each sensor in real time and stores it in a database. The server receives input data from the sensors, converts it into an appropriate format, and records it in the database. This database stores a wealth of time-series data. For example, temperature, humidity, illuminance, and motion data are recorded in time series. The output of this step is the sensor information stored in the database.

[0161] Step 3:

[0162] The server periodically analyzes the historical data stored in the database using an AI module. This analysis learns the behavioral patterns and preferences of users and customers based on the collected data. The server provides the AI ​​module with historical sensor data as input, and the AI ​​module uses clustering techniques to identify significant patterns. For example, it learns preferences for temperature and light intensity at specific times of the day. The output of this step is a model of the behavioral patterns and preferences resulting from the analysis.

[0163] Step 4:

[0164] The server generates a proposal for optimal environmental settings based on the analysis results from the AI ​​module. Specifically, it calculates and generates the optimal air conditioner temperature and lighting level proposals based on the output of the AI ​​module. The server supplies the proposals as input to the generation AI model, which outputs the optimal environmental conditions. For example, when the temperature is 27°C and the humidity is 60%, the air conditioner should be set to 24°C and the lighting adjusted to 500 lumens. The output of this step is the recommended settings.

[0165] Step 5:

[0166] The server notifies the terminal of the generated proposal. The terminal receives the proposal and informs the user or store clerk. The notification may include recommended air conditioner temperature settings or lighting adjustment levels. The terminal may also directly control home appliances based on the notified information. The input to this step is the proposal data from the server, and the output is a notification display on the terminal or control operation of the home appliance.

[0167] Step 6:

[0168] The user or store clerk can manually change the environmental settings based on the notified suggestions, or if automatic control is enabled, the system will automatically change the settings of the home appliances. The user can adjust the temperature and lighting according to the received suggestions using a device such as a smartphone or tablet. Specifically, the user can confirm the suggestions and press the approval button, which causes the server to change the air conditioner and lighting settings. The output of this step is the adjusted home appliance settings.

[0169] Example prompt sentence:

[0170] The temperature in your store is 27°C and the humidity is 60%. The light sensor detects 200 lumens and the motion sensor detects the presence of a customer. To maintain a comfortable environment, set the air conditioning to 24°C and adjust the lighting to 500 lumens.

[0171] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0172] The system for implementing the present invention is composed of multiple sensors installed in the living space, a server that manages and analyzes the data, an emotion engine, and a terminal that notifies the user.

[0173] First, temperature, humidity, light, and motion sensors are placed in the living space to continuously collect various environmental data and send it to a server.

[0174] sensor

[0175] The temperature sensor measures the room temperature in real time and sends the value to the server.

[0176] The humidity sensor measures the humidity level in the room and sends the data to a server.

[0177] The illuminance sensor detects the on / off status of the lights and the brightness level, and sends that information to the server.

[0178] The motion sensor detects the user's movement and reports to the server whether the user is in the room.

[0179] server

[0180] The server receives data (temperature, humidity, illuminance, motion) sent from each sensor and stores it in a database in chronological order. The stored data is organized by user.

[0181] Furthermore, the server is equipped with an emotion engine that can recognize the user's emotions through voice analysis, facial expression recognition, and behavior analysis.

[0182] Emotion Engine

[0183] The server uses an emotion engine to analyze input data from the camera and microphone and determine the user's emotions from their facial expressions and tone of voice. For example, if the user is feeling stressed, the emotion engine will recognize that information.

[0184] The recognized emotion data is also stored in a database along with other sensor data.

[0185] Data analysis and proposal generation

[0186] The AI ​​module installed on the server periodically analyzes the data in the database, using clustering techniques and machine learning algorithms to learn user behavior patterns and preferences.

[0187] The data analysis results are combined with the output of the emotion engine to identify the environmental conditions that make the user feel most comfortable (for example, the optimal air conditioning temperature or lighting brightness). This information is then generated as a suggestion.

[0188] Notification and Execution

[0189] The server then sends the generated suggestions to the user's device, such as a smartphone or tablet. These suggestions include specific suggestions such as "change the air conditioner temperature setting to 24°C" or "adjust the lighting brightness to 70%."

[0190] The user checks the notification and can either manually follow the suggestions or choose to have the system automatically adjust the settings for air conditioning and lighting according to the user's selection.

[0191] Specific examples

[0192] For example, if the temperature sensor detects 30°C in the summer, the humidity sensor indicates 50%, and the emotion engine recognizes that the user is irritated, the AI ​​module will make a comprehensive judgment and generate a suggestion to set the air conditioner to 24°C, which the user finds comfortable. Furthermore, when data from the illuminance sensor and motion sensor confirms that the user is in the room, the AI ​​module will suggest that the optimal lighting level is 70%. These suggestions are notified to the user's device, and if the user agrees, the server will automatically change the air conditioner and lighting settings.

[0193] In this way, the present invention realizes a system that balances user comfort with improved environmental awareness based on the abundant data acquired from sensors and emotion engines.

[0194] The processing flow will be explained below.

[0195] Step 1:

[0196] Sensors periodically measure temperature, humidity, light, and motion within the living space, and transmit the measurements to a server in real time.

[0197] Step 2:

[0198] The server receives data (temperature, humidity, illuminance, motion) sent from each sensor and stores it in a database in chronological order. The stored data is organized by user.

[0199] Step 3:

[0200] The emotion engine installed on the server receives input data from the camera or microphone, analyzes the user's facial expressions and tone of voice from this data, and recognizes the user's emotions. For example, it determines whether the user is stressed or relaxed.

[0201] Step 4:

[0202] The server stores the recognized emotion data in a database along with other sensor data, which associates the emotion data with behavioral patterns and environmental data.

[0203] Step 5:

[0204] An AI module installed on the server periodically analyzes the sensor data and emotion data stored in the database, and uses clustering techniques and machine learning algorithms to learn the user's behavioral and emotional patterns.

[0205] Step 6:

[0206] Based on the analysis results, the server identifies the environmental conditions that the user finds most comfortable (for example, the optimal air conditioning temperature or lighting brightness), and generates optimal environmental setting suggestions, taking into account emotional data.

[0207] Step 7:

[0208] The server then sends the generated suggestions to the user's smartphone, tablet, or other device. The content of the notification includes specific suggestions such as "change the air conditioner temperature setting to 24°C" or "adjust the lighting brightness to 70%."

[0209] Step 8:

[0210] Users receive a notification and can manually configure their preferences based on the suggestions provided, or choose automatic configuration, and in some cases even fine-tune the settings.

[0211] Step 9:

[0212] The server automatically adjusts the temperature settings of the air conditioner and the brightness level of the lights if the user selects automatic settings, but also collects setting change information if the user manually selects settings.

[0213] Step 10:

[0214] The server then stores the results of the changes in the database and uses them for the next analysis and proposal generation. By repeating this process, the system continuously supports users' comfort and eco-conscious lifestyles.

[0215] Example 2

[0216] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0217] In modern society, it is extremely important for users to maintain a comfortable living environment. However, many current systems simply collect and display data from various sensors and are unable to precisely adjust the environment to reflect the user's emotions and behavioral patterns. This poses a problem, requiring users to frequently manually configure the environment. Furthermore, these manual operations can cause stress for users. Therefore, there is a need for a system that automatically provides optimal environmental conditions based on the user's emotions and behavioral patterns.

[0218] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0219] In this invention, the server includes: means for collecting data from multiple sensors installed in the living space; means for storing the data collected from the sensors in a database; emotion recognition means for recognizing a user's emotions based on the data stored in the database; means for analyzing the stored data and the emotion data acquired by the emotion recognition means and learning the user's behavioral patterns and preferences; means for generating a proposal for optimal environmental settings based on the analysis results and the data from the emotion recognition means; and means for notifying the user of the proposal and controlling environmental devices based on the proposal. This enables precise environmental adjustment that reflects the user's emotions and behavioral patterns, eliminating the need for the user to manually adjust settings. Furthermore, it is possible to improve the user's comfort and reduce stress.

[0220] A "sensor" is a device that measures environmental information in living spaces (temperature, humidity, illuminance, motion, etc.) and collects that data.

[0221] The "database" is a system for storing and managing collected sensor data and emotion data in chronological order.

[0222] The "emotion recognition means" is a function that analyzes data obtained from input devices such as cameras and microphones and determines the user's emotional state (for example, stress or a sense of relief).

[0223] A "behavioral pattern" is a set of tendencies and habits extracted based on a user's daily actions and activities.

[0224] "Analysis" is the process of learning user behavior patterns and preferences using data stored in the database and data obtained from the emotion recognition means.

[0225] "Suggestions" are specific instructions or recommendations based on the analysis results to help users create optimal environmental settings that will allow them to feel comfortable.

[0226] "Notification" refers to the act and means of informing the user of the generated proposal via an information display device or the like.

[0227] "Control" is the function of actually changing the settings of environmental equipment such as air conditioners and lighting based on suggestions and performing the operations.

[0228] The system for implementing this invention is composed of multiple sensors installed in the living space, a server that manages and analyzes the data, an emotion recognition engine, and a terminal that notifies the user. A detailed description of each component is provided below.

[0229] Data collection by sensors

[0230] Temperature sensors, humidity sensors, illuminance sensors, and motion sensors are placed in the living space. These sensors measure environmental information in real time and send the data to a server. For example, sensors connected to a Raspberry Pi or Arduino are used. The temperature sensor continuously monitors the temperature in the room, and the humidity sensor measures the humidity level. The illuminance sensor detects the brightness and on / off status of the lights, and the motion sensor detects the user's movement. All of this data is sent to the server.

[0231] Receiving and storing data by the server

[0232] The server is built using web frameworks such as Flask or Django and receives real-time data sent from each sensor. The received data is stored in chronological order in a database system such as PostgreSQL or MySQL. The data is then organized and categorized for each user within the database.

[0233] Emotion data analysis using an emotion recognition engine

[0234] The emotion recognition engine installed on the server analyzes input data from the camera and microphone to recognize the user's emotions. This engine uses libraries such as OpenCV and TensorFlow. For example, video data from the camera is analyzed using OpenCV to recognize facial expressions. Audio data from the microphone is analyzed using TensorFlow to determine the user's emotion from the tone of voice. The recognized emotion data is stored in a database, just like sensor data.

[0235] Data analysis and proposal generation

[0236] The AI ​​module installed on the server periodically analyzes the data in the database. This analysis uses clustering techniques (e.g., K-means clustering) and machine learning algorithms (e.g., random forests). The AI ​​module uses these techniques to learn the user's behavioral patterns and preferences. Based on the analysis results, it identifies the environmental conditions (e.g., air conditioner temperature setting and lighting brightness) that make the user feel most comfortable and generates specific recommendations.

[0237] Notification and Execution

[0238] The server sends the generated suggestions to devices such as smartphones and tablets. The notifications include specific suggestions such as "set the air conditioner to 24°C" and "adjust the lighting to 70%." The user can then review the notifications and choose to follow the suggestions or select automatic settings. If the user agrees, the server will automatically change the air conditioner and lighting settings.

[0239] Specific examples

[0240] For example, if the temperature sensor detects 30°C in the summer, the humidity sensor indicates 50%, and the emotion recognition engine recognizes that the user is irritated, the AI ​​module will make a comprehensive judgment and generate a suggestion to set the air conditioner to 24°C, a temperature that the user finds comfortable. It will also use data from the light and motion sensors to confirm that the user is in the room and suggest an optimal lighting level (70%). These suggestions are sent to the user's device, and if the user agrees, the server will automatically change the air conditioner and lighting settings.

[0241] Example prompts for generative AI models

[0242] "If the room temperature is 30°C and the humidity is 50% in summer, and the emotion recognition engine detects that the user is irritated, what is the optimal air conditioner setting that the system should suggest?"

[0243] In this way, the present invention utilizes the wealth of data acquired from sensors and emotion recognition engines to improve both user comfort and environmental awareness.

[0244] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0245] Step 1:

[0246] Collecting data from sensors

[0247] Input: Temperature sensor, humidity sensor, light sensor, motion sensor

[0248] Processing: Each sensor collects data every minute.

[0249] The temperature sensor measures the temperature in the room and sends the data to the server.

[0250] The humidity sensor measures the humidity level in the room and sends the data to a server.

[0251] The illuminance sensor measures the brightness and on / off status of the light and sends the data to the server.

[0252] The motion sensor detects the user's movements and sends the data to the server.

[0253] Output: Environmental data sent from each sensor to the server

[0254] Step 2:

[0255] Receiving and storing data by the server

[0256] Input: Real-time data sent from each sensor

[0257] Processing: The server receives the data and stores it in the database.

[0258] Use the Flask or Django web framework to receive data via HTTP requests.

[0259] The received data is stored in chronological order in a PostgreSQL or MySQL database.

[0260] Output: Sensor data stored in a database

[0261] Step 3:

[0262] Emotion data analysis using an emotion recognition engine

[0263] Input: Camera video data, microphone audio data

[0264] Processing: The server analyzes the data obtained from the camera and microphone.

[0265] The camera's video data is analyzed using OpenCV to recognize facial expressions.

[0266] The audio data from the microphone is analyzed using TensorFlow, and emotions are determined from the tone of the voice.

[0267] Output: Emotion data analyzed by the emotion recognition engine

[0268] Step 4:

[0269] Data analysis and proposal generation

[0270] Input: Stored sensor data and emotion data

[0271] Processing: An AI module on the server analyzes the data and generates suggestions.

[0272] Periodically analyze the data in the database.

[0273] Analyze the data using clustering techniques (e.g., K-means clustering) and machine learning algorithms (e.g., random forests).

[0274] Based on the analysis results, specific suggestions for optimal environmental settings (e.g., air conditioner temperature settings and lighting brightness) are generated.

[0275] Output: Generated configuration proposal

[0276] Step 5:

[0277] User notification and execution

[0278] Input: Generated configuration proposal

[0279] Processing: The server notifies the user's device of the proposal and controls the environmental devices.

[0280] The proposal will be sent via push notification to your smartphone or tablet.

[0281] The user checks the notified suggestions and selects manual or automatic settings.

[0282] If the user selects automatic settings, the server sends commands to change the air conditioning and lighting settings.

[0283] Output: Notification to user terminal and change of environmental device settings

[0284] In this way, the system can automatically implement environmental settings that improve user comfort through concrete processing at each step.

[0285] (Application example 2)

[0286] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0287] In order to optimize the working environment in factory facilities and improve worker efficiency, a system is needed that can effectively collect and analyze real-time environmental data and worker emotional data, and generate appropriate control instructions based on that data. Conventional methods do not adequately link environmental data and emotional data, making it difficult to immediately improve the working environment or work efficiency.

[0288] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0289] In this invention, the server includes means for collecting data from a plurality of sensors installed in the living space, means for storing the data collected from the sensors in a database, means for analyzing the stored data and learning the user's behavioral patterns and preferences, means for generating suggestions for optimal environmental settings based on the analysis results, means for notifying the user of the suggestions and controlling home appliances based on the suggestions, means for acquiring emotional data of workers in a factory facility, and means for analyzing the acquired emotional data and environmental data and generating control instructions for optimizing the work environment, thereby enabling immediate improvement of the work environment in the factory facility and improvement of the work efficiency of workers.

[0290] "Living space" refers to the space where residents live their daily lives, such as the indoor environment of a home or office.

[0291] A "sensor" is a general term for a device that detects physical environmental data and outputs that data as an electrical signal. Examples include temperature sensors, humidity sensors, illuminance sensors, and motion sensors.

[0292] A "database" is a collection of information in which collected data is systematically stored and managed so that it can be efficiently searched and analyzed as needed.

[0293] "Means for analyzing data" refers to the methods and devices used to convert collected data into useful information such as user behavior patterns and preferences.

[0294] "User" refers to the person who uses this system, and is the target for improving their comfort and convenience.

[0295] "Behavioral patterns" refer to the actions and patterns of behavior that users repeatedly perform in their daily lives.

[0296] "Environmental data" is physical data collected by sensors, including temperature, humidity, light, and motion.

[0297] "Emotional data" refers to data that indicates the emotional state of a worker or user, as determined from their facial expressions, tone of voice, and behavior.

[0298] "Means for generating suggestions" refers to a method or device that creates specific action suggestions or instructions for changing settings to be provided to users based on the analysis results.

[0299] "Means for notifying proposals" is a general term for methods and devices for notifying users of generated proposals, and includes notification functions on mobile devices, etc.

[0300] "Means for controlling home appliances" refers to methods and devices for automatically operating home appliances such as air conditioners and lights based on analysis results and suggestions.

[0301] "Factory facilities" refers to all environments within buildings and premises where manufacturing or production activities take place.

[0302] "Workers" refers to people engaged in manufacturing or production activities within a factory facility.

[0303] "Control instructions" are specific operational commands issued to optimize the working environment and equipment operation within a factory facility.

[0304] "Analyzing means" is a general term for processing methods and devices that process collected data and convert it into useful information.

[0305] "Optimizing the environment" means adjusting the environment to a state where users or workers can work comfortably and efficiently.

[0306] The system for implementing the present invention is composed of multiple sensors installed within a factory facility, a server that manages and analyzes the data, an emotion engine, and a terminal that notifies workers.

[0307] System Configuration

[0308] sensor

[0309] Temperature sensor: Measures the temperature in the factory in real time and sends the value to the server.

[0310] Humidity sensor: Measures the humidity level in the factory and sends the data to a server.

[0311] Illuminance sensor: Detects whether the light is on or off and the brightness level, and sends that information to the server.

[0312] Motion sensor: Detects worker movement and reports the presence of people in the work area to the server.

[0313] server

[0314] The server receives data (temperature, humidity, illuminance, motion) sent from each sensor and stores it in a database in chronological order. The stored data is organized by worker.

[0315] In addition, the server is equipped with an emotion engine that can recognize the emotions of workers through voice analysis, facial expression recognition, and behavior analysis.

[0316] Emotion engine: The server uses the emotion engine to analyze input data from the camera and microphone and determine the emotion of the worker from their facial expressions and tone of voice. For example, if a worker is feeling stressed, the emotion engine will recognize that information.

[0317] Data analysis and proposal generation

[0318] The AI ​​module installed on the server periodically analyzes the data in the database, using clustering techniques and machine learning algorithms to learn the behavioral patterns and preferences of workers.

[0319] The results of data analysis are combined with the output of the emotion engine to identify the working environment conditions (for example, the optimal temperature and lighting brightness in the factory) that will make workers feel most efficient and comfortable. This information is then generated as recommendations.

[0320] Notification and Execution

[0321] The server then sends the generated suggestions to mobile devices, factory robots, and other terminals, including specific suggestions such as "change the cooling device's set temperature to 28°C" or "adjust the lighting brightness to 70%."

[0322] The worker checks the notification and can either manually follow the suggestions or let the system automatically adjust them, and the server automatically changes the settings of the cooling equipment and lighting according to the worker's selection.

[0323] Specific examples

[0324] For example, if the temperature sensor in a factory detects 32°C, the humidity sensor indicates 60%, and the emotion engine recognizes that a worker is experiencing high stress levels, the AI ​​module will make a comprehensive judgment and generate a recommendation to set the cooling equipment to 28°C, a comfortable temperature for the worker. Furthermore, data from the light and motion sensors will suggest that the optimal lighting level is 70% when it is confirmed that a worker is present in the work area. These recommendations are sent to the worker's device, and if the worker agrees, the server will automatically change the cooling equipment and lighting settings.

[0325] Prompt Sentence Examples

[0326] By inputting prompts like the following into the generative AI model, we can have it suggest appropriate countermeasures:

[0327] "Currently, the temperature inside the factory has reached 32 degrees, and analysis of the workers' facial expressions has detected high stress levels. How should the robot respond in this situation? Please propose a method for operating the cooling device as well."

[0328] This system is expected to improve the working environment within the factory facility and increase worker efficiency and safety.

[0329] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0330] Step 1:

[0331] Sensor data collection

[0332] The server periodically acquires data from temperature, humidity, illuminance, and motion sensors installed in the factory. It receives real-time data from the various sensors as input and stores this data in the server's database as output. Specifically, it receives data from each sensor via HTTP requests or MQTT protocols, and stores the data in the database in JSON format.

[0333] Step 2:

[0334] Collecting Emotional Data

[0335] The server collects facial and voice data from the workers from cameras and microphones installed in the factory and analyzes it using an emotion engine. It receives the video and audio data from the cameras and microphones as input, and stores the analyzed emotional data (e.g., stress level, happiness level, etc.) as output in a database. Specifically, it analyzes the video data frame by frame and performs waveform analysis on the audio data to recognize emotions.

[0336] Step 3:

[0337] Data storage and management

[0338] The server organizes and stores the environmental data and emotion data acquired in steps 1 and 2 in a database. It receives various data as input and generates a database organized for each user as output. Specifically, it stores the data in a database system (e.g., MySQL, PostgreSQL, etc.) in chronological order and creates indexes as needed to enable efficient data searches.

[0339] Step 4:

[0340] Data analysis

[0341] The server analyzes the stored environmental and emotional data to learn the behavioral patterns and preferences of workers. It receives the stored data as input and generates analytical results as output. Specifically, it analyzes the data using machine learning algorithms (e.g., clustering, regression analysis) to identify behavioral patterns and emotional changes.

[0342] Step 5:

[0343] Proposal Generation

[0344] The server generates optimal working environment conditions based on the analysis results. It receives the data analysis results as input and generates specific suggestions (e.g., temperature settings for cooling devices, brightness adjustments for lighting) as output. The specific operation is to automatically generate suggestions using an AI module and format the content as a notification message.

[0345] Step 6:

[0346] Proposal Notification

[0347] The server notifies the worker's device of the generated proposal. It receives the generated proposal as input and sends a notification message to the worker as output. The specific operation is to notify the worker of the proposal content in real time using the notification function of the mobile device or factory robot.

[0348] Step 7:

[0349] Automatic Control

[0350] If the worker approves the proposal, the server automatically changes the settings of the cooling equipment and lighting in the factory. It receives the worker's approval as input and generates and sends equipment control instructions as output. Specifically, it uses the IoT platform to send control signals to the corresponding home appliances and implements the setting changes.

[0351] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0352] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0353] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0354] [Second embodiment]

[0355] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0356] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0357] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0358] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0359] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0360] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0361] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0362] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0363] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0364] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0365] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0366] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0367] The system for implementing the present invention is composed of multiple sensors installed in a living space, a server that manages the data, an AI module that performs analysis, and a terminal that notifies the user.

[0368] First, multiple temperature, humidity, illuminance, and motion sensors are placed in the living space. These sensors continuously collect various environmental data and send it to a server.

[0369] sensor

[0370] The temperature sensor measures the room temperature in real time and sends the value to the server.

[0371] The humidity sensor measures the humidity level in the room and sends the data to a server.

[0372] The illuminance sensor detects the on / off status of the lights and the brightness level, and sends that information to the server.

[0373] The motion sensor detects the user's movement and reports to the server whether the user is in the room.

[0374] server

[0375] The server receives the data sent from each sensor in real time and stores it in a database, which accumulates a wealth of data in chronological order.

[0376] The received data is periodically analyzed by an AI module, which is a process that learns user behavior patterns and preferences based on the collected data.

[0377] AI Module

[0378] The AI ​​module analyzes past data and uses clustering techniques to understand user behavior patterns, such as air conditioning usage frequency and lighting preferences, and then derives comfortable environmental settings based on that information.

[0379] Based on the analysis results, suggestions are generated for optimal air conditioning temperatures, lighting levels, and even ways to increase eco-consciousness.

[0380] Terminal

[0381] The server then sends the generated suggestions to the user's device, such as a smartphone or tablet, including recommended air conditioning temperature settings and lighting adjustment levels.

[0382] Users can receive notifications and manually change preferences based on the suggestions provided, but the system also has the ability to automatically change settings on home appliances.

[0383] Specific examples

[0384] For example, if the temperature sensor detects 30°C in the summer and the humidity sensor indicates 50%, the AI ​​module will analyze this and generate a suggestion to set the air conditioner to 24°C, which the user finds comfortable. Furthermore, based on data from the light and motion sensors, it will suggest that the optimal lighting level is 70% when the user is confirmed to be in the room. These suggestions are sent to the user's device, and if the user accepts, the server will automatically change the air conditioner and lighting settings.

[0385] In this way, the present invention utilizes the abundant data obtained from sensors to improve both user comfort and environmental awareness.

[0386] The processing flow will be explained below.

[0387] Step 1:

[0388] Sensors periodically measure temperature, humidity, light, and motion within the living space, and transmit the measurements to a server in real time.

[0389] Step 2:

[0390] The server receives data (temperature, humidity, illuminance, motion) sent from each sensor and stores it in a database in chronological order. The stored data is organized by user.

[0391] Step 3:

[0392] The AI ​​module installed on the server periodically analyzes the data in the database, using clustering techniques and machine learning algorithms to learn user behavior patterns and preferences.

[0393] Step 4:

[0394] Based on the analysis results, the server identifies the environmental conditions that the user finds most comfortable (for example, the optimal air conditioning temperature or lighting brightness). This information is generated as a suggestion.

[0395] Step 5:

[0396] The server then sends the generated suggestions to the user's device, including specific suggestions such as "change the air conditioner temperature setting to 24°C" or "adjust the lighting brightness to 70%."

[0397] Step 6:

[0398] The user checks the notification and can either manually follow the suggestions or choose to have the system automatically adjust the settings for air conditioning and lighting according to the user's selection.

[0399] Step 7:

[0400] The server then stores the results of the configuration changes in the database and uses them for the next analysis and proposal generation. This process is repeated, continuously supporting the user's comfort and eco-conscious lifestyle.

[0401] Example 1

[0402] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0403] In modern residential environments, achieving efficient energy use while increasing user comfort is an important challenge. However, existing systems have difficulty automatically optimizing environmental settings based on user behavior patterns and preferences. Furthermore, environmental settings are often proposed and implemented manually, requiring cumbersome operations for users, making it difficult to maintain a continuously comfortable environment.

[0404] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0405] In this invention, the server includes means for collecting data from a plurality of sensors installed in the living space, means for storing the data collected from the sensors in a database in chronological order, means for analyzing the stored data using clustering technology to learn the user's behavioral patterns and preferences, means for generating suggestions for comfortable environmental settings based on the analysis results, means for notifying the user's terminal of the suggestions and automatically controlling the home appliances based on the suggestions, and means for re-collecting environmental data based on the suggested settings and continuously storing it in the database. This enables automatic and continuous optimization of environmental settings based on the data collected from the sensors, allowing the user to maintain a comfortable living space without having to perform complicated operations.

[0406] A "sensor" is a device installed in a living space to detect temperature, humidity, light, and motion.

[0407] A "server" is a device that receives data collected from multiple sensors, stores it in a database, and then analyzes and generates proposals.

[0408] A "database" is a storage device for storing data collected from sensors in chronological order.

[0409] "Clustering technology" is a method of analyzing stored data and grouping similar data together, and is used to learn user behavior patterns and preferences.

[0410] A "terminal" is a device such as a smartphone or tablet that receives proposals from the server and notifies the user.

[0411] A "suggestion" is a specific instruction for creating a comfortable environment, generated based on the analysis results.

[0412] "Home appliances" are electrical appliances in living spaces, such as air conditioners and lighting, that are controlled based on proposals.

[0413] The system consists of multiple sensors installed in living spaces, a server that manages the data, an AI module that performs analysis, and a terminal that notifies users.

[0414] First, multiple temperature sensors, humidity sensors, illuminance sensors, and motion sensors are placed in the living space. These sensors continuously collect various environmental data and send that data to a server. For example, if the temperature sensor sends data such as "The current temperature is 30°C," the humidity sensor may send data such as "The current humidity is 50%."

[0415] The server then receives this data in real time and stores it in a database in chronological order. For example, at 2:30 PM on October 5, 2023, the following data will be recorded: temperature 30°C, humidity 50%, lighting 100 lux, and user presence. This makes it possible to refer to past environmental data.

[0416] The stored data is analyzed by an AI module that uses clustering technology. This AI module uses past data to learn user behavior patterns and preferences. For example, the analysis may yield results such as "The optimal air conditioner temperature setting in summer is 24°C."

[0417] Based on the analysis results, the AI ​​module will generate specific suggestions for comfortable environmental settings. For example, it may suggest, "Today's weather is hot, so I recommend setting the air conditioner temperature to 24°C." It may also recommend, "It would be good to set the lighting to 70% when you are in the room."

[0418] This suggestion is sent from the server to the user's device, which can be a smartphone or tablet, and the notification includes specific instructions such as, "The current temperature is 30°C and the humidity is 50%. Please change the air conditioner's temperature setting to 24°C."

[0419] If the user approves the proposal, the server automatically changes the settings of the home appliance. For example, if the user approves the proposal to set the air conditioner to 24°C, the server will change the air conditioner's setting to 24°C. This allows the user to maintain a comfortable living space without having to perform complicated operations.

[0420] Furthermore, environmental data after the settings are changed is collected again from the sensors and saved in the database. This allows the AI ​​module to continuously analyze the new data. For example, the data that indicates the user's comfort level after setting the air conditioner to 24°C and humidity to 45% is saved.

[0421] A concrete example of a prompt is:

[0422] "Please suggest the optimal air conditioner setting based on the temperature data from the past week."

[0423] This text is input into the generative AI model.

[0424] This series of steps enables the system to automatically and continuously optimize environmental settings based on data obtained from sensors, providing users with a comfortable and energy-efficient living environment.

[0425] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0426] Step 1:

[0427] The server collects data from multiple sensors installed in the living space. Specifically, it receives data sent from temperature sensors, humidity sensors, illuminance sensors, and motion sensors in real time. The input is data from each sensor, and the output is the received sensor information. For example, it receives data such as temperature 30°C, humidity 50%, illuminance 100 lux, and whether the user is present in the room.

[0428] Step 2:

[0429] The server stores the received sensor data in a database in chronological order. The input is the sensor data received in step 1, and the output is the log-format data stored in the database. Specifically, the following is recorded in the database: "October 5, 2023, 14:30, temperature 30°C, humidity 50%, illuminance 100 lux, user present."

[0430] Step 3:

[0431] The server provides the stored data to the AI ​​module, which analyzes past data using clustering technology to learn the user's behavioral patterns and preferences. The input is environmental data stored in the database, and the output is analyzed patterns and preference information. Specifically, it extracts information such as air conditioner usage frequency and lighting preferences.

[0432] Step 4:

[0433] The AI ​​module generates recommendations for optimal environmental settings based on the analysis results. The input is the behavioral patterns and preference information obtained in step 3, and the output is specific recommendations. For example, a recommendation might be generated such as, "Since the current temperature is 30°C, we suggest setting the air conditioner to 24°C."

[0434] Step 5:

[0435] The server notifies the user's device of the generated suggestion. The input is the suggestion from the AI ​​module, and the output is a notification message displayed on the user's device. Specifically, a notification is sent to the smartphone saying, "Please change the air conditioner's temperature setting to 24°C."

[0436] Step 6:

[0437] The user accepts or rejects the proposal. The input is the proposal displayed on the terminal, and the output is the result of the operation: accept or reject. For example, the user accepts the proposal "change the air conditioner temperature setting to 24°C."

[0438] Step 7:

[0439] The server controls the home appliances based on the user's approval. The input is the user's approval result, and the output is the state of the home appliances after the settings are changed. Specifically, the air conditioner's set temperature is changed to 24°C.

[0440] Step 8:

[0441] The server collects the environmental data again after the settings are changed and stores it in the database. The input is the new data from the sensor, and the output is the log-format data added to the database. Specifically, the air conditioner settings of 24°C and 45% humidity, as well as feedback from the user about their comfort level, are also stored.

[0442] By repeating this process, the system automatically and continuously optimizes environmental settings based on sensor data, providing users with a comfortable and energy-efficient living environment.

[0443] (Application example 1)

[0444] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0445] In modern living spaces, environmental conditions such as temperature, humidity, and illuminance have a significant impact on people's comfort. However, it is difficult to automatically provide optimal environmental settings based on the behavior and preferences of individual users or customers. In particular, in commercial spaces, efficient environmental control while maintaining customer comfort is required. The present invention aims to solve these problems and provide optimal environmental settings in living and commercial spaces.

[0446] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0447] In this invention, the server includes means for collecting data from multiple sensors installed in the living space, means for storing the data collected from the sensors in a database, means for analyzing the stored data and learning user behavior patterns and preferences, means for generating suggestions for optimal environmental settings based on the analysis results, means for notifying the user of the suggestions and controlling electronic devices based on the suggestions, and means for analyzing customer traffic patterns in commercial spaces using data collected from the sensors and suggesting comfort settings. This makes it possible to analyze environmental data in living spaces and commercial spaces in real time and automatically provide optimal environmental settings that increase the comfort of users and customers.

[0448] "Living space" refers to the space in which people live their daily lives, and generally refers to the indoor environment of homes, offices, etc.

[0449] A "sensor" is a device that detects environmental data or conditions, such as temperature, humidity, light, or motion, and transmits that information electronically.

[0450] "Database" means a system or platform for storing and managing collected environmental data.

[0451] "Analysis" is the processing of collected data to extract meaningful information and patterns.

[0452] "Behavioral patterns" refer to the regularity and characteristics of a series of actions or behaviors of users or customers.

[0453] "Preferences" refers to the individual tastes and tendencies of users or customers to find particular environmental conditions or settings comfortable.

[0454] "Suggestions" are information or instructions that recommend optimal environmental settings or actions based on the analysis results.

[0455] "Notification" is the act or means of conveying information such as suggestions or warnings to users.

[0456] "Electronic devices" refers to all electrically operated devices, such as home appliances, computing devices, and lighting equipment.

[0457] "Commercial space" refers to a space used for business purposes, such as a store or office where customers visit.

[0458] "Customer traffic flow" refers to the route or path that customers take within a commercial space.

[0459] "Comfort settings" refers to adjusting environmental conditions to make users or customers feel most comfortable.

[0460] This invention is a system consisting of multiple sensors installed in living and commercial spaces, a server that manages and analyzes the data, and a terminal that notifies the results of the analysis. Each element that makes up this system operates as follows.

[0461] 1. Sensor installation and data collection

[0462] Living and commercial spaces are equipped with multiple temperature, humidity, light, and motion sensors that continuously collect environmental data and transmit it to a server.

[0463] Temperature sensor: Measures the room temperature in real time and sends the value to the server.

[0464] Humidity sensor: Measures the humidity level in the room and sends the data to the server.

[0465] Illuminance sensor: Detects the on / off status of the light and the brightness level, and sends that information to the server.

[0466] Motion sensor: Detects the user's movement and reports to the server whether the user is in the room.

[0467] 2. Saving to the database

[0468] The server receives the data sent from each sensor in real time and stores it in a database, which accumulates a wealth of data in chronological order.

[0469] 3. Analysis and learning by AI module

[0470] The received data is periodically analyzed by an AI module, which is a process that learns the behavioral patterns and preferences of users and customers based on the collected data.

[0471] AI module: Analyzes past data and learns things like air conditioner usage frequency and lighting preferences. To do this, it uses clustering technology. Based on the analysis results, it derives the optimal air conditioner temperature and lighting level.

[0472] 4. Proposal generation and notification

[0473] Based on the analysis results, suggestions for optimal environmental settings are generated, including adjusting the air conditioner temperature setting and lighting levels. The generated suggestions are then sent to the user's device.

[0474] Notifications: Suggestions include recommended air conditioning temperature settings and lighting adjustment levels. Users can receive notifications and manually change their environmental settings according to the suggestions provided, but the system also has the ability to automatically change the settings of their home appliances.

[0475] 5. Examples of applications in commercial spaces

[0476] In commercial spaces in particular, data collected from sensors is used to analyze customer movement patterns and propose comfortable settings.

[0477] Example: If the temperature inside a store is 27°C, humidity is 60%, and the illuminance is 200 lumens, and a motion sensor detects the presence of a customer, a suggestion will be generated to set the air conditioner temperature to 24°C and adjust the lighting to 500 lumens.

[0478] 6. Hardware and Software Used

[0479] Hardware: Temperature, humidity, light, motion sensors (e.g. Bosch BME280 sensor, PIR motion sensor)

[0480] Software: Python, Flask or Django, TensorFlow or PyTorch, SQLite or PostgreSQL

[0481] Example prompt sentence:

[0482] The temperature in your store is 27°C and the humidity is 60%. The light sensor detects 200 lumens and the motion sensor detects the presence of a customer. To maintain a comfortable environment, set the air conditioning to 24°C and adjust the lighting to 500 lumens.

[0483] According to this embodiment, the present invention can analyze environmental data in real time in living spaces and commercial spaces, and automatically provide optimal environmental settings that increase the comfort of users and customers.

[0484] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0485] Step 1:

[0486] Multiple sensors installed in living and commercial spaces collect temperature, humidity, illuminance, and motion data in real time. The input data from the sensors is sent to a server. Specifically, the temperature sensor detects room temperature, the humidity sensor detects humidity level, the illuminance sensor detects lighting brightness level, and the motion sensor detects movement. The output at this stage is raw data sent from each sensor to the server.

[0487] Step 2:

[0488] The server receives the data sent from each sensor in real time and stores it in a database. The server receives input data from the sensors, converts it into an appropriate format, and records it in the database. This database stores a wealth of time-series data. For example, temperature, humidity, illuminance, and motion data are recorded in time series. The output of this step is the sensor information stored in the database.

[0489] Step 3:

[0490] The server periodically analyzes the historical data stored in the database using an AI module. This analysis learns the behavioral patterns and preferences of users and customers based on the collected data. The server provides the AI ​​module with historical sensor data as input, and the AI ​​module uses clustering techniques to identify significant patterns. For example, it learns preferences for temperature and light intensity at specific times of the day. The output of this step is a model of the behavioral patterns and preferences resulting from the analysis.

[0491] Step 4:

[0492] The server generates a proposal for optimal environmental settings based on the analysis results from the AI ​​module. Specifically, it calculates and generates the optimal air conditioner temperature and lighting level proposals based on the output of the AI ​​module. The server supplies the proposals as input to the generation AI model, which outputs the optimal environmental conditions. For example, when the temperature is 27°C and the humidity is 60%, the air conditioner should be set to 24°C and the lighting adjusted to 500 lumens. The output of this step is the recommended settings.

[0493] Step 5:

[0494] The server notifies the terminal of the generated proposal. The terminal receives the proposal and informs the user or store clerk. The notification may include recommended air conditioner temperature settings or lighting adjustment levels. The terminal may also directly control home appliances based on the notified information. The input to this step is the proposal data from the server, and the output is a notification display on the terminal or control operation of the home appliance.

[0495] Step 6:

[0496] The user or store clerk can manually change the environmental settings based on the notified suggestions, or if automatic control is enabled, the system will automatically change the settings of the home appliances. The user can adjust the temperature and lighting according to the received suggestions using a device such as a smartphone or tablet. Specifically, the user can confirm the suggestions and press the approval button, which causes the server to change the air conditioner and lighting settings. The output of this step is the adjusted home appliance settings.

[0497] Example prompt sentence:

[0498] The temperature in your store is 27°C and the humidity is 60%. The light sensor detects 200 lumens and the motion sensor detects the presence of a customer. To maintain a comfortable environment, set the air conditioning to 24°C and adjust the lighting to 500 lumens.

[0499] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0500] The system for implementing the present invention is composed of multiple sensors installed in the living space, a server that manages and analyzes the data, an emotion engine, and a terminal that notifies the user.

[0501] First, temperature, humidity, light, and motion sensors are placed in the living space to continuously collect various environmental data and send it to a server.

[0502] sensor

[0503] The temperature sensor measures the room temperature in real time and sends the value to the server.

[0504] The humidity sensor measures the humidity level in the room and sends the data to a server.

[0505] The illuminance sensor detects the on / off status of the lights and the brightness level, and sends that information to the server.

[0506] The motion sensor detects the user's movement and reports to the server whether the user is in the room.

[0507] server

[0508] The server receives data (temperature, humidity, illuminance, motion) sent from each sensor and stores it in a database in chronological order. The stored data is organized by user.

[0509] Furthermore, the server is equipped with an emotion engine that can recognize the user's emotions through voice analysis, facial expression recognition, and behavior analysis.

[0510] Emotion Engine

[0511] The server uses an emotion engine to analyze input data from the camera and microphone and determine the user's emotions from their facial expressions and tone of voice. For example, if the user is feeling stressed, the emotion engine will recognize that information.

[0512] The recognized emotion data is also stored in a database along with other sensor data.

[0513] Data analysis and proposal generation

[0514] The AI ​​module installed on the server periodically analyzes the data in the database, using clustering techniques and machine learning algorithms to learn user behavior patterns and preferences.

[0515] The data analysis results are combined with the output of the emotion engine to identify the environmental conditions that make the user feel most comfortable (for example, the optimal air conditioning temperature or lighting brightness). This information is then generated as a suggestion.

[0516] Notification and Execution

[0517] The server then sends the generated suggestions to the user's device, such as a smartphone or tablet. These suggestions include specific suggestions such as "change the air conditioner temperature setting to 24°C" or "adjust the lighting brightness to 70%."

[0518] The user checks the notification and can either manually follow the suggestions or choose to have the system automatically adjust the settings for air conditioning and lighting according to the user's selection.

[0519] Specific examples

[0520] For example, if the temperature sensor detects 30°C in the summer, the humidity sensor indicates 50%, and the emotion engine recognizes that the user is irritated, the AI ​​module will make a comprehensive judgment and generate a suggestion to set the air conditioner to 24°C, which the user finds comfortable. Furthermore, when data from the illuminance sensor and motion sensor confirms that the user is in the room, the AI ​​module will suggest that the optimal lighting level is 70%. These suggestions are notified to the user's device, and if the user agrees, the server will automatically change the air conditioner and lighting settings.

[0521] In this way, the present invention realizes a system that balances user comfort with improved environmental awareness based on the abundant data acquired from sensors and emotion engines.

[0522] The processing flow will be explained below.

[0523] Step 1:

[0524] Sensors periodically measure temperature, humidity, light, and motion within the living space, and transmit the measurements to a server in real time.

[0525] Step 2:

[0526] The server receives data (temperature, humidity, illuminance, motion) sent from each sensor and stores it in a database in chronological order. The stored data is organized by user.

[0527] Step 3:

[0528] The emotion engine installed on the server receives input data from the camera or microphone, analyzes the user's facial expressions and tone of voice from this data, and recognizes the user's emotions. For example, it determines whether the user is stressed or relaxed.

[0529] Step 4:

[0530] The server stores the recognized emotion data in a database along with other sensor data, which associates the emotion data with behavioral patterns and environmental data.

[0531] Step 5:

[0532] An AI module installed on the server periodically analyzes the sensor data and emotion data stored in the database, and uses clustering techniques and machine learning algorithms to learn the user's behavioral and emotional patterns.

[0533] Step 6:

[0534] Based on the analysis results, the server identifies the environmental conditions that the user finds most comfortable (for example, the optimal air conditioning temperature or lighting brightness), and generates optimal environmental setting suggestions, taking into account emotional data.

[0535] Step 7:

[0536] The server then sends the generated suggestions to the user's smartphone, tablet, or other device. The content of the notification includes specific suggestions such as "change the air conditioner temperature setting to 24°C" or "adjust the lighting brightness to 70%."

[0537] Step 8:

[0538] Users receive a notification and can manually configure their preferences based on the suggestions provided, or choose automatic configuration, and in some cases even fine-tune the settings.

[0539] Step 9:

[0540] The server automatically adjusts the temperature settings of the air conditioner and the brightness level of the lights if the user selects automatic settings, but also collects setting change information if the user manually selects settings.

[0541] Step 10:

[0542] The server then stores the results of the changes in the database and uses them for the next analysis and proposal generation. By repeating this process, the system continuously supports users' comfort and eco-conscious lifestyles.

[0543] Example 2

[0544] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0545] In modern society, it is extremely important for users to maintain a comfortable living environment. However, many current systems simply collect and display data from various sensors and are unable to precisely adjust the environment to reflect the user's emotions and behavioral patterns. This poses a problem, requiring users to frequently manually configure the environment. Furthermore, these manual operations can cause stress for users. Therefore, there is a need for a system that automatically provides optimal environmental conditions based on the user's emotions and behavioral patterns.

[0546] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0547] In this invention, the server includes: means for collecting data from multiple sensors installed in the living space; means for storing the data collected from the sensors in a database; emotion recognition means for recognizing a user's emotions based on the data stored in the database; means for analyzing the stored data and the emotion data acquired by the emotion recognition means and learning the user's behavioral patterns and preferences; means for generating a proposal for optimal environmental settings based on the analysis results and the data from the emotion recognition means; and means for notifying the user of the proposal and controlling environmental devices based on the proposal. This enables precise environmental adjustment that reflects the user's emotions and behavioral patterns, eliminating the need for the user to manually adjust settings. Furthermore, it is possible to improve the user's comfort and reduce stress.

[0548] A "sensor" is a device that measures environmental information in living spaces (temperature, humidity, illuminance, motion, etc.) and collects that data.

[0549] The "database" is a system for storing and managing collected sensor data and emotion data in chronological order.

[0550] The "emotion recognition means" is a function that analyzes data obtained from input devices such as cameras and microphones and determines the user's emotional state (for example, stress or a sense of relief).

[0551] A "behavioral pattern" is a set of tendencies and habits extracted based on a user's daily actions and activities.

[0552] "Analysis" is the process of learning user behavior patterns and preferences using data stored in the database and data obtained from the emotion recognition means.

[0553] "Suggestions" are specific instructions or recommendations based on the analysis results to help users create optimal environmental settings that will allow them to feel comfortable.

[0554] "Notification" refers to the act and means of informing the user of the generated proposal via an information display device or the like.

[0555] "Control" is the function of actually changing the settings of environmental equipment such as air conditioners and lighting based on suggestions and performing the operations.

[0556] The system for implementing this invention is composed of multiple sensors installed in the living space, a server that manages and analyzes the data, an emotion recognition engine, and a terminal that notifies the user. A detailed description of each component is provided below.

[0557] Data collection by sensors

[0558] Temperature sensors, humidity sensors, illuminance sensors, and motion sensors are placed in the living space. These sensors measure environmental information in real time and send the data to a server. For example, sensors connected to a Raspberry Pi or Arduino are used. The temperature sensor continuously monitors the temperature in the room, and the humidity sensor measures the humidity level. The illuminance sensor detects the brightness and on / off status of the lights, and the motion sensor detects the user's movement. All of this data is sent to the server.

[0559] Receiving and storing data by the server

[0560] The server is built using web frameworks such as Flask or Django and receives real-time data sent from each sensor. The received data is stored in chronological order in a database system such as PostgreSQL or MySQL. The data is then organized and categorized for each user within the database.

[0561] Emotion data analysis using an emotion recognition engine

[0562] The emotion recognition engine installed on the server analyzes input data from the camera and microphone to recognize the user's emotions. This engine uses libraries such as OpenCV and TensorFlow. For example, video data from the camera is analyzed using OpenCV to recognize facial expressions. Audio data from the microphone is analyzed using TensorFlow to determine the user's emotion from the tone of voice. The recognized emotion data is stored in a database, just like sensor data.

[0563] Data analysis and proposal generation

[0564] The AI ​​module installed on the server periodically analyzes the data in the database. This analysis uses clustering techniques (e.g., K-means clustering) and machine learning algorithms (e.g., random forests). The AI ​​module uses these techniques to learn the user's behavioral patterns and preferences. Based on the analysis results, it identifies the environmental conditions (e.g., air conditioner temperature setting and lighting brightness) that make the user feel most comfortable and generates specific recommendations.

[0565] Notification and Execution

[0566] The server sends the generated suggestions to devices such as smartphones and tablets. The notifications include specific suggestions such as "set the air conditioner to 24°C" and "adjust the lighting to 70%." The user can then review the notifications and choose to follow the suggestions or select automatic settings. If the user agrees, the server will automatically change the air conditioner and lighting settings.

[0567] Specific examples

[0568] For example, if the temperature sensor detects 30°C in the summer, the humidity sensor indicates 50%, and the emotion recognition engine recognizes that the user is irritated, the AI ​​module will make a comprehensive judgment and generate a suggestion to set the air conditioner to 24°C, a temperature that the user finds comfortable. It will also use data from the light and motion sensors to confirm that the user is in the room and suggest an optimal lighting level (70%). These suggestions are sent to the user's device, and if the user agrees, the server will automatically change the air conditioner and lighting settings.

[0569] Example prompts for generative AI models

[0570] "If the room temperature is 30°C and the humidity is 50% in summer, and the emotion recognition engine detects that the user is irritated, what is the optimal air conditioner setting that the system should suggest?"

[0571] In this way, the present invention utilizes the wealth of data acquired from sensors and emotion recognition engines to improve both user comfort and environmental awareness.

[0572] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0573] Step 1:

[0574] Collecting data from sensors

[0575] Input: Temperature sensor, humidity sensor, light sensor, motion sensor

[0576] Processing: Each sensor collects data every minute.

[0577] The temperature sensor measures the temperature in the room and sends the data to the server.

[0578] The humidity sensor measures the humidity level in the room and sends the data to a server.

[0579] The illuminance sensor measures the brightness and on / off status of the light and sends the data to the server.

[0580] The motion sensor detects the user's movements and sends the data to the server.

[0581] Output: Environmental data sent from each sensor to the server

[0582] Step 2:

[0583] Receiving and storing data by the server

[0584] Input: Real-time data sent from each sensor

[0585] Processing: The server receives the data and stores it in the database.

[0586] Use the Flask or Django web framework to receive data via HTTP requests.

[0587] The received data is stored in chronological order in a PostgreSQL or MySQL database.

[0588] Output: Sensor data stored in a database

[0589] Step 3:

[0590] Emotion data analysis using an emotion recognition engine

[0591] Input: Camera video data, microphone audio data

[0592] Processing: The server analyzes the data obtained from the camera and microphone.

[0593] The camera's video data is analyzed using OpenCV to recognize facial expressions.

[0594] The audio data from the microphone is analyzed using TensorFlow, and emotions are determined from the tone of the voice.

[0595] Output: Emotion data analyzed by the emotion recognition engine

[0596] Step 4:

[0597] Data analysis and proposal generation

[0598] Input: Stored sensor data and emotion data

[0599] Processing: An AI module on the server analyzes the data and generates suggestions.

[0600] Periodically analyze the data in the database.

[0601] Analyze the data using clustering techniques (e.g., K-means clustering) and machine learning algorithms (e.g., random forests).

[0602] Based on the analysis results, specific suggestions for optimal environmental settings (e.g., air conditioner temperature settings and lighting brightness) are generated.

[0603] Output: Generated configuration proposal

[0604] Step 5:

[0605] User notification and execution

[0606] Input: Generated configuration proposal

[0607] Processing: The server notifies the user's device of the proposal and controls the environmental devices.

[0608] The proposal will be sent via push notification to your smartphone or tablet.

[0609] The user checks the notified suggestions and selects manual or automatic settings.

[0610] If the user selects automatic settings, the server sends commands to change the air conditioning and lighting settings.

[0611] Output: Notification to user terminal and change of environmental device settings

[0612] In this way, the system can automatically implement environmental settings that improve user comfort through concrete processing at each step.

[0613] (Application example 2)

[0614] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0615] In order to optimize the working environment in factory facilities and improve worker efficiency, a system is needed that can effectively collect and analyze real-time environmental data and worker emotional data, and generate appropriate control instructions based on that data. Conventional methods do not adequately link environmental data and emotional data, making it difficult to immediately improve the working environment or work efficiency.

[0616] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0617] In this invention, the server includes means for collecting data from a plurality of sensors installed in the living space, means for storing the data collected from the sensors in a database, means for analyzing the stored data and learning the user's behavioral patterns and preferences, means for generating suggestions for optimal environmental settings based on the analysis results, means for notifying the user of the suggestions and controlling home appliances based on the suggestions, means for acquiring emotional data of workers in a factory facility, and means for analyzing the acquired emotional data and environmental data and generating control instructions for optimizing the work environment, thereby enabling immediate improvement of the work environment in the factory facility and improvement of the work efficiency of workers.

[0618] "Living space" refers to the space where residents live their daily lives, such as the indoor environment of a home or office.

[0619] A "sensor" is a general term for a device that detects physical environmental data and outputs that data as an electrical signal. Examples include temperature sensors, humidity sensors, illuminance sensors, and motion sensors.

[0620] A "database" is a collection of information in which collected data is systematically stored and managed so that it can be efficiently searched and analyzed as needed.

[0621] "Means for analyzing data" refers to the methods and devices used to convert collected data into useful information such as user behavior patterns and preferences.

[0622] "User" refers to the person who uses this system, and is the target for improving their comfort and convenience.

[0623] "Behavioral patterns" refer to the actions and patterns of behavior that users repeatedly perform in their daily lives.

[0624] "Environmental data" is physical data collected by sensors, including temperature, humidity, light, and motion.

[0625] "Emotional data" refers to data that indicates the emotional state of a worker or user, as determined from their facial expressions, tone of voice, and behavior.

[0626] "Means for generating suggestions" refers to a method or device that creates specific action suggestions or instructions for changing settings to be provided to users based on the analysis results.

[0627] "Means for notifying proposals" is a general term for methods and devices for notifying users of generated proposals, and includes notification functions on mobile devices, etc.

[0628] "Means for controlling home appliances" refers to methods and devices for automatically operating home appliances such as air conditioners and lights based on analysis results and suggestions.

[0629] "Factory facilities" refers to all environments within buildings and premises where manufacturing or production activities take place.

[0630] "Workers" refers to people engaged in manufacturing or production activities within a factory facility.

[0631] "Control instructions" are specific operational commands issued to optimize the working environment and equipment operation within a factory facility.

[0632] "Analyzing means" is a general term for processing methods and devices that process collected data and convert it into useful information.

[0633] "Optimizing the environment" means adjusting the environment to a state where users or workers can work comfortably and efficiently.

[0634] The system for implementing the present invention is composed of multiple sensors installed within a factory facility, a server that manages and analyzes the data, an emotion engine, and a terminal that notifies workers.

[0635] System Configuration

[0636] sensor

[0637] Temperature sensor: Measures the temperature in the factory in real time and sends the value to the server.

[0638] Humidity sensor: Measures the humidity level in the factory and sends the data to a server.

[0639] Illuminance sensor: Detects whether the light is on or off and the brightness level, and sends that information to the server.

[0640] Motion sensor: Detects worker movement and reports the presence of people in the work area to the server.

[0641] server

[0642] The server receives data (temperature, humidity, illuminance, motion) sent from each sensor and stores it in a database in chronological order. The stored data is organized by worker.

[0643] In addition, the server is equipped with an emotion engine that can recognize the emotions of workers through voice analysis, facial expression recognition, and behavior analysis.

[0644] Emotion engine: The server uses the emotion engine to analyze input data from the camera and microphone and determine the emotion of the worker from their facial expressions and tone of voice. For example, if a worker is feeling stressed, the emotion engine will recognize that information.

[0645] Data analysis and proposal generation

[0646] The AI ​​module installed on the server periodically analyzes the data in the database, using clustering techniques and machine learning algorithms to learn the behavioral patterns and preferences of workers.

[0647] The results of data analysis are combined with the output of the emotion engine to identify the working environment conditions (for example, the optimal temperature and lighting brightness in the factory) that will make workers feel most efficient and comfortable. This information is then generated as recommendations.

[0648] Notification and Execution

[0649] The server then sends the generated suggestions to mobile devices, factory robots, and other terminals, including specific suggestions such as "change the cooling device's set temperature to 28°C" or "adjust the lighting brightness to 70%."

[0650] The worker checks the notification and can either manually follow the suggestions or let the system automatically adjust them, and the server automatically changes the settings of the cooling equipment and lighting according to the worker's selection.

[0651] Specific examples

[0652] For example, if the temperature sensor in a factory detects 32°C, the humidity sensor indicates 60%, and the emotion engine recognizes that a worker is experiencing high stress levels, the AI ​​module will make a comprehensive judgment and generate a recommendation to set the cooling equipment to 28°C, a comfortable temperature for the worker. Furthermore, data from the light and motion sensors will suggest that the optimal lighting level is 70% when it is confirmed that a worker is present in the work area. These recommendations are sent to the worker's device, and if the worker agrees, the server will automatically change the cooling equipment and lighting settings.

[0653] Prompt Sentence Examples

[0654] By inputting prompts like the following into the generative AI model, we can have it suggest appropriate countermeasures:

[0655] "Currently, the temperature inside the factory has reached 32 degrees, and analysis of the workers' facial expressions has detected high stress levels. How should the robot respond in this situation? Please propose a method for operating the cooling device as well."

[0656] This system is expected to improve the working environment within the factory facility and increase worker efficiency and safety.

[0657] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0658] Step 1:

[0659] Sensor data collection

[0660] The server periodically acquires data from temperature, humidity, illuminance, and motion sensors installed in the factory. It receives real-time data from the various sensors as input and stores this data in the server's database as output. Specifically, it receives data from each sensor via HTTP requests or MQTT protocols, and stores the data in the database in JSON format.

[0661] Step 2:

[0662] Collecting Emotional Data

[0663] The server collects facial and voice data from the workers from cameras and microphones installed in the factory and analyzes it using an emotion engine. It receives the video and audio data from the cameras and microphones as input, and stores the analyzed emotional data (e.g., stress level, happiness level, etc.) as output in a database. Specifically, it analyzes the video data frame by frame and performs waveform analysis on the audio data to recognize emotions.

[0664] Step 3:

[0665] Data storage and management

[0666] The server organizes and stores the environmental data and emotion data acquired in steps 1 and 2 in a database. It receives various data as input and generates a database organized for each user as output. Specifically, it stores the data in a database system (e.g., MySQL, PostgreSQL, etc.) in chronological order and creates indexes as needed to enable efficient data searches.

[0667] Step 4:

[0668] Data analysis

[0669] The server analyzes the stored environmental and emotional data to learn the behavioral patterns and preferences of workers. It receives the stored data as input and generates analytical results as output. Specifically, it analyzes the data using machine learning algorithms (e.g., clustering, regression analysis) to identify behavioral patterns and emotional changes.

[0670] Step 5:

[0671] Proposal Generation

[0672] The server generates optimal working environment conditions based on the analysis results. It receives the data analysis results as input and generates specific suggestions (e.g., temperature settings for cooling devices, brightness adjustments for lighting) as output. The specific operation is to automatically generate suggestions using an AI module and format the content as a notification message.

[0673] Step 6:

[0674] Proposal Notification

[0675] The server notifies the worker's device of the generated proposal. It receives the generated proposal as input and sends a notification message to the worker as output. The specific operation is to notify the worker of the proposal content in real time using the notification function of the mobile device or factory robot.

[0676] Step 7:

[0677] Automatic Control

[0678] If the worker approves the proposal, the server automatically changes the settings of the cooling equipment and lighting in the factory. It receives the worker's approval as input and generates and sends equipment control instructions as output. Specifically, it uses the IoT platform to send control signals to the corresponding home appliances and implements the setting changes.

[0679] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0680] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0681] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0682] [Third embodiment]

[0683] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0684] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0685] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0686] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0687] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0688] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0689] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0690] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0691] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0692] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0693] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0694] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0695] The system for implementing the present invention is composed of multiple sensors installed in a living space, a server that manages the data, an AI module that performs analysis, and a terminal that notifies the user.

[0696] First, multiple temperature, humidity, illuminance, and motion sensors are placed in the living space. These sensors continuously collect various environmental data and send it to a server.

[0697] sensor

[0698] The temperature sensor measures the room temperature in real time and sends the value to the server.

[0699] The humidity sensor measures the humidity level in the room and sends the data to a server.

[0700] The illuminance sensor detects the on / off status of the lights and the brightness level, and sends that information to the server.

[0701] The motion sensor detects the user's movement and reports to the server whether the user is in the room.

[0702] server

[0703] The server receives the data sent from each sensor in real time and stores it in a database, which accumulates a wealth of data in chronological order.

[0704] The received data is periodically analyzed by an AI module, which is a process that learns user behavior patterns and preferences based on the collected data.

[0705] AI Module

[0706] The AI ​​module analyzes past data and uses clustering techniques to understand user behavior patterns, such as air conditioning usage frequency and lighting preferences, and then derives comfortable environmental settings based on that information.

[0707] Based on the analysis results, suggestions are generated for optimal air conditioning temperatures, lighting levels, and even ways to increase eco-consciousness.

[0708] Terminal

[0709] The server then sends the generated suggestions to the user's device, such as a smartphone or tablet, including recommended air conditioning temperature settings and lighting adjustment levels.

[0710] Users can receive notifications and manually change preferences based on the suggestions provided, but the system also has the ability to automatically change settings on home appliances.

[0711] Specific examples

[0712] For example, if the temperature sensor detects 30°C in the summer and the humidity sensor indicates 50%, the AI ​​module will analyze this and generate a suggestion to set the air conditioner to 24°C, which the user finds comfortable. Furthermore, based on data from the light and motion sensors, it will suggest that the optimal lighting level is 70% when the user is confirmed to be in the room. These suggestions are sent to the user's device, and if the user accepts, the server will automatically change the air conditioner and lighting settings.

[0713] In this way, the present invention utilizes the abundant data obtained from sensors to improve both user comfort and environmental awareness.

[0714] The processing flow will be explained below.

[0715] Step 1:

[0716] Sensors periodically measure temperature, humidity, light, and motion within the living space, and transmit the measurements to a server in real time.

[0717] Step 2:

[0718] The server receives data (temperature, humidity, illuminance, motion) sent from each sensor and stores it in a database in chronological order. The stored data is organized by user.

[0719] Step 3:

[0720] The AI ​​module installed on the server periodically analyzes the data in the database, using clustering techniques and machine learning algorithms to learn user behavior patterns and preferences.

[0721] Step 4:

[0722] Based on the analysis results, the server identifies the environmental conditions that the user finds most comfortable (for example, the optimal air conditioning temperature or lighting brightness). This information is generated as a suggestion.

[0723] Step 5:

[0724] The server then sends the generated suggestions to the user's device, including specific suggestions such as "change the air conditioner temperature setting to 24°C" or "adjust the lighting brightness to 70%."

[0725] Step 6:

[0726] The user checks the notification and can either manually follow the suggestions or choose to have the system automatically adjust the settings for air conditioning and lighting according to the user's selection.

[0727] Step 7:

[0728] The server then stores the results of the configuration changes in the database and uses them for the next analysis and proposal generation. This process is repeated, continuously supporting the user's comfort and eco-conscious lifestyle.

[0729] Example 1

[0730] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0731] In modern residential environments, achieving efficient energy use while increasing user comfort is an important challenge. However, existing systems have difficulty automatically optimizing environmental settings based on user behavior patterns and preferences. Furthermore, environmental settings are often proposed and implemented manually, requiring cumbersome operations for users, making it difficult to maintain a continuously comfortable environment.

[0732] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0733] In this invention, the server includes means for collecting data from a plurality of sensors installed in the living space, means for storing the data collected from the sensors in a database in chronological order, means for analyzing the stored data using clustering technology to learn the user's behavioral patterns and preferences, means for generating suggestions for comfortable environmental settings based on the analysis results, means for notifying the user's terminal of the suggestions and automatically controlling the home appliances based on the suggestions, and means for re-collecting environmental data based on the suggested settings and continuously storing it in the database. This enables automatic and continuous optimization of environmental settings based on the data collected from the sensors, allowing the user to maintain a comfortable living space without having to perform complicated operations.

[0734] A "sensor" is a device installed in a living space to detect temperature, humidity, light, and motion.

[0735] A "server" is a device that receives data collected from multiple sensors, stores it in a database, and then analyzes and generates proposals.

[0736] A "database" is a storage device for storing data collected from sensors in chronological order.

[0737] "Clustering technology" is a method of analyzing stored data and grouping similar data together, and is used to learn user behavior patterns and preferences.

[0738] A "terminal" is a device such as a smartphone or tablet that receives proposals from the server and notifies the user.

[0739] A "suggestion" is a specific instruction for creating a comfortable environment, generated based on the analysis results.

[0740] "Home appliances" are electrical appliances in living spaces, such as air conditioners and lighting, that are controlled based on proposals.

[0741] The system consists of multiple sensors installed in living spaces, a server that manages the data, an AI module that performs analysis, and a terminal that notifies users.

[0742] First, multiple temperature sensors, humidity sensors, illuminance sensors, and motion sensors are placed in the living space. These sensors continuously collect various environmental data and send that data to a server. For example, if the temperature sensor sends data such as "The current temperature is 30°C," the humidity sensor may send data such as "The current humidity is 50%."

[0743] The server then receives this data in real time and stores it in a database in chronological order. For example, at 2:30 PM on October 5, 2023, the following data will be recorded: temperature 30°C, humidity 50%, lighting 100 lux, and user presence. This makes it possible to refer to past environmental data.

[0744] The stored data is analyzed by an AI module that uses clustering technology. This AI module uses past data to learn user behavior patterns and preferences. For example, the analysis may yield results such as "The optimal air conditioner temperature setting in summer is 24°C."

[0745] Based on the analysis results, the AI ​​module will generate specific suggestions for comfortable environmental settings. For example, it may suggest, "Today's weather is hot, so I recommend setting the air conditioner temperature to 24°C." It may also recommend, "It would be good to set the lighting to 70% when you are in the room."

[0746] This suggestion is sent from the server to the user's device, which can be a smartphone or tablet, and the notification includes specific instructions such as, "The current temperature is 30°C and the humidity is 50%. Please change the air conditioner's temperature setting to 24°C."

[0747] If the user approves the proposal, the server automatically changes the settings of the home appliance. For example, if the user approves the proposal to set the air conditioner to 24°C, the server will change the air conditioner's setting to 24°C. This allows the user to maintain a comfortable living space without having to perform complicated operations.

[0748] Furthermore, environmental data after the settings are changed is collected again from the sensors and saved in the database. This allows the AI ​​module to continuously analyze the new data. For example, the data that indicates the user's comfort level after setting the air conditioner to 24°C and humidity to 45% is saved.

[0749] A concrete example of a prompt is:

[0750] "Please suggest the optimal air conditioner setting based on the temperature data from the past week."

[0751] This text is input into the generative AI model.

[0752] This series of steps enables the system to automatically and continuously optimize environmental settings based on data obtained from sensors, providing users with a comfortable and energy-efficient living environment.

[0753] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0754] Step 1:

[0755] The server collects data from multiple sensors installed in the living space. Specifically, it receives data sent from temperature sensors, humidity sensors, illuminance sensors, and motion sensors in real time. The input is data from each sensor, and the output is the received sensor information. For example, it receives data such as temperature 30°C, humidity 50%, illuminance 100 lux, and whether the user is present in the room.

[0756] Step 2:

[0757] The server stores the received sensor data in a database in chronological order. The input is the sensor data received in step 1, and the output is the log-format data stored in the database. Specifically, the following is recorded in the database: "October 5, 2023, 14:30, temperature 30°C, humidity 50%, illuminance 100 lux, user present."

[0758] Step 3:

[0759] The server provides the stored data to the AI ​​module, which analyzes past data using clustering technology to learn the user's behavioral patterns and preferences. The input is environmental data stored in the database, and the output is analyzed patterns and preference information. Specifically, it extracts information such as air conditioner usage frequency and lighting preferences.

[0760] Step 4:

[0761] The AI ​​module generates recommendations for optimal environmental settings based on the analysis results. The input is the behavioral patterns and preference information obtained in step 3, and the output is specific recommendations. For example, a recommendation might be generated such as, "Since the current temperature is 30°C, we suggest setting the air conditioner to 24°C."

[0762] Step 5:

[0763] The server notifies the user's device of the generated suggestion. The input is the suggestion from the AI ​​module, and the output is a notification message displayed on the user's device. Specifically, a notification is sent to the smartphone saying, "Please change the air conditioner's temperature setting to 24°C."

[0764] Step 6:

[0765] The user accepts or rejects the proposal. The input is the proposal displayed on the terminal, and the output is the result of the operation: accept or reject. For example, the user accepts the proposal "change the air conditioner temperature setting to 24°C."

[0766] Step 7:

[0767] The server controls the home appliances based on the user's approval. The input is the user's approval result, and the output is the state of the home appliances after the settings are changed. Specifically, the air conditioner's set temperature is changed to 24°C.

[0768] Step 8:

[0769] The server collects the environmental data again after the settings are changed and stores it in the database. The input is the new data from the sensor, and the output is the log-format data added to the database. Specifically, the air conditioner settings of 24°C and 45% humidity, as well as feedback from the user about their comfort level, are also stored.

[0770] By repeating this process, the system automatically and continuously optimizes environmental settings based on sensor data, providing users with a comfortable and energy-efficient living environment.

[0771] (Application example 1)

[0772] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0773] In modern living spaces, environmental conditions such as temperature, humidity, and illuminance have a significant impact on people's comfort. However, it is difficult to automatically provide optimal environmental settings based on the behavior and preferences of individual users or customers. In particular, in commercial spaces, efficient environmental control while maintaining customer comfort is required. The present invention aims to solve these problems and provide optimal environmental settings in living and commercial spaces.

[0774] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0775] In this invention, the server includes means for collecting data from multiple sensors installed in the living space, means for storing the data collected from the sensors in a database, means for analyzing the stored data and learning user behavior patterns and preferences, means for generating suggestions for optimal environmental settings based on the analysis results, means for notifying the user of the suggestions and controlling electronic devices based on the suggestions, and means for analyzing customer traffic patterns in commercial spaces using data collected from the sensors and suggesting comfort settings. This makes it possible to analyze environmental data in living spaces and commercial spaces in real time and automatically provide optimal environmental settings that increase the comfort of users and customers.

[0776] "Living space" refers to the space in which people live their daily lives, and generally refers to the indoor environment of homes, offices, etc.

[0777] A "sensor" is a device that detects environmental data or conditions, such as temperature, humidity, light, or motion, and transmits that information electronically.

[0778] "Database" means a system or platform for storing and managing collected environmental data.

[0779] "Analysis" is the processing of collected data to extract meaningful information and patterns.

[0780] "Behavioral patterns" refer to the regularity and characteristics of a series of actions or behaviors of users or customers.

[0781] "Preferences" refers to the individual tastes and tendencies of users or customers to find particular environmental conditions or settings comfortable.

[0782] "Suggestions" are information or instructions that recommend optimal environmental settings or actions based on the analysis results.

[0783] "Notification" is the act or means of conveying information such as suggestions or warnings to users.

[0784] "Electronic devices" refers to all electrically operated devices, such as home appliances, computing devices, and lighting equipment.

[0785] "Commercial space" refers to a space used for business purposes, such as a store or office where customers visit.

[0786] "Customer traffic flow" refers to the route or path that customers take within a commercial space.

[0787] "Comfort settings" refers to adjusting environmental conditions to make users or customers feel most comfortable.

[0788] This invention is a system consisting of multiple sensors installed in living and commercial spaces, a server that manages and analyzes the data, and a terminal that notifies the results of the analysis. Each element that makes up this system operates as follows.

[0789] 1. Sensor installation and data collection

[0790] Living and commercial spaces are equipped with multiple temperature, humidity, light, and motion sensors that continuously collect environmental data and transmit it to a server.

[0791] Temperature sensor: Measures the room temperature in real time and sends the value to the server.

[0792] Humidity sensor: Measures the humidity level in the room and sends the data to the server.

[0793] Illuminance sensor: Detects the on / off status of the light and the brightness level, and sends that information to the server.

[0794] Motion sensor: Detects the user's movement and reports to the server whether the user is in the room.

[0795] 2. Saving to the database

[0796] The server receives the data sent from each sensor in real time and stores it in a database, which accumulates a wealth of data in chronological order.

[0797] 3. Analysis and learning by AI module

[0798] The received data is periodically analyzed by an AI module, which is a process that learns the behavioral patterns and preferences of users and customers based on the collected data.

[0799] AI module: Analyzes past data and learns things like air conditioner usage frequency and lighting preferences. To do this, it uses clustering technology. Based on the analysis results, it derives the optimal air conditioner temperature and lighting level.

[0800] 4. Proposal generation and notification

[0801] Based on the analysis results, suggestions for optimal environmental settings are generated, including adjusting the air conditioner temperature setting and lighting levels. The generated suggestions are then sent to the user's device.

[0802] Notifications: Suggestions include recommended air conditioning temperature settings and lighting adjustment levels. Users can receive notifications and manually change their environmental settings according to the suggestions provided, but the system also has the ability to automatically change the settings of their home appliances.

[0803] 5. Examples of applications in commercial spaces

[0804] In commercial spaces in particular, data collected from sensors is used to analyze customer movement patterns and propose comfortable settings.

[0805] Example: If the temperature inside a store is 27°C, humidity is 60%, and the illuminance is 200 lumens, and a motion sensor detects the presence of a customer, a suggestion will be generated to set the air conditioner temperature to 24°C and adjust the lighting to 500 lumens.

[0806] 6. Hardware and Software Used

[0807] Hardware: Temperature, humidity, light, motion sensors (e.g. Bosch BME280 sensor, PIR motion sensor)

[0808] Software: Python, Flask or Django, TensorFlow or PyTorch, SQLite or PostgreSQL

[0809] Example prompt sentence:

[0810] The temperature in your store is 27°C and the humidity is 60%. The light sensor detects 200 lumens and the motion sensor detects the presence of a customer. To maintain a comfortable environment, set the air conditioning to 24°C and adjust the lighting to 500 lumens.

[0811] According to this embodiment, the present invention can analyze environmental data in real time in living spaces and commercial spaces, and automatically provide optimal environmental settings that increase the comfort of users and customers.

[0812] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0813] Step 1:

[0814] Multiple sensors installed in living and commercial spaces collect temperature, humidity, illuminance, and motion data in real time. The input data from the sensors is sent to a server. Specifically, the temperature sensor detects room temperature, the humidity sensor detects humidity level, the illuminance sensor detects lighting brightness level, and the motion sensor detects movement. The output at this stage is raw data sent from each sensor to the server.

[0815] Step 2:

[0816] The server receives the data sent from each sensor in real time and stores it in a database. The server receives input data from the sensors, converts it into an appropriate format, and records it in the database. This database stores a wealth of time-series data. For example, temperature, humidity, illuminance, and motion data are recorded in time series. The output of this step is the sensor information stored in the database.

[0817] Step 3:

[0818] The server periodically analyzes the historical data stored in the database using an AI module. This analysis learns the behavioral patterns and preferences of users and customers based on the collected data. The server provides the AI ​​module with historical sensor data as input, and the AI ​​module uses clustering techniques to identify significant patterns. For example, it learns preferences for temperature and light intensity at specific times of the day. The output of this step is a model of the behavioral patterns and preferences resulting from the analysis.

[0819] Step 4:

[0820] The server generates a proposal for optimal environmental settings based on the analysis results from the AI ​​module. Specifically, it calculates and generates the optimal air conditioner temperature and lighting level proposals based on the output of the AI ​​module. The server supplies the proposals as input to the generation AI model, which outputs the optimal environmental conditions. For example, when the temperature is 27°C and the humidity is 60%, the air conditioner should be set to 24°C and the lighting adjusted to 500 lumens. The output of this step is the recommended settings.

[0821] Step 5:

[0822] The server notifies the terminal of the generated proposal. The terminal receives the proposal and informs the user or store clerk. The notification may include recommended air conditioner temperature settings or lighting adjustment levels. The terminal may also directly control home appliances based on the notified information. The input to this step is the proposal data from the server, and the output is a notification display on the terminal or control operation of the home appliance.

[0823] Step 6:

[0824] The user or store clerk can manually change the environmental settings based on the notified suggestions, or if automatic control is enabled, the system will automatically change the settings of the home appliances. The user can adjust the temperature and lighting according to the received suggestions using a device such as a smartphone or tablet. Specifically, the user can confirm the suggestions and press the approval button, which causes the server to change the air conditioner and lighting settings. The output of this step is the adjusted home appliance settings.

[0825] Example prompt sentence:

[0826] The temperature in your store is 27°C and the humidity is 60%. The light sensor detects 200 lumens and the motion sensor detects the presence of a customer. To maintain a comfortable environment, set the air conditioning to 24°C and adjust the lighting to 500 lumens.

[0827] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0828] The system for implementing the present invention is composed of multiple sensors installed in the living space, a server that manages and analyzes the data, an emotion engine, and a terminal that notifies the user.

[0829] First, temperature, humidity, light, and motion sensors are placed in the living space to continuously collect various environmental data and send it to a server.

[0830] sensor

[0831] The temperature sensor measures the room temperature in real time and sends the value to the server.

[0832] The humidity sensor measures the humidity level in the room and sends the data to a server.

[0833] The illuminance sensor detects the on / off status of the lights and the brightness level, and sends that information to the server.

[0834] The motion sensor detects the user's movement and reports to the server whether the user is in the room.

[0835] server

[0836] The server receives data (temperature, humidity, illuminance, motion) sent from each sensor and stores it in a database in chronological order. The stored data is organized by user.

[0837] Furthermore, the server is equipped with an emotion engine that can recognize the user's emotions through voice analysis, facial expression recognition, and behavior analysis.

[0838] Emotion Engine

[0839] The server uses an emotion engine to analyze input data from the camera and microphone and determine the user's emotions from their facial expressions and tone of voice. For example, if the user is feeling stressed, the emotion engine will recognize that information.

[0840] The recognized emotion data is also stored in a database along with other sensor data.

[0841] Data analysis and proposal generation

[0842] The AI ​​module installed on the server periodically analyzes the data in the database, using clustering techniques and machine learning algorithms to learn user behavior patterns and preferences.

[0843] The data analysis results are combined with the output of the emotion engine to identify the environmental conditions that make the user feel most comfortable (for example, the optimal air conditioning temperature or lighting brightness). This information is then generated as a suggestion.

[0844] Notification and Execution

[0845] The server then sends the generated suggestions to the user's device, such as a smartphone or tablet. These suggestions include specific suggestions such as "change the air conditioner temperature setting to 24°C" or "adjust the lighting brightness to 70%."

[0846] The user checks the notification and can either manually follow the suggestions or choose to have the system automatically adjust the settings for air conditioning and lighting according to the user's selection.

[0847] Specific examples

[0848] For example, if the temperature sensor detects 30°C in the summer, the humidity sensor indicates 50%, and the emotion engine recognizes that the user is irritated, the AI ​​module will make a comprehensive judgment and generate a suggestion to set the air conditioner to 24°C, which the user finds comfortable. Furthermore, when data from the illuminance sensor and motion sensor confirms that the user is in the room, the AI ​​module will suggest that the optimal lighting level is 70%. These suggestions are notified to the user's device, and if the user agrees, the server will automatically change the air conditioner and lighting settings.

[0849] In this way, the present invention realizes a system that balances user comfort with improved environmental awareness based on the abundant data acquired from sensors and emotion engines.

[0850] The processing flow will be explained below.

[0851] Step 1:

[0852] Sensors periodically measure temperature, humidity, light, and motion within the living space, and transmit the measurements to a server in real time.

[0853] Step 2:

[0854] The server receives data (temperature, humidity, illuminance, motion) sent from each sensor and stores it in a database in chronological order. The stored data is organized by user.

[0855] Step 3:

[0856] The emotion engine installed on the server receives input data from the camera or microphone, analyzes the user's facial expressions and tone of voice from this data, and recognizes the user's emotions. For example, it determines whether the user is stressed or relaxed.

[0857] Step 4:

[0858] The server stores the recognized emotion data in a database along with other sensor data, which associates the emotion data with behavioral patterns and environmental data.

[0859] Step 5:

[0860] An AI module installed on the server periodically analyzes the sensor data and emotion data stored in the database, and uses clustering techniques and machine learning algorithms to learn the user's behavioral and emotional patterns.

[0861] Step 6:

[0862] Based on the analysis results, the server identifies the environmental conditions that the user finds most comfortable (for example, the optimal air conditioning temperature or lighting brightness), and generates optimal environmental setting suggestions, taking into account emotional data.

[0863] Step 7:

[0864] The server then sends the generated suggestions to the user's smartphone, tablet, or other device. The content of the notification includes specific suggestions such as "change the air conditioner temperature setting to 24°C" or "adjust the lighting brightness to 70%."

[0865] Step 8:

[0866] Users receive a notification and can manually configure their preferences based on the suggestions provided, or choose automatic configuration, and in some cases even fine-tune the settings.

[0867] Step 9:

[0868] The server automatically adjusts the temperature settings of the air conditioner and the brightness level of the lights if the user selects automatic settings, but also collects setting change information if the user manually selects settings.

[0869] Step 10:

[0870] The server then stores the results of the changes in the database and uses them for the next analysis and proposal generation. By repeating this process, the system continuously supports users' comfort and eco-conscious lifestyles.

[0871] Example 2

[0872] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0873] In modern society, it is extremely important for users to maintain a comfortable living environment. However, many current systems simply collect and display data from various sensors and are unable to precisely adjust the environment to reflect the user's emotions and behavioral patterns. This poses a problem, requiring users to frequently manually configure the environment. Furthermore, these manual operations can cause stress for users. Therefore, there is a need for a system that automatically provides optimal environmental conditions based on the user's emotions and behavioral patterns.

[0874] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0875] In this invention, the server includes: means for collecting data from multiple sensors installed in the living space; means for storing the data collected from the sensors in a database; emotion recognition means for recognizing a user's emotions based on the data stored in the database; means for analyzing the stored data and the emotion data acquired by the emotion recognition means and learning the user's behavioral patterns and preferences; means for generating a proposal for optimal environmental settings based on the analysis results and the data from the emotion recognition means; and means for notifying the user of the proposal and controlling environmental devices based on the proposal. This enables precise environmental adjustment that reflects the user's emotions and behavioral patterns, eliminating the need for the user to manually adjust settings. Furthermore, it is possible to improve the user's comfort and reduce stress.

[0876] A "sensor" is a device that measures environmental information in living spaces (temperature, humidity, illuminance, motion, etc.) and collects that data.

[0877] The "database" is a system for storing and managing collected sensor data and emotion data in chronological order.

[0878] The "emotion recognition means" is a function that analyzes data obtained from input devices such as cameras and microphones and determines the user's emotional state (for example, stress or a sense of relief).

[0879] A "behavioral pattern" is a set of tendencies and habits extracted based on a user's daily actions and activities.

[0880] "Analysis" is the process of learning user behavior patterns and preferences using data stored in the database and data obtained from the emotion recognition means.

[0881] "Suggestions" are specific instructions or recommendations based on the analysis results to help users create optimal environmental settings that will allow them to feel comfortable.

[0882] "Notification" refers to the act and means of informing the user of the generated proposal via an information display device or the like.

[0883] "Control" is the function of actually changing the settings of environmental equipment such as air conditioners and lighting based on suggestions and performing the operations.

[0884] The system for implementing this invention is composed of multiple sensors installed in the living space, a server that manages and analyzes the data, an emotion recognition engine, and a terminal that notifies the user. A detailed description of each component is provided below.

[0885] Data collection by sensors

[0886] Temperature sensors, humidity sensors, illuminance sensors, and motion sensors are placed in the living space. These sensors measure environmental information in real time and send the data to a server. For example, sensors connected to a Raspberry Pi or Arduino are used. The temperature sensor continuously monitors the temperature in the room, and the humidity sensor measures the humidity level. The illuminance sensor detects the brightness and on / off status of the lights, and the motion sensor detects the user's movement. All of this data is sent to the server.

[0887] Receiving and storing data by the server

[0888] The server is built using web frameworks such as Flask or Django and receives real-time data sent from each sensor. The received data is stored in chronological order in a database system such as PostgreSQL or MySQL. The data is then organized and categorized for each user within the database.

[0889] Emotion data analysis using an emotion recognition engine

[0890] The emotion recognition engine installed on the server analyzes input data from the camera and microphone to recognize the user's emotions. This engine uses libraries such as OpenCV and TensorFlow. For example, video data from the camera is analyzed using OpenCV to recognize facial expressions. Audio data from the microphone is analyzed using TensorFlow to determine the user's emotion from the tone of voice. The recognized emotion data is stored in a database, just like sensor data.

[0891] Data analysis and proposal generation

[0892] The AI ​​module installed on the server periodically analyzes the data in the database. This analysis uses clustering techniques (e.g., K-means clustering) and machine learning algorithms (e.g., random forests). The AI ​​module uses these techniques to learn the user's behavioral patterns and preferences. Based on the analysis results, it identifies the environmental conditions (e.g., air conditioner temperature setting and lighting brightness) that make the user feel most comfortable and generates specific recommendations.

[0893] Notification and Execution

[0894] The server sends the generated suggestions to devices such as smartphones and tablets. The notifications include specific suggestions such as "set the air conditioner to 24°C" and "adjust the lighting to 70%." The user can then review the notifications and choose to follow the suggestions or select automatic settings. If the user agrees, the server will automatically change the air conditioner and lighting settings.

[0895] Specific examples

[0896] For example, if the temperature sensor detects 30°C in the summer, the humidity sensor indicates 50%, and the emotion recognition engine recognizes that the user is irritated, the AI ​​module will make a comprehensive judgment and generate a suggestion to set the air conditioner to 24°C, a temperature that the user finds comfortable. It will also use data from the light and motion sensors to confirm that the user is in the room and suggest an optimal lighting level (70%). These suggestions are sent to the user's device, and if the user agrees, the server will automatically change the air conditioner and lighting settings.

[0897] Example prompts for generative AI models

[0898] "If the room temperature is 30°C and the humidity is 50% in summer, and the emotion recognition engine detects that the user is irritated, what is the optimal air conditioner setting that the system should suggest?"

[0899] In this way, the present invention utilizes the wealth of data acquired from sensors and emotion recognition engines to improve both user comfort and environmental awareness.

[0900] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0901] Step 1:

[0902] Collecting data from sensors

[0903] Input: Temperature sensor, humidity sensor, light sensor, motion sensor

[0904] Processing: Each sensor collects data every minute.

[0905] The temperature sensor measures the temperature in the room and sends the data to the server.

[0906] The humidity sensor measures the humidity level in the room and sends the data to a server.

[0907] The illuminance sensor measures the brightness and on / off status of the light and sends the data to the server.

[0908] The motion sensor detects the user's movements and sends the data to the server.

[0909] Output: Environmental data sent from each sensor to the server

[0910] Step 2:

[0911] Receiving and storing data by the server

[0912] Input: Real-time data sent from each sensor

[0913] Processing: The server receives the data and stores it in the database.

[0914] Use the Flask or Django web framework to receive data via HTTP requests.

[0915] The received data is stored in chronological order in a PostgreSQL or MySQL database.

[0916] Output: Sensor data stored in a database

[0917] Step 3:

[0918] Emotion data analysis using an emotion recognition engine

[0919] Input: Camera video data, microphone audio data

[0920] Processing: The server analyzes the data obtained from the camera and microphone.

[0921] The camera's video data is analyzed using OpenCV to recognize facial expressions.

[0922] The audio data from the microphone is analyzed using TensorFlow, and emotions are determined from the tone of the voice.

[0923] Output: Emotion data analyzed by the emotion recognition engine

[0924] Step 4:

[0925] Data analysis and proposal generation

[0926] Input: Stored sensor data and emotion data

[0927] Processing: An AI module on the server analyzes the data and generates suggestions.

[0928] Periodically analyze the data in the database.

[0929] Analyze the data using clustering techniques (e.g., K-means clustering) and machine learning algorithms (e.g., random forests).

[0930] Based on the analysis results, specific suggestions for optimal environmental settings (e.g., air conditioner temperature settings and lighting brightness) are generated.

[0931] Output: Generated configuration proposal

[0932] Step 5:

[0933] User notification and execution

[0934] Input: Generated configuration proposal

[0935] Processing: The server notifies the user's device of the proposal and controls the environmental devices.

[0936] The proposal will be sent via push notification to your smartphone or tablet.

[0937] The user checks the notified suggestions and selects manual or automatic settings.

[0938] If the user selects automatic settings, the server sends commands to change the air conditioning and lighting settings.

[0939] Output: Notification to user terminal and change of environmental device settings

[0940] In this way, the system can automatically implement environmental settings that improve user comfort through concrete processing at each step.

[0941] (Application example 2)

[0942] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0943] In order to optimize the working environment in factory facilities and improve worker efficiency, a system is needed that can effectively collect and analyze real-time environmental data and worker emotional data, and generate appropriate control instructions based on that data. Conventional methods do not adequately link environmental data and emotional data, making it difficult to immediately improve the working environment or work efficiency.

[0944] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0945] In this invention, the server includes means for collecting data from a plurality of sensors installed in the living space, means for storing the data collected from the sensors in a database, means for analyzing the stored data and learning the user's behavioral patterns and preferences, means for generating suggestions for optimal environmental settings based on the analysis results, means for notifying the user of the suggestions and controlling home appliances based on the suggestions, means for acquiring emotional data of workers in a factory facility, and means for analyzing the acquired emotional data and environmental data and generating control instructions for optimizing the work environment, thereby enabling immediate improvement of the work environment in the factory facility and improvement of the work efficiency of workers.

[0946] "Living space" refers to the space where residents live their daily lives, such as the indoor environment of a home or office.

[0947] A "sensor" is a general term for a device that detects physical environmental data and outputs that data as an electrical signal. Examples include temperature sensors, humidity sensors, illuminance sensors, and motion sensors.

[0948] A "database" is a collection of information in which collected data is systematically stored and managed so that it can be efficiently searched and analyzed as needed.

[0949] "Means for analyzing data" refers to the methods and devices used to convert collected data into useful information such as user behavior patterns and preferences.

[0950] "User" refers to the person who uses this system, and is the target for improving their comfort and convenience.

[0951] "Behavioral patterns" refer to the actions and patterns of behavior that users repeatedly perform in their daily lives.

[0952] "Environmental data" is physical data collected by sensors, including temperature, humidity, light, and motion.

[0953] "Emotional data" refers to data that indicates the emotional state of a worker or user, as determined from their facial expressions, tone of voice, and behavior.

[0954] "Means for generating suggestions" refers to a method or device that creates specific action suggestions or instructions for changing settings to be provided to users based on the analysis results.

[0955] "Means for notifying proposals" is a general term for methods and devices for notifying users of generated proposals, and includes notification functions on mobile devices, etc.

[0956] "Means for controlling home appliances" refers to methods and devices for automatically operating home appliances such as air conditioners and lights based on analysis results and suggestions.

[0957] "Factory facilities" refers to all environments within buildings and premises where manufacturing or production activities take place.

[0958] "Workers" refers to people engaged in manufacturing or production activities within a factory facility.

[0959] "Control instructions" are specific operational commands issued to optimize the working environment and equipment operation within a factory facility.

[0960] "Analyzing means" is a general term for processing methods and devices that process collected data and convert it into useful information.

[0961] "Optimizing the environment" means adjusting the environment to a state where users or workers can work comfortably and efficiently.

[0962] The system for implementing the present invention is composed of multiple sensors installed within a factory facility, a server that manages and analyzes the data, an emotion engine, and a terminal that notifies workers.

[0963] System Configuration

[0964] sensor

[0965] Temperature sensor: Measures the temperature in the factory in real time and sends the value to the server.

[0966] Humidity sensor: Measures the humidity level in the factory and sends the data to a server.

[0967] Illuminance sensor: Detects whether the light is on or off and the brightness level, and sends that information to the server.

[0968] Motion sensor: Detects worker movement and reports the presence of people in the work area to the server.

[0969] server

[0970] The server receives data (temperature, humidity, illuminance, motion) sent from each sensor and stores it in a database in chronological order. The stored data is organized by worker.

[0971] In addition, the server is equipped with an emotion engine that can recognize the emotions of workers through voice analysis, facial expression recognition, and behavior analysis.

[0972] Emotion engine: The server uses the emotion engine to analyze input data from the camera and microphone and determine the emotion of the worker from their facial expressions and tone of voice. For example, if a worker is feeling stressed, the emotion engine will recognize that information.

[0973] Data analysis and proposal generation

[0974] The AI ​​module installed on the server periodically analyzes the data in the database, using clustering techniques and machine learning algorithms to learn the behavioral patterns and preferences of workers.

[0975] The results of data analysis are combined with the output of the emotion engine to identify the working environment conditions (for example, the optimal temperature and lighting brightness in the factory) that will make workers feel most efficient and comfortable. This information is then generated as recommendations.

[0976] Notification and Execution

[0977] The server then sends the generated suggestions to mobile devices, factory robots, and other terminals, including specific suggestions such as "change the cooling device's set temperature to 28°C" or "adjust the lighting brightness to 70%."

[0978] The worker checks the notification and can either manually follow the suggestions or let the system automatically adjust them, and the server automatically changes the settings of the cooling equipment and lighting according to the worker's selection.

[0979] Specific examples

[0980] For example, if the temperature sensor in a factory detects 32°C, the humidity sensor indicates 60%, and the emotion engine recognizes that a worker is experiencing high stress levels, the AI ​​module will make a comprehensive judgment and generate a recommendation to set the cooling equipment to 28°C, a comfortable temperature for the worker. Furthermore, data from the light and motion sensors will suggest that the optimal lighting level is 70% when it is confirmed that a worker is present in the work area. These recommendations are sent to the worker's device, and if the worker agrees, the server will automatically change the cooling equipment and lighting settings.

[0981] Prompt Sentence Examples

[0982] By inputting prompts like the following into the generative AI model, we can have it suggest appropriate countermeasures:

[0983] "Currently, the temperature inside the factory has reached 32 degrees, and analysis of the workers' facial expressions has detected high stress levels. How should the robot respond in this situation? Please propose a method for operating the cooling device as well."

[0984] This system is expected to improve the working environment within the factory facility and increase worker efficiency and safety.

[0985] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0986] Step 1:

[0987] Sensor data collection

[0988] The server periodically acquires data from temperature, humidity, illuminance, and motion sensors installed in the factory. It receives real-time data from the various sensors as input and stores this data in the server's database as output. Specifically, it receives data from each sensor via HTTP requests or MQTT protocols, and stores the data in the database in JSON format.

[0989] Step 2:

[0990] Collecting Emotional Data

[0991] The server collects facial and voice data from the workers from cameras and microphones installed in the factory and analyzes it using an emotion engine. It receives the video and audio data from the cameras and microphones as input, and stores the analyzed emotional data (e.g., stress level, happiness level, etc.) as output in a database. Specifically, it analyzes the video data frame by frame and performs waveform analysis on the audio data to recognize emotions.

[0992] Step 3:

[0993] Data storage and management

[0994] The server organizes and stores the environmental data and emotion data acquired in steps 1 and 2 in a database. It receives various data as input and generates a database organized for each user as output. Specifically, it stores the data in a database system (e.g., MySQL, PostgreSQL, etc.) in chronological order and creates indexes as needed to enable efficient data searches.

[0995] Step 4:

[0996] Data analysis

[0997] The server analyzes the stored environmental and emotional data to learn the behavioral patterns and preferences of workers. It receives the stored data as input and generates analytical results as output. Specifically, it analyzes the data using machine learning algorithms (e.g., clustering, regression analysis) to identify behavioral patterns and emotional changes.

[0998] Step 5:

[0999] Proposal Generation

[1000] The server generates optimal working environment conditions based on the analysis results. It receives the data analysis results as input and generates specific suggestions (e.g., temperature settings for cooling devices, brightness adjustments for lighting) as output. The specific operation is to automatically generate suggestions using an AI module and format the content as a notification message.

[1001] Step 6:

[1002] Proposal Notification

[1003] The server notifies the worker's device of the generated proposal. It receives the generated proposal as input and sends a notification message to the worker as output. The specific operation is to notify the worker of the proposal content in real time using the notification function of the mobile device or factory robot.

[1004] Step 7:

[1005] Automatic Control

[1006] If the worker approves the proposal, the server automatically changes the settings of the cooling equipment and lighting in the factory. It receives the worker's approval as input and generates and sends equipment control instructions as output. Specifically, it uses the IoT platform to send control signals to the corresponding home appliances and implements the setting changes.

[1007] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1008] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1009] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1010] [Fourth embodiment]

[1011] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1012] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1013] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1014] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1015] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1016] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1017] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1018] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1019] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1020] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1021] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1022] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1023] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1024] The system for implementing the present invention is composed of multiple sensors installed in a living space, a server that manages the data, an AI module that performs analysis, and a terminal that notifies the user.

[1025] First, multiple temperature, humidity, illuminance, and motion sensors are placed in the living space. These sensors continuously collect various environmental data and send it to a server.

[1026] sensor

[1027] The temperature sensor measures the room temperature in real time and sends the value to the server.

[1028] The humidity sensor measures the humidity level in the room and sends the data to a server.

[1029] The illuminance sensor detects the on / off status of the lights and the brightness level, and sends that information to the server.

[1030] The motion sensor detects the user's movement and reports to the server whether the user is in the room.

[1031] server

[1032] The server receives the data sent from each sensor in real time and stores it in a database, which accumulates a wealth of data in chronological order.

[1033] The received data is periodically analyzed by an AI module, which is a process that learns user behavior patterns and preferences based on the collected data.

[1034] AI Module

[1035] The AI ​​module analyzes past data and uses clustering techniques to understand user behavior patterns, such as air conditioning usage frequency and lighting preferences, and then derives comfortable environmental settings based on that information.

[1036] Based on the analysis results, suggestions are generated for optimal air conditioning temperatures, lighting levels, and even ways to increase eco-consciousness.

[1037] Terminal

[1038] The server then sends the generated suggestions to the user's device, such as a smartphone or tablet, including recommended air conditioning temperature settings and lighting adjustment levels.

[1039] Users can receive notifications and manually change preferences based on the suggestions provided, but the system also has the ability to automatically change settings on home appliances.

[1040] Specific examples

[1041] For example, if the temperature sensor detects 30°C in the summer and the humidity sensor indicates 50%, the AI ​​module will analyze this and generate a suggestion to set the air conditioner to 24°C, which the user finds comfortable. Furthermore, based on data from the light and motion sensors, it will suggest that the optimal lighting level is 70% when the user is confirmed to be in the room. These suggestions are sent to the user's device, and if the user accepts, the server will automatically change the air conditioner and lighting settings.

[1042] In this way, the present invention utilizes the abundant data obtained from sensors to improve both user comfort and environmental awareness.

[1043] The processing flow will be explained below.

[1044] Step 1:

[1045] Sensors periodically measure temperature, humidity, light, and motion within the living space, and transmit the measurements to a server in real time.

[1046] Step 2:

[1047] The server receives data (temperature, humidity, illuminance, motion) sent from each sensor and stores it in a database in chronological order. The stored data is organized by user.

[1048] Step 3:

[1049] The AI ​​module installed on the server periodically analyzes the data in the database, using clustering techniques and machine learning algorithms to learn user behavior patterns and preferences.

[1050] Step 4:

[1051] Based on the analysis results, the server identifies the environmental conditions that the user finds most comfortable (for example, the optimal air conditioning temperature or lighting brightness). This information is generated as a suggestion.

[1052] Step 5:

[1053] The server then sends the generated suggestions to the user's device, including specific suggestions such as "change the air conditioner temperature setting to 24°C" or "adjust the lighting brightness to 70%."

[1054] Step 6:

[1055] The user checks the notification and can either manually follow the suggestions or choose to have the system automatically adjust the settings for air conditioning and lighting according to the user's selection.

[1056] Step 7:

[1057] The server then stores the results of the configuration changes in the database and uses them for the next analysis and proposal generation. This process is repeated, continuously supporting the user's comfort and eco-conscious lifestyle.

[1058] Example 1

[1059] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1060] In modern residential environments, achieving efficient energy use while increasing user comfort is an important challenge. However, existing systems have difficulty automatically optimizing environmental settings based on user behavior patterns and preferences. Furthermore, environmental settings are often proposed and implemented manually, requiring cumbersome operations for users, making it difficult to maintain a continuously comfortable environment.

[1061] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1062] In this invention, the server includes means for collecting data from a plurality of sensors installed in the living space, means for storing the data collected from the sensors in a database in chronological order, means for analyzing the stored data using clustering technology to learn the user's behavioral patterns and preferences, means for generating suggestions for comfortable environmental settings based on the analysis results, means for notifying the user's terminal of the suggestions and automatically controlling the home appliances based on the suggestions, and means for re-collecting environmental data based on the suggested settings and continuously storing it in the database. This enables automatic and continuous optimization of environmental settings based on the data collected from the sensors, allowing the user to maintain a comfortable living space without having to perform complicated operations.

[1063] A "sensor" is a device installed in a living space to detect temperature, humidity, light, and motion.

[1064] A "server" is a device that receives data collected from multiple sensors, stores it in a database, and then analyzes and generates proposals.

[1065] A "database" is a storage device for storing data collected from sensors in chronological order.

[1066] "Clustering technology" is a method of analyzing stored data and grouping similar data together, and is used to learn user behavior patterns and preferences.

[1067] A "terminal" is a device such as a smartphone or tablet that receives proposals from the server and notifies the user.

[1068] A "suggestion" is a specific instruction for creating a comfortable environment, generated based on the analysis results.

[1069] "Home appliances" are electrical appliances in living spaces, such as air conditioners and lighting, that are controlled based on proposals.

[1070] The system consists of multiple sensors installed in living spaces, a server that manages the data, an AI module that performs analysis, and a terminal that notifies users.

[1071] First, multiple temperature sensors, humidity sensors, illuminance sensors, and motion sensors are placed in the living space. These sensors continuously collect various environmental data and send that data to a server. For example, if the temperature sensor sends data such as "The current temperature is 30°C," the humidity sensor may send data such as "The current humidity is 50%."

[1072] The server then receives this data in real time and stores it in a database in chronological order. For example, at 2:30 PM on October 5, 2023, the following data will be recorded: temperature 30°C, humidity 50%, lighting 100 lux, and user presence. This makes it possible to refer to past environmental data.

[1073] The stored data is analyzed by an AI module that uses clustering technology. This AI module uses past data to learn user behavior patterns and preferences. For example, the analysis may yield results such as "The optimal air conditioner temperature setting in summer is 24°C."

[1074] Based on the analysis results, the AI ​​module will generate specific suggestions for comfortable environmental settings. For example, it may suggest, "Today's weather is hot, so I recommend setting the air conditioner temperature to 24°C." It may also recommend, "It would be good to set the lighting to 70% when you are in the room."

[1075] This suggestion is sent from the server to the user's device, which can be a smartphone or tablet, and the notification includes specific instructions such as, "The current temperature is 30°C and the humidity is 50%. Please change the air conditioner's temperature setting to 24°C."

[1076] If the user approves the proposal, the server automatically changes the settings of the home appliance. For example, if the user approves the proposal to set the air conditioner to 24°C, the server will change the air conditioner's setting to 24°C. This allows the user to maintain a comfortable living space without having to perform complicated operations.

[1077] Furthermore, environmental data after the settings are changed is collected again from the sensors and saved in the database. This allows the AI ​​module to continuously analyze the new data. For example, the data that indicates the user's comfort level after setting the air conditioner to 24°C and humidity to 45% is saved.

[1078] A concrete example of a prompt is:

[1079] "Please suggest the optimal air conditioner setting based on the temperature data from the past week."

[1080] This text is input into the generative AI model.

[1081] This series of steps enables the system to automatically and continuously optimize environmental settings based on data obtained from sensors, providing users with a comfortable and energy-efficient living environment.

[1082] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1083] Step 1:

[1084] The server collects data from multiple sensors installed in the living space. Specifically, it receives data sent from temperature sensors, humidity sensors, illuminance sensors, and motion sensors in real time. The input is data from each sensor, and the output is the received sensor information. For example, it receives data such as temperature 30°C, humidity 50%, illuminance 100 lux, and whether the user is present in the room.

[1085] Step 2:

[1086] The server stores the received sensor data in a database in chronological order. The input is the sensor data received in step 1, and the output is the log-format data stored in the database. Specifically, the following is recorded in the database: "October 5, 2023, 14:30, temperature 30°C, humidity 50%, illuminance 100 lux, user present."

[1087] Step 3:

[1088] The server provides the stored data to the AI ​​module, which analyzes past data using clustering technology to learn the user's behavioral patterns and preferences. The input is environmental data stored in the database, and the output is analyzed patterns and preference information. Specifically, it extracts information such as air conditioner usage frequency and lighting preferences.

[1089] Step 4:

[1090] The AI ​​module generates recommendations for optimal environmental settings based on the analysis results. The input is the behavioral patterns and preference information obtained in step 3, and the output is specific recommendations. For example, a recommendation might be generated such as, "Since the current temperature is 30°C, we suggest setting the air conditioner to 24°C."

[1091] Step 5:

[1092] The server notifies the user's device of the generated suggestion. The input is the suggestion from the AI ​​module, and the output is a notification message displayed on the user's device. Specifically, a notification is sent to the smartphone saying, "Please change the air conditioner's temperature setting to 24°C."

[1093] Step 6:

[1094] The user accepts or rejects the proposal. The input is the proposal displayed on the terminal, and the output is the result of the operation: accept or reject. For example, the user accepts the proposal "change the air conditioner temperature setting to 24°C."

[1095] Step 7:

[1096] The server controls the home appliances based on the user's approval. The input is the user's approval result, and the output is the state of the home appliances after the settings are changed. Specifically, the air conditioner's set temperature is changed to 24°C.

[1097] Step 8:

[1098] The server collects the environmental data again after the settings are changed and stores it in the database. The input is the new data from the sensor, and the output is the log-format data added to the database. Specifically, the air conditioner settings of 24°C and 45% humidity, as well as feedback from the user about their comfort level, are also stored.

[1099] By repeating this process, the system automatically and continuously optimizes environmental settings based on sensor data, providing users with a comfortable and energy-efficient living environment.

[1100] (Application example 1)

[1101] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1102] In modern living spaces, environmental conditions such as temperature, humidity, and illuminance have a significant impact on people's comfort. However, it is difficult to automatically provide optimal environmental settings based on the behavior and preferences of individual users or customers. In particular, in commercial spaces, efficient environmental control while maintaining customer comfort is required. The present invention aims to solve these problems and provide optimal environmental settings in living and commercial spaces.

[1103] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1104] In this invention, the server includes means for collecting data from multiple sensors installed in the living space, means for storing the data collected from the sensors in a database, means for analyzing the stored data and learning user behavior patterns and preferences, means for generating suggestions for optimal environmental settings based on the analysis results, means for notifying the user of the suggestions and controlling electronic devices based on the suggestions, and means for analyzing customer traffic patterns in commercial spaces using data collected from the sensors and suggesting comfort settings. This makes it possible to analyze environmental data in living spaces and commercial spaces in real time and automatically provide optimal environmental settings that increase the comfort of users and customers.

[1105] "Living space" refers to the space in which people live their daily lives, and generally refers to the indoor environment of homes, offices, etc.

[1106] A "sensor" is a device that detects environmental data or conditions, such as temperature, humidity, light, or motion, and transmits that information electronically.

[1107] "Database" means a system or platform for storing and managing collected environmental data.

[1108] "Analysis" is the processing of collected data to extract meaningful information and patterns.

[1109] "Behavioral patterns" refer to the regularity and characteristics of a series of actions or behaviors of users or customers.

[1110] "Preferences" refers to the individual tastes and tendencies of users or customers to find particular environmental conditions or settings comfortable.

[1111] "Suggestions" are information or instructions that recommend optimal environmental settings or actions based on the analysis results.

[1112] "Notification" is the act or means of conveying information such as suggestions or warnings to users.

[1113] "Electronic devices" refers to all electrically operated devices, such as home appliances, computing devices, and lighting equipment.

[1114] "Commercial space" refers to a space used for business purposes, such as a store or office where customers visit.

[1115] "Customer traffic flow" refers to the route or path that customers take within a commercial space.

[1116] "Comfort settings" refers to adjusting environmental conditions to make users or customers feel most comfortable.

[1117] This invention is a system consisting of multiple sensors installed in living and commercial spaces, a server that manages and analyzes the data, and a terminal that notifies the results of the analysis. Each element that makes up this system operates as follows.

[1118] 1. Sensor installation and data collection

[1119] Living and commercial spaces are equipped with multiple temperature, humidity, light, and motion sensors that continuously collect environmental data and transmit it to a server.

[1120] Temperature sensor: Measures the room temperature in real time and sends the value to the server.

[1121] Humidity sensor: Measures the humidity level in the room and sends the data to the server.

[1122] Illuminance sensor: Detects the on / off status of the light and the brightness level, and sends that information to the server.

[1123] Motion sensor: Detects the user's movement and reports to the server whether the user is in the room.

[1124] 2. Saving to the database

[1125] The server receives the data sent from each sensor in real time and stores it in a database, which accumulates a wealth of data in chronological order.

[1126] 3. Analysis and learning by AI module

[1127] The received data is periodically analyzed by an AI module, which is a process that learns the behavioral patterns and preferences of users and customers based on the collected data.

[1128] AI module: Analyzes past data and learns things like air conditioner usage frequency and lighting preferences. To do this, it uses clustering technology. Based on the analysis results, it derives the optimal air conditioner temperature and lighting level.

[1129] 4. Proposal generation and notification

[1130] Based on the analysis results, suggestions for optimal environmental settings are generated, including adjusting the air conditioner temperature setting and lighting levels. The generated suggestions are then sent to the user's device.

[1131] Notifications: Suggestions include recommended air conditioning temperature settings and lighting adjustment levels. Users can receive notifications and manually change their environmental settings according to the suggestions provided, but the system also has the ability to automatically change the settings of their home appliances.

[1132] 5. Examples of applications in commercial spaces

[1133] In commercial spaces in particular, data collected from sensors is used to analyze customer movement patterns and propose comfortable settings.

[1134] Example: If the temperature inside a store is 27°C, humidity is 60%, and the illuminance is 200 lumens, and a motion sensor detects the presence of a customer, a suggestion will be generated to set the air conditioner temperature to 24°C and adjust the lighting to 500 lumens.

[1135] 6. Hardware and Software Used

[1136] Hardware: Temperature, humidity, light, motion sensors (e.g. Bosch BME280 sensor, PIR motion sensor)

[1137] Software: Python, Flask or Django, TensorFlow or PyTorch, SQLite or PostgreSQL

[1138] Example prompt sentence:

[1139] The temperature in your store is 27°C and the humidity is 60%. The light sensor detects 200 lumens and the motion sensor detects the presence of a customer. To maintain a comfortable environment, set the air conditioning to 24°C and adjust the lighting to 500 lumens.

[1140] According to this embodiment, the present invention can analyze environmental data in real time in living spaces and commercial spaces, and automatically provide optimal environmental settings that increase the comfort of users and customers.

[1141] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1142] Step 1:

[1143] Multiple sensors installed in living and commercial spaces collect temperature, humidity, illuminance, and motion data in real time. The input data from the sensors is sent to a server. Specifically, the temperature sensor detects room temperature, the humidity sensor detects humidity level, the illuminance sensor detects lighting brightness level, and the motion sensor detects movement. The output at this stage is raw data sent from each sensor to the server.

[1144] Step 2:

[1145] The server receives the data sent from each sensor in real time and stores it in a database. The server receives input data from the sensors, converts it into an appropriate format, and records it in the database. This database stores a wealth of time-series data. For example, temperature, humidity, illuminance, and motion data are recorded in time series. The output of this step is the sensor information stored in the database.

[1146] Step 3:

[1147] The server periodically analyzes the historical data stored in the database using an AI module. This analysis learns the behavioral patterns and preferences of users and customers based on the collected data. The server provides the AI ​​module with historical sensor data as input, and the AI ​​module uses clustering techniques to identify significant patterns. For example, it learns preferences for temperature and light intensity at specific times of the day. The output of this step is a model of the behavioral patterns and preferences resulting from the analysis.

[1148] Step 4:

[1149] The server generates a proposal for optimal environmental settings based on the analysis results from the AI ​​module. Specifically, it calculates and generates the optimal air conditioner temperature and lighting level proposals based on the output of the AI ​​module. The server supplies the proposals as input to the generation AI model, which outputs the optimal environmental conditions. For example, when the temperature is 27°C and the humidity is 60%, the air conditioner should be set to 24°C and the lighting adjusted to 500 lumens. The output of this step is the recommended settings.

[1150] Step 5:

[1151] The server notifies the terminal of the generated proposal. The terminal receives the proposal and informs the user or store clerk. The notification may include recommended air conditioner temperature settings or lighting adjustment levels. The terminal may also directly control home appliances based on the notified information. The input to this step is the proposal data from the server, and the output is a notification display on the terminal or control operation of the home appliance.

[1152] Step 6:

[1153] The user or store clerk can manually change the environmental settings based on the notified suggestions, or if automatic control is enabled, the system will automatically change the settings of the home appliances. The user can adjust the temperature and lighting according to the received suggestions using a device such as a smartphone or tablet. Specifically, the user can confirm the suggestions and press the approval button, which causes the server to change the air conditioner and lighting settings. The output of this step is the adjusted home appliance settings.

[1154] Example prompt sentence:

[1155] The temperature in your store is 27°C and the humidity is 60%. The light sensor detects 200 lumens and the motion sensor detects the presence of a customer. To maintain a comfortable environment, set the air conditioning to 24°C and adjust the lighting to 500 lumens.

[1156] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1157] The system for implementing the present invention is composed of multiple sensors installed in the living space, a server that manages and analyzes the data, an emotion engine, and a terminal that notifies the user.

[1158] First, temperature, humidity, light, and motion sensors are placed in the living space to continuously collect various environmental data and send it to a server.

[1159] sensor

[1160] The temperature sensor measures the room temperature in real time and sends the value to the server.

[1161] The humidity sensor measures the humidity level in the room and sends the data to a server.

[1162] The illuminance sensor detects the on / off status of the lights and the brightness level, and sends that information to the server.

[1163] The motion sensor detects the user's movement and reports to the server whether the user is in the room.

[1164] server

[1165] The server receives data (temperature, humidity, illuminance, motion) sent from each sensor and stores it in a database in chronological order. The stored data is organized by user.

[1166] Furthermore, the server is equipped with an emotion engine that can recognize the user's emotions through voice analysis, facial expression recognition, and behavior analysis.

[1167] Emotion Engine

[1168] The server uses an emotion engine to analyze input data from the camera and microphone and determine the user's emotions from their facial expressions and tone of voice. For example, if the user is feeling stressed, the emotion engine will recognize that information.

[1169] The recognized emotion data is also stored in a database along with other sensor data.

[1170] Data analysis and proposal generation

[1171] The AI ​​module installed on the server periodically analyzes the data in the database, using clustering techniques and machine learning algorithms to learn user behavior patterns and preferences.

[1172] The data analysis results are combined with the output of the emotion engine to identify the environmental conditions that make the user feel most comfortable (for example, the optimal air conditioning temperature or lighting brightness). This information is then generated as a suggestion.

[1173] Notification and Execution

[1174] The server then sends the generated suggestions to the user's device, such as a smartphone or tablet. These suggestions include specific suggestions such as "change the air conditioner temperature setting to 24°C" or "adjust the lighting brightness to 70%."

[1175] The user checks the notification and can either manually follow the suggestions or choose to have the system automatically adjust the settings for air conditioning and lighting according to the user's selection.

[1176] Specific examples

[1177] For example, if the temperature sensor detects 30°C in the summer, the humidity sensor indicates 50%, and the emotion engine recognizes that the user is irritated, the AI ​​module will make a comprehensive judgment and generate a suggestion to set the air conditioner to 24°C, which the user finds comfortable. Furthermore, when data from the illuminance sensor and motion sensor confirms that the user is in the room, the AI ​​module will suggest that the optimal lighting level is 70%. These suggestions are notified to the user's device, and if the user agrees, the server will automatically change the air conditioner and lighting settings.

[1178] In this way, the present invention realizes a system that balances user comfort with improved environmental awareness based on the abundant data acquired from sensors and emotion engines.

[1179] The processing flow will be explained below.

[1180] Step 1:

[1181] Sensors periodically measure temperature, humidity, light, and motion within the living space, and transmit the measurements to a server in real time.

[1182] Step 2:

[1183] The server receives data (temperature, humidity, illuminance, motion) sent from each sensor and stores it in a database in chronological order. The stored data is organized by user.

[1184] Step 3:

[1185] The emotion engine installed on the server receives input data from the camera or microphone, analyzes the user's facial expressions and tone of voice from this data, and recognizes the user's emotions. For example, it determines whether the user is stressed or relaxed.

[1186] Step 4:

[1187] The server stores the recognized emotion data in a database along with other sensor data, which associates the emotion data with behavioral patterns and environmental data.

[1188] Step 5:

[1189] An AI module installed on the server periodically analyzes the sensor data and emotion data stored in the database, and uses clustering techniques and machine learning algorithms to learn the user's behavioral and emotional patterns.

[1190] Step 6:

[1191] Based on the analysis results, the server identifies the environmental conditions that the user finds most comfortable (for example, the optimal air conditioning temperature or lighting brightness), and generates optimal environmental setting suggestions, taking into account emotional data.

[1192] Step 7:

[1193] The server then sends the generated suggestions to the user's smartphone, tablet, or other device. The content of the notification includes specific suggestions such as "change the air conditioner temperature setting to 24°C" or "adjust the lighting brightness to 70%."

[1194] Step 8:

[1195] Users receive a notification and can manually configure their preferences based on the suggestions provided, or choose automatic configuration, and in some cases even fine-tune the settings.

[1196] Step 9:

[1197] The server automatically adjusts the temperature settings of the air conditioner and the brightness level of the lights if the user selects automatic settings, but also collects setting change information if the user manually selects settings.

[1198] Step 10:

[1199] The server then stores the results of the changes in the database and uses them for the next analysis and proposal generation. By repeating this process, the system continuously supports users' comfort and eco-conscious lifestyles.

[1200] Example 2

[1201] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1202] In modern society, it is extremely important for users to maintain a comfortable living environment. However, many current systems simply collect and display data from various sensors and are unable to precisely adjust the environment to reflect the user's emotions and behavioral patterns. This poses a problem, requiring users to frequently manually configure the environment. Furthermore, these manual operations can cause stress for users. Therefore, there is a need for a system that automatically provides optimal environmental conditions based on the user's emotions and behavioral patterns.

[1203] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1204] In this invention, the server includes: means for collecting data from multiple sensors installed in the living space; means for storing the data collected from the sensors in a database; emotion recognition means for recognizing a user's emotions based on the data stored in the database; means for analyzing the stored data and the emotion data acquired by the emotion recognition means and learning the user's behavioral patterns and preferences; means for generating a proposal for optimal environmental settings based on the analysis results and the data from the emotion recognition means; and means for notifying the user of the proposal and controlling environmental devices based on the proposal. This enables precise environmental adjustment that reflects the user's emotions and behavioral patterns, eliminating the need for the user to manually adjust settings. Furthermore, it is possible to improve the user's comfort and reduce stress.

[1205] A "sensor" is a device that measures environmental information in living spaces (temperature, humidity, illuminance, motion, etc.) and collects that data.

[1206] The "database" is a system for storing and managing collected sensor data and emotion data in chronological order.

[1207] The "emotion recognition means" is a function that analyzes data obtained from input devices such as cameras and microphones and determines the user's emotional state (for example, stress or a sense of relief).

[1208] A "behavioral pattern" is a set of tendencies and habits extracted based on a user's daily actions and activities.

[1209] "Analysis" is the process of learning user behavior patterns and preferences using data stored in the database and data obtained from the emotion recognition means.

[1210] "Suggestions" are specific instructions or recommendations based on the analysis results to help users create optimal environmental settings that will allow them to feel comfortable.

[1211] "Notification" refers to the act and means of informing the user of the generated proposal via an information display device or the like.

[1212] "Control" is the function of actually changing the settings of environmental equipment such as air conditioners and lighting based on suggestions and performing the operations.

[1213] The system for implementing this invention is composed of multiple sensors installed in the living space, a server that manages and analyzes the data, an emotion recognition engine, and a terminal that notifies the user. A detailed description of each component is provided below.

[1214] Data collection by sensors

[1215] Temperature sensors, humidity sensors, illuminance sensors, and motion sensors are placed in the living space. These sensors measure environmental information in real time and send the data to a server. For example, sensors connected to a Raspberry Pi or Arduino are used. The temperature sensor continuously monitors the temperature in the room, and the humidity sensor measures the humidity level. The illuminance sensor detects the brightness and on / off status of the lights, and the motion sensor detects the user's movement. All of this data is sent to the server.

[1216] Receiving and storing data by the server

[1217] The server is built using web frameworks such as Flask or Django and receives real-time data sent from each sensor. The received data is stored in chronological order in a database system such as PostgreSQL or MySQL. The data is then organized and categorized for each user within the database.

[1218] Emotion data analysis using an emotion recognition engine

[1219] The emotion recognition engine installed on the server analyzes input data from the camera and microphone to recognize the user's emotions. This engine uses libraries such as OpenCV and TensorFlow. For example, video data from the camera is analyzed using OpenCV to recognize facial expressions. Audio data from the microphone is analyzed using TensorFlow to determine the user's emotion from the tone of voice. The recognized emotion data is stored in a database, just like sensor data.

[1220] Data analysis and proposal generation

[1221] The AI ​​module installed on the server periodically analyzes the data in the database. This analysis uses clustering techniques (e.g., K-means clustering) and machine learning algorithms (e.g., random forests). The AI ​​module uses these techniques to learn the user's behavioral patterns and preferences. Based on the analysis results, it identifies the environmental conditions (e.g., air conditioner temperature setting and lighting brightness) that make the user feel most comfortable and generates specific recommendations.

[1222] Notification and Execution

[1223] The server sends the generated suggestions to devices such as smartphones and tablets. The notifications include specific suggestions such as "set the air conditioner to 24°C" and "adjust the lighting to 70%." The user can then review the notifications and choose to follow the suggestions or select automatic settings. If the user agrees, the server will automatically change the air conditioner and lighting settings.

[1224] Specific examples

[1225] For example, if the temperature sensor detects 30°C in the summer, the humidity sensor indicates 50%, and the emotion recognition engine recognizes that the user is irritated, the AI ​​module will make a comprehensive judgment and generate a suggestion to set the air conditioner to 24°C, a temperature that the user finds comfortable. It will also use data from the light and motion sensors to confirm that the user is in the room and suggest an optimal lighting level (70%). These suggestions are sent to the user's device, and if the user agrees, the server will automatically change the air conditioner and lighting settings.

[1226] Example prompts for generative AI models

[1227] "If the room temperature is 30°C and the humidity is 50% in summer, and the emotion recognition engine detects that the user is irritated, what is the optimal air conditioner setting that the system should suggest?"

[1228] In this way, the present invention utilizes the wealth of data acquired from sensors and emotion recognition engines to improve both user comfort and environmental awareness.

[1229] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1230] Step 1:

[1231] Collecting data from sensors

[1232] Input: Temperature sensor, humidity sensor, light sensor, motion sensor

[1233] Processing: Each sensor collects data every minute.

[1234] The temperature sensor measures the temperature in the room and sends the data to the server.

[1235] The humidity sensor measures the humidity level in the room and sends the data to a server.

[1236] The illuminance sensor measures the brightness and on / off status of the light and sends the data to the server.

[1237] The motion sensor detects the user's movements and sends the data to the server.

[1238] Output: Environmental data sent from each sensor to the server

[1239] Step 2:

[1240] Receiving and storing data by the server

[1241] Input: Real-time data sent from each sensor

[1242] Processing: The server receives the data and stores it in the database.

[1243] Use the Flask or Django web framework to receive data via HTTP requests.

[1244] The received data is stored in chronological order in a PostgreSQL or MySQL database.

[1245] Output: Sensor data stored in a database

[1246] Step 3:

[1247] Emotion data analysis using an emotion recognition engine

[1248] Input: Camera video data, microphone audio data

[1249] Processing: The server analyzes the data obtained from the camera and microphone.

[1250] The camera's video data is analyzed using OpenCV to recognize facial expressions.

[1251] The audio data from the microphone is analyzed using TensorFlow, and emotions are determined from the tone of the voice.

[1252] Output: Emotion data analyzed by the emotion recognition engine

[1253] Step 4:

[1254] Data analysis and proposal generation

[1255] Input: Stored sensor data and emotion data

[1256] Processing: An AI module on the server analyzes the data and generates suggestions.

[1257] Periodically analyze the data in the database.

[1258] Analyze the data using clustering techniques (e.g., K-means clustering) and machine learning algorithms (e.g., random forests).

[1259] Based on the analysis results, specific suggestions for optimal environmental settings (e.g., air conditioner temperature settings and lighting brightness) are generated.

[1260] Output: Generated configuration proposal

[1261] Step 5:

[1262] User notification and execution

[1263] Input: Generated configuration proposal

[1264] Processing: The server notifies the user's device of the proposal and controls the environmental devices.

[1265] The proposal will be sent via push notification to your smartphone or tablet.

[1266] The user checks the notified suggestions and selects manual or automatic settings.

[1267] If the user selects automatic settings, the server sends commands to change the air conditioning and lighting settings.

[1268] Output: Notification to user terminal and change of environmental device settings

[1269] In this way, the system can automatically implement environmental settings that improve user comfort through concrete processing at each step.

[1270] (Application example 2)

[1271] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1272] In order to optimize the working environment in factory facilities and improve worker efficiency, a system is needed that can effectively collect and analyze real-time environmental data and worker emotional data, and generate appropriate control instructions based on that data. Conventional methods do not adequately link environmental data and emotional data, making it difficult to immediately improve the working environment or work efficiency.

[1273] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1274] In this invention, the server includes means for collecting data from a plurality of sensors installed in the living space, means for storing the data collected from the sensors in a database, means for analyzing the stored data and learning the user's behavioral patterns and preferences, means for generating suggestions for optimal environmental settings based on the analysis results, means for notifying the user of the suggestions and controlling home appliances based on the suggestions, means for acquiring emotional data of workers in a factory facility, and means for analyzing the acquired emotional data and environmental data and generating control instructions for optimizing the work environment, thereby enabling immediate improvement of the work environment in the factory facility and improvement of the work efficiency of workers.

[1275] "Living space" refers to the space where residents live their daily lives, such as the indoor environment of a home or office.

[1276] A "sensor" is a general term for a device that detects physical environmental data and outputs that data as an electrical signal. Examples include temperature sensors, humidity sensors, illuminance sensors, and motion sensors.

[1277] A "database" is a collection of information in which collected data is systematically stored and managed so that it can be efficiently searched and analyzed as needed.

[1278] "Means for analyzing data" refers to the methods and devices used to convert collected data into useful information such as user behavior patterns and preferences.

[1279] "User" refers to the person who uses this system, and is the target for improving their comfort and convenience.

[1280] "Behavioral patterns" refer to the actions and patterns of behavior that users repeatedly perform in their daily lives.

[1281] "Environmental data" is physical data collected by sensors, including temperature, humidity, light, and motion.

[1282] "Emotional data" refers to data that indicates the emotional state of a worker or user, as determined from their facial expressions, tone of voice, and behavior.

[1283] "Means for generating suggestions" refers to a method or device that creates specific action suggestions or instructions for changing settings to be provided to users based on the analysis results.

[1284] "Means for notifying proposals" is a general term for methods and devices for notifying users of generated proposals, and includes notification functions on mobile devices, etc.

[1285] "Means for controlling home appliances" refers to methods and devices for automatically operating home appliances such as air conditioners and lights based on analysis results and suggestions.

[1286] "Factory facilities" refers to all environments within buildings and premises where manufacturing or production activities take place.

[1287] "Workers" refers to people engaged in manufacturing or production activities within a factory facility.

[1288] "Control instructions" are specific operational commands issued to optimize the working environment and equipment operation within a factory facility.

[1289] "Analyzing means" is a general term for processing methods and devices that process collected data and convert it into useful information.

[1290] "Optimizing the environment" means adjusting the environment to a state where users or workers can work comfortably and efficiently.

[1291] The system for implementing the present invention is composed of multiple sensors installed within a factory facility, a server that manages and analyzes the data, an emotion engine, and a terminal that notifies workers.

[1292] System Configuration

[1293] sensor

[1294] Temperature sensor: Measures the temperature in the factory in real time and sends the value to the server.

[1295] Humidity sensor: Measures the humidity level in the factory and sends the data to a server.

[1296] Illuminance sensor: Detects whether the light is on or off and the brightness level, and sends that information to the server.

[1297] Motion sensor: Detects worker movement and reports the presence of people in the work area to the server.

[1298] server

[1299] The server receives data (temperature, humidity, illuminance, motion) sent from each sensor and stores it in a database in chronological order. The stored data is organized by worker.

[1300] In addition, the server is equipped with an emotion engine that can recognize the emotions of workers through voice analysis, facial expression recognition, and behavior analysis.

[1301] Emotion engine: The server uses the emotion engine to analyze input data from the camera and microphone and determine the emotion of the worker from their facial expressions and tone of voice. For example, if a worker is feeling stressed, the emotion engine will recognize that information.

[1302] Data analysis and proposal generation

[1303] The AI ​​module installed on the server periodically analyzes the data in the database, using clustering techniques and machine learning algorithms to learn the behavioral patterns and preferences of workers.

[1304] The results of data analysis are combined with the output of the emotion engine to identify the working environment conditions (for example, the optimal temperature and lighting brightness in the factory) that will make workers feel most efficient and comfortable. This information is then generated as recommendations.

[1305] Notification and Execution

[1306] The server then sends the generated suggestions to mobile devices, factory robots, and other terminals, including specific suggestions such as "change the cooling device's set temperature to 28°C" or "adjust the lighting brightness to 70%."

[1307] The worker checks the notification and can either manually follow the suggestions or let the system automatically adjust them, and the server automatically changes the settings of the cooling equipment and lighting according to the worker's selection.

[1308] Specific examples

[1309] For example, if the temperature sensor in a factory detects 32°C, the humidity sensor indicates 60%, and the emotion engine recognizes that a worker is experiencing high stress levels, the AI ​​module will make a comprehensive judgment and generate a recommendation to set the cooling equipment to 28°C, a comfortable temperature for the worker. Furthermore, data from the light and motion sensors will suggest that the optimal lighting level is 70% when it is confirmed that a worker is present in the work area. These recommendations are sent to the worker's device, and if the worker agrees, the server will automatically change the cooling equipment and lighting settings.

[1310] Prompt Sentence Examples

[1311] By inputting prompts like the following into the generative AI model, we can have it suggest appropriate countermeasures:

[1312] "Currently, the temperature inside the factory has reached 32 degrees, and analysis of the workers' facial expressions has detected high stress levels. How should the robot respond in this situation? Please propose a method for operating the cooling device as well."

[1313] This system is expected to improve the working environment within the factory facility and increase worker efficiency and safety.

[1314] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1315] Step 1:

[1316] Sensor data collection

[1317] The server periodically acquires data from temperature, humidity, illuminance, and motion sensors installed in the factory. It receives real-time data from the various sensors as input and stores this data in the server's database as output. Specifically, it receives data from each sensor via HTTP requests or MQTT protocols, and stores the data in the database in JSON format.

[1318] Step 2:

[1319] Collecting Emotional Data

[1320] The server collects facial and voice data from the workers from cameras and microphones installed in the factory and analyzes it using an emotion engine. It receives the video and audio data from the cameras and microphones as input, and stores the analyzed emotional data (e.g., stress level, happiness level, etc.) as output in a database. Specifically, it analyzes the video data frame by frame and performs waveform analysis on the audio data to recognize emotions.

[1321] Step 3:

[1322] Data storage and management

[1323] The server organizes and stores the environmental data and emotion data acquired in steps 1 and 2 in a database. It receives various data as input and generates a database organized for each user as output. Specifically, it stores the data in a database system (e.g., MySQL, PostgreSQL, etc.) in chronological order and creates indexes as needed to enable efficient data searches.

[1324] Step 4:

[1325] Data analysis

[1326] The server analyzes the stored environmental and emotional data to learn the behavioral patterns and preferences of workers. It receives the stored data as input and generates analytical results as output. Specifically, it analyzes the data using machine learning algorithms (e.g., clustering, regression analysis) to identify behavioral patterns and emotional changes.

[1327] Step 5:

[1328] Proposal Generation

[1329] The server generates optimal working environment conditions based on the analysis results. It receives the data analysis results as input and generates specific suggestions (e.g., temperature settings for cooling devices, brightness adjustments for lighting) as output. The specific operation is to automatically generate suggestions using an AI module and format the content as a notification message.

[1330] Step 6:

[1331] Proposal Notification

[1332] The server notifies the worker's device of the generated proposal. It receives the generated proposal as input and sends a notification message to the worker as output. The specific operation is to notify the worker of the proposal content in real time using the notification function of the mobile device or factory robot.

[1333] Step 7:

[1334] Automatic Control

[1335] If the worker approves the proposal, the server automatically changes the settings of the cooling equipment and lighting in the factory. It receives the worker's approval as input and generates and sends equipment control instructions as output. Specifically, it uses the IoT platform to send control signals to the corresponding home appliances and implements the setting changes.

[1336] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1337] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1338] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1339] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1340] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1341] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1342] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1343] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1344] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1345] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1346] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1347] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1348] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1349] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1350] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1351] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1352] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1353] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1354] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1355] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1356] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1357] The following is further disclosed regarding the above embodiment.

[1358] (Claim 1)

[1359] A means of collecting data from multiple sensors installed in living spaces;

[1360] means for storing data collected from said sensors in a database;

[1361] means for analyzing the stored data to learn user behavior patterns and preferences;

[1362] means for generating a proposal for optimal environmental settings based on the analysis results;

[1363] means for notifying a user of the proposal and controlling a home appliance based on the proposal;

[1364] A system including:

[1365] (Claim 2)

[1366] The system of claim 1 , wherein the sensors detect temperature, humidity, light, and motion.

[1367] (Claim 3)

[1368] The system of claim 1, wherein the means for notifying the user uses a notification function of the mobile device.

[1369] "Example 1"

[1370] (Claim 1)

[1371] A means of collecting data from multiple sensors installed in living spaces;

[1372] means for storing data collected from the sensors in a database in chronological order;

[1373] means for analyzing the stored data using clustering techniques to learn user behavior patterns and preferences;

[1374] means for generating a proposal for comfortable environmental settings based on the analysis results;

[1375] means for notifying a user terminal of the proposal and automatically controlling the home appliance based on the proposal;

[1376] a means for recollecting and continuously storing environmental data based on the proposed configuration in a database;

[1377] A system including:

[1378] (Claim 2)

[1379] The system of claim 1 , wherein the sensors detect temperature, humidity, light, and motion.

[1380] (Claim 3)

[1381] The system of claim 1, wherein the means for notifying the user uses a notification function of the mobile device.

[1382] "Application Example 1"

[1383] (Claim 1)

[1384] A means of collecting data from multiple sensors installed in living spaces;

[1385] means for storing data collected from said sensors in a database;

[1386] means for analyzing the stored data to learn user behavior patterns and preferences;

[1387] means for generating a proposal for optimal environmental settings based on the analysis results;

[1388] means for notifying a user of the proposal and controlling an electronic device based on the proposal;

[1389] Using data collected from sensors, we analyze customer movements in commercial spaces and propose settings that improve comfort.

[1390] A system including:

[1391] (Claim 2)

[1392] The system of claim 1 , wherein the sensors detect temperature, humidity, light, and motion.

[1393] (Claim 3)

[1394] The system of claim 1, wherein the means for notifying the user uses a notification function of the mobile device.

[1395] "Example 2: Combining Emotion Engines"

[1396] (Claim 1)

[1397] A means of collecting data from multiple sensors installed in living spaces;

[1398] means for storing data collected from said sensors in a database;

[1399] emotion recognition means for recognizing the emotion of a user based on the data stored in the database;

[1400] means for analyzing the stored data and the emotion data obtained by the emotion recognition means to learn user behavior patterns and preferences;

[1401] means for generating recommendations for optimal environment settings based on the analysis results and data from the emotion recognition means;

[1402] a means for notifying a user of the proposal and controlling an environmental device based on the proposal;

[1403] A system including:

[1404] (Claim 2)

[1405] 10. The system of claim 1, wherein the sensors detect temperature, humidity, light, and motion and store the data in a database in chronological order.

[1406] (Claim 3)

[1407] 2. The system according to claim 1, wherein the means for notifying the user uses a notification function of an information display device.

[1408] "Application example 2 when combining emotion engines"

[1409] (Claim 1)

[1410] A means of collecting data from multiple sensors installed in living spaces;

[1411] means for storing data collected from said sensors in a database;

[1412] means for analyzing the stored data to learn user behavior patterns and preferences;

[1413] means for generating a proposal for optimal environmental settings based on the analysis results;

[1414] means for notifying a user of the proposal and controlling a home appliance based on the proposal;

[1415] A means for acquiring emotion data of workers in a factory facility;

[1416] means for analyzing the acquired emotion data and environmental data and generating control instructions for optimizing the work environment;

[1417] A system including:

[1418] (Claim 2)

[1419] 10. The system of claim 1, further comprising sensors for detecting temperature, humidity, light, and motion, and a device for analyzing worker emotions.

[1420] (Claim 3)

[1421] The system according to claim 1, wherein the means for notifying the user uses notification functions of a mobile device and a factory robot. [Explanation of symbols]

[1422] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting data from multiple sensors installed in living spaces; means for storing data collected from said sensors in a database; means for analyzing the stored data to learn user behavior patterns and preferences; means for generating a proposal for optimal environmental settings based on the analysis results; means for notifying a user of the proposal and controlling a home appliance based on the proposal; A system including:

2. The system of claim 1 , wherein the sensors detect temperature, humidity, light, and motion.

3. The system according to claim 1 , wherein the means for notifying the user uses a notification function of the mobile device.

Images