System

Abstract

A system is provided.SOLUTION: A system including means for collecting operational status data from various IoT devices, means for acquiring external environment data, means for performing analysis using a machine learning algorithm based on the collected data and calculating an optimal device operating condition, means for transmitting a control instruction to the various IoT devices based on the analysis result, means for providing a user interface for displaying the analysis result and the content of the control instruction to a user, and means for analyzing data accumulated over a long period of time on a large scale and specifying a trend or a pattern.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] IoT home appliances are becoming increasingly common, but only a limited number of users are able to fully utilize their potential. This is because the operation of each appliance is user-dependent, requiring knowledge and skills to determine optimal settings. Furthermore, energy consumption is often not adequately optimized for environmental considerations, resulting in unnecessary power consumption. This situation not only reduces user convenience but also increases environmental impact. Therefore, the purpose of this invention is to improve user convenience and energy efficiency while reducing environmental impact by automatically managing and controlling optimal operating conditions for various IoT home appliances based on their operating status and environmental data. [Means for solving the problem]

[0005] To solve the above problems, the system of the present invention includes the following means: means for collecting operation status data from various IoT devices; means for acquiring external environmental data; means for analyzing the collected data using a machine learning algorithm and calculating optimal device operating conditions; means for sending control instructions to the various IoT devices based on the analysis results; means for providing a user interface for displaying the analysis results and the contents of the control instructions to the user; and means for analyzing data accumulated over a long period of time on a large scale to identify trends and patterns. This system optimizes home energy consumption based on the collected data and analysis results, thereby realizing a sustainable living environment.

[0006] "IoT devices" are home appliances and devices that are connected to the Internet and can be remotely controlled and data collected.

[0007] "Operation status data" refers to information such as the current operating status, settings, power consumption, and operating time of each IoT device.

[0008] "Environmental data" refers to information about environmental conditions such as outside temperature, humidity, weather forecast, and air quality.

[0009] A "machine learning algorithm" is a computational method for analyzing collected data, learning patterns and trends, and estimating optimal operating conditions.

[0010] "Analysis" is the process of processing collected data, identifying trends and patterns, and determining optimal control conditions.

[0011] "Operating conditions" refers to the setting information and operating mode required for each IoT device to perform optimally.

[0012] "Control instructions" are specific operational instructions sent to each IoT device based on the analysis results.

[0013] A "user interface" is a screen or operating method that acts as an intermediary between a system and a user and enables the display and input of information.

[0014] "Data accumulated over a long period of time" refers to a data set that has been collected and stored continuously for a certain period of time or longer.

[0015] "Large-scale analysis" means using large amounts of data to extract trends and patterns and make highly accurate predictions and optimizations.

[0016] "Trends and patterns" are fluctuations and regularities over time that become apparent through data analysis.

[0017] A "sustainable living environment" is a living space that realizes sustainable living by optimizing energy efficiency and environmental impact. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] This invention relates to an "IoT AI Commander" system that works in conjunction with multiple IoT devices to achieve optimal operation and improve energy efficiency. This system is primarily composed of the following modules: a data collection module, a data analysis module, a control module, a user interface module, and a big data analysis module.

[0040] Explanation of the specific processing of the system

[0041] 1. Data Collection Module

[0042] The server periodically collects operational status data from each IoT device, including temperature settings, power consumption, and operating time. Environmental data from external temperature sensors and weather forecast APIs is also collected at the same time.

[0043] The terminal periodically transmits the operating status of each IoT device to the server.

[0044] Example: The server receives data from the air conditioner, such as "The current room temperature is 25°C, and the set temperature is 22°C." At the same time, it obtains data from the weather forecast API, such as "The current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C."

[0045] 2. Data Analysis Module

[0046] The server analyzes the collected data using machine learning algorithms to calculate the optimal operating conditions for the devices. In this process, it learns the efficient operating patterns of each IoT device and derives the settings that are best suited to the user's lifestyle and environmental conditions.

[0047] Example: The server performs an analysis and concludes that "it would be best to change the air conditioner's temperature setting to 23°C."

[0048] 3. Control Module

[0049] The server generates control instructions for each IoT device based on the analysis results. These control instructions are designed to reflect the optimal operating settings for each device.

[0050] The terminal receives control instructions from the server and changes the device settings accordingly.

[0051] Example: The server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends this instruction to the air conditioner. The air conditioner changes the set temperature to 23°C.

[0052] 4. User Interface Module

[0053] The server generates a dashboard that displays the analysis results and control instructions in an easy-to-understand format for users, and users can access this dashboard via their smartphones or PCs.

[0054] The dashboard allows users to view current device status and control settings and change them if necessary.

[0055] Example: A user receives a notification on their smartphone that says, "The air conditioner temperature setting has been changed to 23°C" and confirms it.

[0056] 5. Big Data Analysis Module

[0057] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[0058] Example: A server accumulates energy consumption data for the home on a yearly basis, analyzes seasonal usage patterns, and suggests that "it would be more energy efficient to not lower the set temperature any further in winter."

[0059] Example

[0060] For example, when a user leaves the house during the day in summer, the system analyzes the outside temperature, the temperature inside the house, and the weather forecast to automatically adjust the air conditioner's temperature setting. Even if the outside temperature is high and the house is predicted to heat up quickly, the system will maintain the optimal setting and maximize energy efficiency by returning the room to an appropriate temperature before the user returns. Long-term data analysis also identifies energy consumption trends in specific regions and seasons, allowing for predictive optimization.

[0061] In this way, the IoT AI Commander improves user convenience and reduces energy waste, ultimately reducing the environmental impact and helping to create a sustainable living environment.

[0062] The processing flow will be explained below.

[0063] Step 1:

[0064] The server periodically sends requests to collect operational status data from each IoT device, which triggers each IoT device to send its current operating status and configuration information to the server.

[0065] Step 2:

[0066] In response to the received request, the terminal sends the operating status data of each IoT device (temperature setting, power consumption, operating time, etc.) to the server.

[0067] Example: An air conditioner sends data to the server saying, "The current room temperature is 25°C, and the set temperature is 22°C."

[0068] Step 3:

[0069] The server accesses external environmental sensors and weather forecast APIs to obtain environmental data such as outside temperature, humidity, and weather forecasts.

[0070] Example: The server retrieves data from a weather forecast API: "The current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C."

[0071] Step 4:

[0072] The server preprocesses the collected operational status data and environmental data, which includes data cleanup, filling in errors and missing values, and standardizing the format.

[0073] Step 5:

[0074] The server uses machine learning algorithms to perform analysis based on the preprocessed data, which then calculates the optimal operating conditions for each IoT device.

[0075] Example: As a result of the analysis, the conclusion is reached that "it is optimal to change the air conditioner's set temperature to 23°C."

[0076] Step 6:

[0077] The server generates specific control instructions based on the analysis results, which are designed to reflect optimal operating settings for each IoT device.

[0078] Example: The server generates the instruction "Change the air conditioner temperature setting to 23°C."

[0079] Step 7:

[0080] The server then sends the generated control instructions to each IoT device, which then changes their operating settings in accordance with the control instructions.

[0081] Step 8:

[0082] The terminal executes the control instructions received from the server and changes the settings of the corresponding IoT device.

[0083] Example: An air conditioner receives an instruction to change the set temperature to 23°C.

[0084] Step 9:

[0085] The server generates a dashboard to display the analysis results and control instructions to the user, who can access the dashboard via a smartphone or PC.

[0086] Step 10:

[0087] The dashboard allows users to view current device status and control settings and change them if necessary.

[0088] Example: A user sees a notification on their smartphone that says, "The air conditioner temperature setting has been changed to 23°C."

[0089] Step 11:

[0090] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[0091] Step 12:

[0092] Based on the results of big data analysis, the server makes new predictions and optimizations, providing even more accurate control instructions.

[0093] Example: Based on winter usage data, we suggest that "it would be more energy efficient to not lower the set temperature any further."

[0094] Example 1

[0095] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0096] In systems using conventional IoT devices, efficient operation of each device and improvement of energy efficiency have not been sufficiently achieved. Furthermore, in many systems, it is difficult to effectively analyze collected data and derive optimal operating conditions. This poses a challenge in optimizing energy consumption in homes and businesses and providing a sustainable living environment.

[0097] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0098] In this invention, the server includes: means for collecting operational status data from various IoT devices; means for acquiring external environmental data; means for analyzing the collected data using a machine learning algorithm to calculate optimal device operating conditions; means for transmitting control instructions to various IoT devices based on the analysis results; means for providing a user interface for displaying the analysis results and the control instructions to the user; means for analyzing data accumulated over a long period of time on a large scale and identifying trends and patterns; means including a terminal for transmitting data collected from each IoT device to the server; and means for the server to generate and transmit control instructions to each IoT device. This enables integrated analysis of various data and rapid determination of optimal operating conditions. This significantly improves energy efficiency and realizes a sustainable living environment.

[0099] An "IoT device" is an electronic device that can connect to the Internet and collect, send, and receive data.

[0100] "Operational status data" refers to information about the operating status and behavior of IoT devices, including data such as temperature settings, power consumption, and operating time.

[0101] "Environmental data" refers to information about the state of the environment in which an IoT device is located, including, for example, outside temperature, humidity, and weather information.

[0102] A "machine learning algorithm" refers to a mathematical method or model that analyzes large amounts of data to learn patterns and rules and make predictions and classifications.

[0103] "Control instructions" refer to commands and configuration change instructions sent from a server to an IoT device, and are used to optimize the device's operation.

[0104] "User interface" refers to the software and hardware components that provide the visual and operational means for a user to interact with a system.

[0105] "Trends and patterns" refer to regularities and tendencies extracted from data collected over a long period of time, and are used to predict and optimize the future.

[0106] "Terminal" refers to an electronic device that functions as a relay point for transmitting data collected from IoT devices to a server.

[0107] "Server" refers to a central management device that collects and analyzes data, and generates and sends control instructions.

[0108] This invention provides an "IoT AI system" that works in conjunction with multiple IoT devices to achieve optimal operation and improve energy efficiency. This system is primarily composed of the following modules: a data collection module, a data analysis module, a control module, a user interface module, and a big data analysis module.

[0109] First, let's explain the data collection module. The server periodically collects operating status data from each IoT device. This data includes temperature settings, power usage, and operating hours. Environmental data is also simultaneously collected through outdoor temperature sensors and weather forecast APIs. The terminals send the operating status data collected from the IoT devices to the server. As a concrete example, the server receives data from an air conditioner that "the current room temperature is 25°C, the set temperature is 22°C," and obtains data from the weather forecast API that "the current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C."

[0110] Next, we will explain the data analysis module. The server uses the collected data to run machine learning algorithms and calculate the optimal device operating conditions. In this process, the server learns the efficient operating patterns of each IoT device and derives the settings that are most suitable for the user's lifestyle and environmental conditions. As a concrete example, the server concludes that "it is optimal to change the air conditioner's temperature setting to 23°C."

[0111] Next, we will explain the control module. The server generates and sends control instructions to each IoT device based on the analysis results. The terminal receives the control instructions from the server and changes the device settings accordingly. As a concrete example, the server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends this instruction to the air conditioner. The air conditioner changes the set temperature to 23°C.

[0112] Next, we will explain the user interface module. The server generates a dashboard to display the analysis results and control instructions in an easy-to-understand manner to the user. This dashboard can be accessed by the user via a smartphone or PC. The user can use the dashboard to check the current device status and control settings and change them as necessary. As a concrete example, the user receives a notification on their smartphone saying, "The air conditioner's set temperature has been changed to 23°C," and confirms this.

[0113] Finally, we will explain the big data analysis module. The server collects daily data and accumulates the analysis results over the long term, saving them as big data. This big data is used to further analyze trends and patterns and create future prediction models. As a concrete example, the server accumulates household energy consumption data on an annual basis, analyzes seasonal usage patterns, and suggests that "it would be more energy-efficient to not lower the temperature setting any further in winter."

[0114] For example, when a user goes out during the day in summer, the system analyzes the outside temperature, the temperature inside the house, and the weather forecast, and automatically adjusts the air conditioner's temperature setting. Even if the outside temperature is high and the house is predicted to heat up quickly, the system maintains the optimal setting and maximizes energy efficiency by returning the room to an appropriate temperature before the user returns home. Long-term data analysis also identifies energy consumption trends in specific regions and seasons, and performs optimization based on predictions. In this way, the IoT AI system improves user convenience and reduces energy waste. As a result, it is possible to reduce environmental impact and aim to create a sustainable living environment.

[0115] An example of a prompt for a generative AI model is, "Please explain what specific processing the IoT AI system performs at each step. As an example, please show a process that includes how to optimize the air conditioner's temperature setting."

[0116] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0117] Step 1:

[0118] Starting Data Collection

[0119] The server periodically sends data collection requests to each IoT device. These requests include instructions for operational status data such as temperature setting, power usage, and operating time. The terminal receives these requests and collects the necessary data from each IoT device. As a specific example, the terminal obtains data from an air conditioner that "the current room temperature is 25°C, and the set temperature is 22°C." The input is a data collection request from the server, and the output is operational status data from each IoT device.

[0120] Step 2:

[0121] Obtaining environmental data

[0122] The server obtains environmental data through an external temperature sensor or weather forecast API. Specifically, it obtains the current outside temperature and weather information. For example, the server uses a weather forecast API to obtain data such as "The current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C." The input is a request from an external sensor or API, and the output is environmental data.

[0123] Step 3:

[0124] Data transmission

[0125] The device sends the collected operational status data to the server. The server stores the environmental data obtained from the weather forecast API. Specifically, the device sends data to the server such as "current room temperature is 25°C, set temperature is 22°C." The input is the operational status data from the device, and the output is the operational status data and environmental data stored on the server.

[0126] Step 4:

[0127] Data integration

[0128] The server integrates the operational status data and environmental data from each IoT device. The input is the operational status data and environmental data, and the output is an integrated data set.

[0129] Step 5:

[0130] Running machine learning algorithms

[0131] The server uses the integrated data to run a machine learning algorithm and calculate the optimal device operating conditions. This process involves data preprocessing, feature extraction, and model training. For example, the server may conclude that "setting the air conditioner temperature to 23°C is optimal." The input is the integrated data set, and the output is the optimal operating conditions.

[0132] Step 6:

[0133] Control instruction generation

[0134] The server generates control instructions for each IoT device based on the analysis results. Specifically, the server generates an instruction to "change the air conditioner's set temperature to 23°C." The input is the optimal operating conditions, and the output is the generated control instruction.

[0135] Step 7:

[0136] Sending control instructions

[0137] The server sends the generated control instructions to each IoT device. The terminal receives the control instructions from the server and changes the device settings accordingly. Specifically, the server sends the instruction "change the air conditioner's set temperature to 23°C," and the air conditioner changes its set temperature. The input is the generated control instruction, and the output is the setting change of the IoT device that received the control instruction.

[0138] Step 8:

[0139] Generate a dashboard

[0140] The server generates a dashboard to display the analysis results and control instructions in an easy-to-understand manner for the user. The inputs are the analysis results and control instructions, and the output is the generated dashboard.

[0141] Step 9:

[0142] Accessing the User Interface

[0143] Users access the dashboard using a smartphone or PC to check the current device status and control settings. Specifically, the user receives a notification on their smartphone saying, "The air conditioner temperature setting has been changed to 23°C," and confirms it. The input is the dashboard URL and app, and the output is status information that the user can check.

[0144] Step 10:

[0145] Accumulation and analysis of big data

[0146] The server collects daily data and accumulates the analysis results over the long term, saving them as big data. Data accumulated over a long period of time is analyzed on a large scale to identify trends and patterns. Specifically, the server accumulates household energy consumption data on an annual basis and suggests, "It would be more energy-efficient to not lower the temperature setting any further in winter." The input is the daily collected data and analysis results, and the output is the accumulated big data and its analysis results.

[0147] (Application example 1)

[0148] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0149] Modern self-driving vehicles operate using numerous information and communication technology devices, but it is difficult to efficiently manage the operating status of these devices and external environmental data, and calculate optimal operating conditions in real time. Furthermore, systems that automatically set optimal routes and speeds taking into account traffic information and weather conditions have not yet been fully realized. As a result, there are problems with wasted energy consumption and reduced operating efficiency.

[0150] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0151] In this invention, the server includes means for collecting operation status data from various information and communication technology devices, means for acquiring external environment data, means for analyzing the collected data using a machine learning algorithm and calculating optimal device operating conditions, means for sending control instructions to the various information and communication technology devices based on the analysis results, means for providing a user interface for displaying the analysis results and the content of the control instructions to the user, means for analyzing data accumulated over a long period of time on a large scale and identifying trends and patterns, means for collecting and analyzing vehicle status and traffic information to calculate optimal driving routes and speed settings, and means for sending control instructions to the autonomously driven vehicle based on the analysis results, thereby optimizing the operating conditions of the autonomously driven vehicle and enabling improved energy efficiency and operating efficiency.

[0152] "Information and communication technology equipment" means electronic devices for data collection, communication, control, etc.

[0153] "Operation status data" refers to information relating to the operating status of the device, such as its usage status, power consumption, temperature, and usage time.

[0154] "External environment data" is information about surrounding conditions such as weather, temperature, humidity, and traffic conditions.

[0155] A "machine learning algorithm" is a mathematical method for learning patterns and relationships from data and making predictions and classifications.

[0156] A "control instruction" is an instruction to change or adjust the operation of a device.

[0157] A "user interface" is the screen or method of operation that allows a user to interact with a system.

[0158] "Trends and patterns" are tendencies and regularities that emerge from data collected over a long period of time.

[0159] An "autonomous vehicle" is a vehicle that operates autonomously without the need for human operation.

[0160] A "travel route" is a specific route that a vehicle will take to reach its destination.

[0161] "Speed ​​setting" is a parameter that determines the vehicle's traveling speed.

[0162] This invention relates to an "Autonomous Vehicle AI Optimization Commander" system that works in conjunction with multiple information and communication technology devices to optimize operation and improve energy efficiency. This enables vehicle operation to be optimized and energy efficiency to be improved. The system is primarily composed of the following modules: a data collection module, a data analysis module, a control module, a user interface module, and a big data analysis module.

[0163] Explanation of the specific processing of the system

[0164] 1. Data Collection Module

[0165] The server periodically collects operational status data from the autonomous vehicle, including remaining battery power, driving speed, distance to destination, etc. It also simultaneously obtains environmental data from traffic information APIs and weather forecast APIs.

[0166] The terminal transmits operation-related data, including traffic and weather information, to the server.

[0167] Example: A server receives data from an autonomous vehicle, such as "Current battery remaining is 80%, driving speed is 60km / h, distance to destination is 20km." At the same time, it obtains data from a traffic information API that "Congestion is occurring" and from a weather forecast API that "Current weather is rain."

[0168] 2. Data Analysis Module

[0169] The server analyzes the collected data using machine learning algorithms to calculate the optimal vehicle route and speed. In this process, the server learns the optimal driving pattern based on data obtained from various information and communication technology devices, and derives the settings that are most suitable for real-time environmental conditions.

[0170] Example: The server performs an analysis and concludes, "Change driving speed to 55 km / h and re-route to avoid traffic jams."

[0171] 3. Control Module

[0172] The server generates control instructions for the autonomous vehicle based on the analysis results, which reflect the optimal driving route and speed settings.

[0173] The terminal receives control instructions from the server and changes the settings of the autonomous vehicle accordingly.

[0174] Example: The server generates an instruction to "change driving speed to 55 km / h and re-plan the route" and sends this instruction to an autonomous vehicle. The autonomous vehicle changes its driving speed to 55 km / h and re-plans the route.

[0175] 4. User Interface Module

[0176] The server generates a dashboard that displays the analysis results and control instructions in an easy-to-understand format for users, and users can access this dashboard via their smartphones or PCs.

[0177] The dashboard allows users to view the current vehicle status and control settings and change them if necessary.

[0178] Example: A user receives a notification on their smartphone that says "Your driving speed has been changed to 55km / h" and confirms it.

[0179] 5. Big Data Analysis Module

[0180] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[0181] Example: A server accumulates energy consumption data of autonomous vehicles on a yearly basis, analyzes seasonal usage patterns, and suggests that "it would be more energy-efficient to not reduce the speed any further in winter."

[0182] Prompt Sentence Examples

[0183] As a concrete example, the following prompt sentences can be input to a generative AI model:

[0184] "Vehicle ID 12345's current speed is 60 km / h, battery remaining is 80%, distance to destination is 20 km, there is traffic congestion information, and it is raining. Please predict the optimal driving speed under these conditions."

[0185] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0186] Step 1:

[0187] Data collection

[0188] The server collects operational status data from the autonomous vehicles (remaining battery level, driving speed, distance to destination, etc.) and simultaneously obtains environmental data (traffic conditions, weather information) from traffic information APIs and weather forecast APIs.

[0189] Input: Data from autonomous vehicles, traffic information API, weather forecast API

[0190] Output: Collected data (remaining battery level, driving speed, distance to destination, traffic conditions, weather information)

[0191] Specific operation: The server periodically connects to the autonomous vehicle to obtain operating status data, and at the same time calls the API to obtain external environment data.

[0192] Step 2:

[0193] Data analysis

[0194] The server then inputs the collected data into a machine learning algorithm to calculate the optimal route and speed, taking into account the vehicle's remaining battery power, current speed, distance to destination, traffic conditions, weather information, and more.

[0195] Input: Collected data (battery level, driving speed, distance to destination, traffic conditions, weather information)

[0196] Output: Optimal driving path and speed settings

[0197] Specific operation: The server uses a machine learning model (e.g., a Linear Regression model) to analyze the input data and calculate the optimal driving route and speed.

[0198] Step 3:

[0199] Control instruction generation

[0200] The server generates control instructions for the autonomous vehicle based on the results of the data analysis, including instructions on optimal driving routes and speed settings.

[0201] Input: Data analysis results (optimal driving route and speed settings)

[0202] Output: Control instructions for the autonomous vehicle

[0203] Specific operation: The server executes a control algorithm based on the analysis results and generates control instructions for the autonomous vehicle.

[0204] Step 4:

[0205] Sending control instructions

[0206] The server then sends the generated control instructions to the autonomous vehicle, which then adjusts its route and speed based on these instructions.

[0207] Input: Generated control instructions

[0208] Output: The behavior of the autonomous vehicle based on the executed control instructions.

[0209] Specific operation: The server sends instructions to the autonomous vehicle to adjust its speed and route, and the autonomous vehicle carries out the instructions.

[0210] Step 5:

[0211] Display in the user interface

[0212] The server generates a user interface to display the analysis results and control instructions to the user, who can then check them on their smartphone or PC.

[0213] Input: Analysis results, control instructions

[0214] Output: What is displayed on the user interface

[0215] Specific operation: The server generates a dashboard and sends information to a smartphone or PC to display the analysis results and control instructions.

[0216] Step 6:

[0217] Big data accumulation and analysis

[0218] The server collects daily data and accumulates the analysis results over the long term, saving them as big data. This big data can then be used to analyze more detailed trends and patterns.

[0219] Input: Daily data collection results, analysis results

[0220] Output: Long-term trends and patterns

[0221] Specific operation: The server periodically analyzes the accumulated data and updates the future prediction model.

[0222] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0223] This invention relates to an "IoT AI Commander" system that works in conjunction with multiple IoT devices to achieve optimal operation and improve energy efficiency. This system aims to provide a more user-friendly and comfortable environment by combining it with an emotion engine that recognizes the user's emotions. The specific processing of the system is described below.

[0224] Explanation of the specific processing of the system

[0225] 1. Data Collection Module

[0226] The server periodically sends requests to collect operational status data from each IoT device, including the device's operating status, configuration information, power consumption, and operating time.

[0227] The terminal periodically sends operational status data of the IoT device to the server.

[0228] Example: An air conditioner sends data to the server saying, "The current room temperature is 25°C, and the set temperature is 22°C."

[0229] 2. Emotion Recognition by Emotion Engine

[0230] The server recognizes emotions by analyzing the user's facial expressions and tone of voice, using cameras and microphones to collect data in real time.

[0231] Example: If a user is in their living room and has a relaxed expression, the emotion engine will recognize that expression and determine that they are "relaxed."

[0232] 3. Data integration and preprocessing

[0233] The server integrates the operational status data of IoT devices, environmental data (outside temperature, weather forecast, etc.), and emotion data, and performs preprocessing, which includes data cleansing, missing value completion, and format standardization.

[0234] 4. Analysis using machine learning algorithms

[0235] The server then uses machine learning algorithms to analyze the preprocessed data, with the goal of calculating the optimal operating conditions for each IoT device.

[0236] Example: As a result of the analysis, the conclusion is reached that "it is optimal to change the air conditioner's set temperature to 23°C."

[0237] 5. Integrated analysis with emotion data

[0238] The server further adjusts the device's operating conditions based on the user's emotional data obtained by the emotion engine. If the user is feeling stressed, the server may make additional adjustments, such as changing the lighting to a warmer color and playing music.

[0239] Example: If the user is determined to be stressed, not only will the air conditioner temperature be changed to 23°C, but the color temperature of the lighting will be changed to 3000K (warm color) to create a relaxing environment.

[0240] 6. Generating and Sending Control Instructions

[0241] The server generates the necessary control instructions based on the analysis results and sends them to each IoT device.

[0242] The terminal receives control instructions from the server and changes the device settings accordingly.

[0243] Example: The server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends it to the air conditioner. The air conditioner changes the set temperature to 23°C.

[0244] 7. User Interface

[0245] The server generates a dashboard to display the analysis results and control instructions to users. Users can access the dashboard on their smartphones or PCs to check and change the current device status and control settings.

[0246] Users can monitor the operation of IoT devices based on the information displayed on the dashboard and manually change settings as needed.

[0247] Example: A user sees a notification on their smartphone that says "The air conditioner temperature setting has been changed to 23°C" and manually adjusts the temperature setting further if necessary.

[0248] 8. Big Data Analysis

[0249] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[0250] Example: Analyzing energy consumption data for each season (summer and winter) and proposing more efficient operation methods.

[0251] In this way, the IoT AI Commander improves user convenience and reduces energy waste while taking emotions into consideration, ultimately reducing the environmental impact and helping to create a sustainable living environment.

[0252] The processing flow will be explained below.

[0253] Step 1:

[0254] The server periodically sends requests to collect operational status data from each IoT device, prompting each device to send its current operating status, configuration information, power consumption, operating time, and other information.

[0255] Step 2:

[0256] The terminal receives a request from the server and transmits the operational status data of each IoT device to the server.

[0257] Example: An air conditioner sends information to the server that "the current room temperature is 25°C, and the set temperature is 22°C."

[0258] Step 3:

[0259] The server accesses external environmental data sources (e.g., weather forecast APIs or external sensors) to obtain environmental data such as outside temperature, humidity, and weather forecasts.

[0260] Example: The server retrieves data from a weather forecast API: "The current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C."

[0261] Step 4:

[0262] The server runs an emotion engine to obtain emotion data by analyzing the user's facial expressions and voice. Data is collected in real time via cameras and microphones.

[0263] Example: A camera and microphone analyze a user's facial expressions and tone of voice to determine whether they are relaxed or stressed.

[0264] Step 5:

[0265] The server integrates the collected operational status data, environmental data, and emotion data and performs preprocessing, which cleanses the data, fills in missing values, and standardizes the format.

[0266] Step 6:

[0267] The server uses machine learning algorithms to analyze the preprocessed data and calculate the optimal operating conditions for each IoT device.

[0268] Example: After analysis, you conclude that the optimal temperature setting for an air conditioner is 23°C.

[0269] Step 7:

[0270] The server optimizes the analysis results by integrating the emotional data, generating different control instructions depending on whether the user is relaxed or stressed.

[0271] Example: If the user is determined to be stressed, the system will instruct the user to change the air conditioner temperature setting to 23°C, change the lighting color temperature to warmer colors, and play relaxing music.

[0272] Step 8:

[0273] The server then sends the generated control instructions to each IoT device, which then changes their behavior in accordance with the new settings.

[0274] Step 9:

[0275] The terminal receives control instructions from the server and changes the settings of the corresponding IoT device.

[0276] Example: The air conditioner changes the set temperature to 23°C, the smart lighting changes the color temperature to 3000K (warm), and the music player starts playing relaxing music.

[0277] Step 10:

[0278] The server generates a dashboard to display the analysis results and control instructions to the user, which can be viewed on a smartphone or PC.

[0279] Step 11:

[0280] Users can use the dashboard to view current device status and control settings, and manually change settings if necessary.

[0281] Example: A user sees a notification on their smartphone that says "The air conditioner temperature setting has been changed to 23°C" and then changes the temperature setting to 25°C.

[0282] Step 12:

[0283] The server accumulates daily data over the long term and stores it as big data. This data is used to analyze trends and patterns and create predictive models for the future.

[0284] Example: Analyzing past data, understanding seasonal energy consumption patterns, and proposing optimal energy consumption plans.

[0285] The above is the specific processing flow of the "IoT AI Commander" system, which combines user emotional data. This system improves user convenience and comfort while reducing energy waste and realizing a sustainable living environment.

[0286] Example 2

[0287] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0288] Modern IoT devices collect large amounts of data, improving user convenience. However, analyzing the data and calculating optimal device operating conditions requires advanced technology. Furthermore, existing systems do not adequately consider user emotions and comfort, creating challenges in improving the user experience. Furthermore, there is a need for more methods to utilize collected data over the long term to optimize energy efficiency and realize sustainable living environments.

[0289] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0290] In this invention, the server includes means for collecting operation status data from various IoT devices, means for acquiring external environmental data, means for analyzing the collected data using a machine learning algorithm and calculating optimal device operating conditions, means for sending control instructions to the various IoT devices based on the analysis results, means for providing a user interface for displaying the analysis results and the control instructions to the user, means for analyzing data accumulated over a long period of time on a large scale and identifying trends and patterns, means for analyzing the user's facial expressions and tone of voice to collect emotion data, and means for adjusting the IoT device operating conditions based on the emotion data. This makes it possible to calculate optimal device operating conditions taking user emotions into consideration, improve energy efficiency, and realize a sustainable living environment.

[0291] "IoT devices" is a general term for devices that are connected to the Internet and can collect and control data by communicating with each other.

[0292] "Operational status data" refers to information about the operation of an IoT device, such as its operating status, settings, power consumption, and operating time.

[0293] "Environmental data" refers to information about the external environment, such as outside temperature, humidity, and weather forecasts.

[0294] "Machine learning algorithms" refers to technologies that include mathematical models and statistical methods for analyzing collected data and calculating optimal equipment operating conditions.

[0295] "Control instructions" refer to specific operational commands sent to IoT devices to achieve optimal device operating conditions.

[0296] "User interface" refers to the display screen and operating means that allow a user to interact with the system and check or change the system's status.

[0297] "Large-scale analysis" refers to techniques and methods that use large amounts of data accumulated over a long period of time to identify trends and patterns.

[0298] "Emotion data" refers to information that indicates the user's emotional state, obtained from the user's facial expression, tone of voice, etc.

[0299] "Trend" or "pattern" refers to a statistical tendency or recurring characteristic derived from collected data.

[0300] "Operating conditions" refers to the settings and operating modes required for optimal operation of IoT devices.

[0301] A "sustainable living environment" refers to a sustainable living space that improves energy efficiency, reduces waste, and reduces the environmental impact.

[0302] This invention relates to an "IoT AI Commander" system that aims to achieve optimal operation and improved energy efficiency by linking multiple IoT devices. This system aims to provide a more user-friendly and comfortable environment by combining it with an emotion engine that recognizes the user's emotions.

[0303] The system's main components are a server, a device, and a user. The server collects operational status data from various IoT devices and acquires external environmental data, then analyzes all collected data using a machine learning algorithm. Based on the analysis results, it sends control instructions to the IoT devices and displays them to the user through a user interface. In addition, an emotion engine analyzes the user's facial expressions and tone of voice, collects emotional data, and adjusts the operating conditions of the IoT devices. This increases user comfort and improves energy efficiency.

[0304] Hardware and Software Examples

[0305] Servers: High-performance API servers, database management systems (e.g., MySQL), machine learning frameworks (e.g., TensorFlow)

[0306] Terminal: IoT device (e.g. smart speaker, smart air conditioner, smart light)

[0307] Emotion engine: Image and speech recognition technologies (e.g., OpenCV, Google Speech-to-Text API)

[0308] Explanation of program processing

[0309] Data collection

[0310] The server periodically collects operational status data from various IoT devices. Specifically, it sends API requests and receives data such as operating status, configuration information, and power consumption from the IoT devices. The terminal is responsible for sending operational status data to the server and providing the specific operating status of air conditioners and lighting.

[0311] Example: An air conditioner sends the following data to the server: "The current room temperature is 25°C, the set temperature is 22°C, and the power consumption is 1.2kW."

[0312] Example prompt: "Please report your current air conditioning settings and operating status."

[0313] Collecting Emotional Data

[0314] The server analyzes the user's facial expressions and tone of voice in real time through a camera and microphone. The emotion engine then sends the collected emotional data to the server.

[0315] Example: The server analyzes video data from a camera to detect whether the user is laughing, and also determines from the tone of voice that the user is relaxed.

[0316] Example prompt: "Analyze the user's facial expression and tone of voice and report their current emotions."

[0317] Data preprocessing and integrated analysis

[0318] The server integrates the collected IoT device data and emotion data, cleansing the data, filling in missing values, and standardizing the format. It then analyzes the data using machine learning algorithms to calculate the optimal operating conditions for the IoT device. The analysis results and emotion data are integrated to generate the final control instructions.

[0319] Example: A machine learning model calculates the optimal temperature setting for an air conditioner as 23°C. If the user is relaxing, change the color temperature of the lights to a warmer color and play music.

[0320] Example prompt: "Analyze the data and calculate the optimal temperature setting for your air conditioner."

[0321] Generating and sending control instructions

[0322] The server generates specific control instructions based on the analysis results and sends them to the IoT device. The device receives the control instructions from the server and changes the device settings accordingly.

[0323] Example: The server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends it to the air conditioner. The air conditioner changes the set temperature to 23°C.

[0324] Example prompt: "Please tell the air conditioner to change the temperature setting to 23°C."

[0325] Providing a user interface

[0326] The server generates a dashboard to display the analysis results and control instructions to the user, providing a user interface. Users can access the dashboard on their smartphones or PCs to check and change the current device status and control settings.

[0327] Example: A user sees a notification on their smartphone that says "The air conditioner temperature setting has been changed to 23°C" and manually adjusts the temperature setting further if necessary.

[0328] Example prompt: "Show me the current air conditioner temperature setting and operating status."

[0329] Big Data Analysis

[0330] The server collects daily data and accumulates the analysis results over the long term, saving them as big data. This big data is used to analyze trends and patterns and create future prediction models.

[0331] Example: A server analyzes historical energy consumption data and proposes new algorithms to improve energy efficiency.

[0332] Example prompt: "Based on historical energy consumption data, create a proposal to improve the operational efficiency of your air conditioner."

[0333] Through the above process, this system optimizes energy usage while maintaining user comfort, providing a sustainable living environment.

[0334] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0335] Step 1: Collecting IoT device data

[0336] The server sends a request to each IoT device to collect operational status data.

[0337] Input: Data collection request sent from the server to the IoT device.

[0338] Output: Operational status data sent from IoT devices (e.g., room temperature, set temperature, power consumption).

[0339] Specific operation: The server sends a request to the air conditioner, and the air conditioner replies with the data "Current room temperature is 25°C, set temperature is 22°C, power consumption is 1.2kW."

[0340] Step 2: Obtaining environmental data

[0341] The server obtains external environmental data (e.g., outside temperature, weather forecast).

[0342] Input: Real-time environmental data that the server retrieves from external data sources.

[0343] Output: The captured environmental data.

[0344] Specific operation: The server accesses the weather API to obtain the current outside temperature and weather forecast.

[0345] Step 3: Collecting emotion data

[0346] The server collects the user's facial expressions and tone of voice in real time through a camera and microphone.

[0347] Input: Video data from the camera and audio data from the microphone.

[0348] Output: Emotion data indicating the user's emotions.

[0349] How it works: The server analyzes the camera footage to detect whether the user is smiling and also determines from the tone of their voice that they are relaxed.

[0350] Step 4: Preprocessing the data

[0351] The server integrates and preprocesses data from IoT devices, environmental data, and emotion data.

[0352] Input: Operational status data from IoT devices, external environmental data, and user emotion data.

[0353] Output: Preprocessed integrated data.

[0354] Specific operation: The server fills in missing values ​​of each data with the average value and unifies the data format.

[0355] Step 5: Machine learning analysis

[0356] The server uses machine learning algorithms to perform analysis on the pre-processed data.

[0357] Input: Preprocessed integrated data.

[0358] Output: Optimal operating conditions for IoT devices.

[0359] What it does: A machine learning model calculates the optimal temperature setting for an air conditioner: 23°C. The algorithm then derives settings that minimize energy consumption and maximize user comfort.

[0360] Step 6: Integrated analysis with emotion data

[0361] The server readjusts the analysis results by taking into account the emotion data obtained from the emotion engine.

[0362] Input: Analysis results from machine learning algorithms, emotion data.

[0363] Output: Optimal operating conditions taking emotions into account.

[0364] Specific behavior: If the user is determined to be relaxed, the air conditioner temperature setting is changed to 23°C and the lighting color temperature is changed to 3000K (warm color).

[0365] Step 7: Generate and send control instructions

[0366] The server generates control instructions based on the analysis results and sends them to each IoT device.

[0367] Input: Optimal operating conditions taking emotions into account.

[0368] Output: Specific control instructions for each IoT device.

[0369] Specific operation: The server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends it to the air conditioner. The air conditioner changes the set temperature to 23°C.

[0370] Step 8: Update the User Interface

[0371] The server displays the analysis results and the contents of the control instructions on a dashboard and provides them to the user.

[0372] Input: Control instructions and analysis results.

[0373] Output: The dashboard content that is displayed to the user.

[0374] Specific behavior: The user sees a notification on their smartphone that says "The air conditioner temperature setting has been changed to 23°C" and adjusts it as needed.

[0375] Step 9: Analyze the big data

[0376] The server collects daily data and stores the analysis results over the long term, and performs large-scale analysis.

[0377] Input: Data accumulated over a long period of time.

[0378] Output: Analysis results that identify trends and patterns.

[0379] Specific operation: The server analyzes past energy consumption data and proposes new algorithms to improve energy efficiency.

[0380] Through these steps, the system optimizes energy consumption and realizes sustainable living while maintaining user comfort.

[0381] (Application example 2)

[0382] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0383] In existing IoT environments, it is difficult to set optimal device operation settings that take user emotions into account, which means that users cannot always enjoy comfort and convenience.In addition, the effective use of collected data and the optimization of long-term energy consumption are not sufficiently implemented, making it difficult to achieve a sustainable living environment.

[0384] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting operation status data from various sensing devices, means for acquiring external environmental information, means for analyzing the collected data and user emotion data using a machine learning algorithm and calculating optimal device operating conditions, means for sending control instructions to various devices based on the analysis results, means for providing a user interface for displaying the analysis results and the content of the control instructions to the user, means for analyzing data accumulated over a long period of time on a large scale and identifying trends and patterns, means for recognizing emotions by analyzing the user's facial expressions and tone of voice, and means for adjusting device operation settings based on the emotion data. This makes it possible to provide a comfortable environment that is in tune with the user's emotions, while optimizing energy consumption and realizing a sustainable living environment.

[0385] A "sensing device" is a device that detects information about the state of the environment or an object and collects that data.

[0386] "Environmental information" refers to data about the external environment, such as temperature, humidity, weather, and illuminance.

[0387] "Device" refers to various equipment that collects operational status data and is controlled in conjunction with external systems.

[0388] "Emotion data" is information about the user's psychological state obtained by analyzing the user's facial expressions and tone of voice.

[0389] A "machine learning algorithm" is a computational method for building patterns and predictive models based on collected data and calculating optimal operating conditions.

[0390] A "user interface" is a system that allows users to visually check and operate analysis results and control instructions.

[0391] "Trend" refers to a tendency or pattern of fluctuation that can be found by analyzing collected data over a long period of time.

[0392] "Data collection means" refers to the function for collecting necessary data from various sensing devices.

[0393] "External information acquisition means" refers to a function for acquiring data related to the external environment.

[0394] "Analysis means" refers to the function of analyzing collected data using machine learning algorithms to derive optimal operating conditions.

[0395] "Control instruction sending means" refers to a function for sending instructions to various devices based on the analysis results.

[0396] "Emotion recognition means" refers to a function for recognizing emotions by analyzing the user's facial expressions and tone of voice.

[0397] "Operation setting adjustment means" refers to a function for adjusting the operation settings of a device based on emotion data.

[0398] A system for implementing the present invention is configured as follows.

[0399] Specific system configuration

[0400] 1. Data Collection Module

[0401] The server periodically collects operational status data from various sensing devices, including the operating status, configuration information, power consumption, and operating time of each device.

[0402] The terminal periodically sends operational status data of the IoT device to the server.

[0403] Example: A lighting system sends data to a server saying "Current brightness is 500 lumens, set brightness is 400 lumens."

[0404] 2. Environmental Information Acquisition Module

[0405] The server acquires external environmental information (temperature, humidity, weather information, etc.).

[0406] Example: Obtain information from a weather data provider such as "The current outside temperature is 28°C and the humidity is 70%."

[0407] 3. Emotion Recognition Module

[0408] The server analyzes the user's facial expressions and tone of voice to recognize their emotions, using cameras and microphones to collect data in real time.

[0409] Example: A camera in a store captures the user's facial expression and determines that they are "relaxed."

[0410] 4. Analysis Module

[0411] The server uses machine learning algorithms to analyze the collected data and calculate the optimal operating conditions for the device.

[0412] Example: Analysis results in the conclusion that "adjusting the lighting brightness to 420 lumens is optimal."

[0413] 5. Control instruction transmission module

[0414] The server generates the necessary control instructions based on the analysis results and sends them to the various devices.

[0415] The terminal receives control instructions from the server and changes the device settings accordingly.

[0416] Example: The server generates an instruction to "change the lighting brightness to 420 lumens" and sends it to the lighting system. The lighting system changes the set brightness to 420 lumens.

[0417] 6. User Interface

[0418] The server provides an interface for displaying the analysis results and control instructions to the user. Users can access the interface on their smartphones or PCs to check and change the current device status and settings.

[0419] Based on the information displayed in the interface, the user can monitor the device's operation and manually change settings if necessary.

[0420] Example: A user sees a notification on their smartphone that says "Light setting brightness changed to 420 lumens" and manually adjusts the setting if necessary.

[0421] 7. Big Data Analysis Module

[0422] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[0423] Example: Analyzing seasonal energy consumption data and proposing more efficient operation methods.

[0424] 8. Operational Settings Adjustment Module

[0425] The server adjusts the device's operational settings based on the user's emotional data. If the user is feeling stressed, additional adjustments are made, such as changing the lighting to warmer colors and playing music.

[0426] Example: If the user is determined to be stressed, the color temperature of the lighting is changed to 3000K (warm color) to create a relaxing environment.

[0427] Specific examples

[0428] 1. While customers are browsing products in a store, smart glasses analyze their facial expressions and, if they are smiling, change the background music to a brighter tone.

[0429] 2. If customers seem stressed, reset the store's air conditioning to a comfortable temperature and change the lighting to a warmer color.

[0430] Prompt Sentence Examples

[0431] "Please provide a blueprint for an application that uses smart glasses and IoT devices to optimize the environment in a physical store according to the customer's emotions. As a specific example, please include a scenario in which cheerful music is played for smiling customers, and the air conditioning and lighting are adjusted for stressed customers."

[0432] The above is a detailed example of how the "IoT AI Commander" can be applied in a brick-and-mortar store. By using this system, it is possible to provide a comfortable environment that is sensitive to the user's emotions, while optimizing energy consumption and realizing a sustainable living environment.

[0433] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0434] Step 1:

[0435] Data collection

[0436] The server collects operational status data from various sensing devices. Input data includes the operating status, setting information, power consumption, and operating time of each device. This data is collected and stored in the server's database. Specifically, the lighting system sends data such as "current brightness is 500 lumens, set brightness is 400 lumens."

[0437] Step 2:

[0438] Obtaining environmental information

[0439] The server obtains external environmental information (temperature, humidity, weather information, etc.) from various APIs. Input data includes weather data and data from environmental monitors, which are collected and used for analysis. Specifically, it obtains information such as "The current outside temperature is 28°C and humidity is 70%" from a weather data provider.

[0440] Step 3:

[0441] Collecting Emotional Data

[0442] The device collects the user's facial expressions and tone of voice through a camera and microphone, and converts them into emotional data using an emotion recognition model (using, for example, TensorFlow and Keras). The input data is video and audio data, and the output is the user's emotional state (relaxed, stressed, etc.). Specifically, the device captures the user's facial expressions with a camera inside the store and determines that the user is "relaxed."

[0443] Step 4:

[0444] Data Preprocessing

[0445] The server integrates the collected operational status data, environmental information, and emotion data, and performs preprocessing such as data cleansing, missing value completion, and data format unification. The input data is the data collected in steps 1 to 3, and the output is a clean, unified, and analyzable dataset.

[0446] Step 5:

[0447] Analysis and Optimization

[0448] The server analyzes the preprocessed data using machine learning algorithms (e.g., random forest, regression, etc.) to calculate the optimal device operating conditions. The input data is the preprocessed data, and the output is specific operating instructions (e.g., adjust the lighting brightness to 420 lumens).

[0449] Step 6:

[0450] Generating and sending control instructions

[0451] The server generates control instructions based on the analysis results and sends them to various devices. The input data are optimized operation instructions, and the output is specific operation instructions for each device. Specifically, an instruction such as "change the lighting brightness to 420 lumens" is generated and sent to the lighting system.

[0452] Step 7:

[0453] Providing a user interface

[0454] The server provides an interface for displaying the analysis results and control instructions to the user. The input data are the analysis results and control instructions, and the output is a dashboard and notifications that the user can access. The user can check the current device status and settings from their smartphone or PC and manually adjust them as necessary. Specifically, the user receives a notification on their smartphone that "the lighting setting brightness has been changed to 420 lumens."

[0455] Step 8:

[0456] Big Data Analysis

[0457] The server collects daily data and stores the analysis results over the long term, and performs big data analysis. The input data is data collected over a long period of time, and the output is the analysis results on trends and patterns. This data is used to create future prediction models that are useful for proposing energy efficiency improvements. Specifically, it analyzes energy consumption data by season and proposes more efficient operating methods.

[0458] Step 9:

[0459] Adjusting Operation Settings

[0460] The server adjusts the device's operational settings based on the user's emotional data. The input data is the result of emotion recognition, and the output is the emotion-based operational settings. Specifically, if the user is feeling stressed, the server adjusts the lighting color temperature to 3000K (warm color) to create a relaxing environment.

[0461] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0462] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0463] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0464] [Second embodiment]

[0465] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0466] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0467] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0468] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0469] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0470] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0471] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0472] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0473] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0474] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0475] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0476] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0477] This invention relates to an "IoT AI Commander" system that works in conjunction with multiple IoT devices to achieve optimal operation and improve energy efficiency. This system is primarily composed of the following modules: a data collection module, a data analysis module, a control module, a user interface module, and a big data analysis module.

[0478] Explanation of the specific processing of the system

[0479] 1. Data Collection Module

[0480] The server periodically collects operational status data from each IoT device, including temperature settings, power consumption, and operating time. Environmental data from external temperature sensors and weather forecast APIs is also collected at the same time.

[0481] The terminal periodically transmits the operating status of each IoT device to the server.

[0482] Example: The server receives data from the air conditioner, such as "The current room temperature is 25°C, and the set temperature is 22°C." At the same time, it obtains data from the weather forecast API, such as "The current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C."

[0483] 2. Data Analysis Module

[0484] The server analyzes the collected data using machine learning algorithms to calculate the optimal operating conditions for the devices. In this process, it learns the efficient operating patterns of each IoT device and derives the settings that are best suited to the user's lifestyle and environmental conditions.

[0485] Example: The server performs an analysis and concludes that "it would be best to change the air conditioner's temperature setting to 23°C."

[0486] 3. Control Module

[0487] The server generates control instructions for each IoT device based on the analysis results. These control instructions are designed to reflect the optimal operating settings for each device.

[0488] The terminal receives control instructions from the server and changes the device settings accordingly.

[0489] Example: The server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends this instruction to the air conditioner. The air conditioner changes the set temperature to 23°C.

[0490] 4. User Interface Module

[0491] The server generates a dashboard that displays the analysis results and control instructions in an easy-to-understand format for users, and users can access this dashboard via their smartphones or PCs.

[0492] The dashboard allows users to view current device status and control settings and change them if necessary.

[0493] Example: A user receives a notification on their smartphone that says, "The air conditioner temperature setting has been changed to 23°C" and confirms it.

[0494] 5. Big Data Analysis Module

[0495] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[0496] Example: A server accumulates energy consumption data for the home on a yearly basis, analyzes seasonal usage patterns, and suggests that "it would be more energy efficient to not lower the set temperature any further in winter."

[0497] Example

[0498] For example, when a user leaves the house during the day in summer, the system analyzes the outside temperature, the temperature inside the house, and the weather forecast to automatically adjust the air conditioner's temperature setting. Even if the outside temperature is high and the house is predicted to heat up quickly, the system will maintain the optimal setting and maximize energy efficiency by returning the room to an appropriate temperature before the user returns. Long-term data analysis also identifies energy consumption trends in specific regions and seasons, allowing for predictive optimization.

[0499] In this way, the IoT AI Commander improves user convenience and reduces energy waste, ultimately reducing the environmental impact and helping to create a sustainable living environment.

[0500] The processing flow will be explained below.

[0501] Step 1:

[0502] The server periodically sends requests to collect operational status data from each IoT device, which triggers each IoT device to send its current operating status and configuration information to the server.

[0503] Step 2:

[0504] In response to the received request, the terminal sends the operating status data of each IoT device (temperature setting, power consumption, operating time, etc.) to the server.

[0505] Example: An air conditioner sends data to the server saying, "The current room temperature is 25°C, and the set temperature is 22°C."

[0506] Step 3:

[0507] The server accesses external environmental sensors and weather forecast APIs to obtain environmental data such as outside temperature, humidity, and weather forecasts.

[0508] Example: The server retrieves data from a weather forecast API: "The current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C."

[0509] Step 4:

[0510] The server preprocesses the collected operational status data and environmental data, which includes data cleanup, filling in errors and missing values, and standardizing the format.

[0511] Step 5:

[0512] The server uses machine learning algorithms to perform analysis based on the preprocessed data, which then calculates the optimal operating conditions for each IoT device.

[0513] Example: As a result of the analysis, the conclusion is reached that "it is optimal to change the air conditioner's set temperature to 23°C."

[0514] Step 6:

[0515] The server generates specific control instructions based on the analysis results, which are designed to reflect optimal operating settings for each IoT device.

[0516] Example: The server generates the instruction "Change the air conditioner temperature setting to 23°C."

[0517] Step 7:

[0518] The server then sends the generated control instructions to each IoT device, which then changes their operating settings in accordance with the control instructions.

[0519] Step 8:

[0520] The terminal executes the control instructions received from the server and changes the settings of the corresponding IoT device.

[0521] Example: An air conditioner receives an instruction to change the set temperature to 23°C.

[0522] Step 9:

[0523] The server generates a dashboard to display the analysis results and control instructions to the user, who can access the dashboard via a smartphone or PC.

[0524] Step 10:

[0525] The dashboard allows users to view current device status and control settings and change them if necessary.

[0526] Example: A user sees a notification on their smartphone that says, "The air conditioner temperature setting has been changed to 23°C."

[0527] Step 11:

[0528] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[0529] Step 12:

[0530] Based on the results of big data analysis, the server makes new predictions and optimizations, providing even more accurate control instructions.

[0531] Example: Based on winter usage data, we suggest that "it would be more energy efficient to not lower the set temperature any further."

[0532] Example 1

[0533] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0534] In systems using conventional IoT devices, efficient operation of each device and improvement of energy efficiency have not been sufficiently achieved. Furthermore, in many systems, it is difficult to effectively analyze collected data and derive optimal operating conditions. This poses a challenge in optimizing energy consumption in homes and businesses and providing a sustainable living environment.

[0535] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0536] In this invention, the server includes: means for collecting operational status data from various IoT devices; means for acquiring external environmental data; means for analyzing the collected data using a machine learning algorithm to calculate optimal device operating conditions; means for transmitting control instructions to various IoT devices based on the analysis results; means for providing a user interface for displaying the analysis results and the control instructions to the user; means for analyzing data accumulated over a long period of time on a large scale and identifying trends and patterns; means including a terminal for transmitting data collected from each IoT device to the server; and means for the server to generate and transmit control instructions to each IoT device. This enables integrated analysis of various data and rapid determination of optimal operating conditions. This significantly improves energy efficiency and realizes a sustainable living environment.

[0537] An "IoT device" is an electronic device that can connect to the Internet and collect, send, and receive data.

[0538] "Operational status data" refers to information about the operating status and behavior of IoT devices, including data such as temperature settings, power consumption, and operating time.

[0539] "Environmental data" refers to information about the state of the environment in which an IoT device is located, including, for example, outside temperature, humidity, and weather information.

[0540] A "machine learning algorithm" refers to a mathematical method or model that analyzes large amounts of data to learn patterns and rules and make predictions and classifications.

[0541] "Control instructions" refer to commands and configuration change instructions sent from a server to an IoT device, and are used to optimize the device's operation.

[0542] "User interface" refers to the software and hardware components that provide the visual and operational means for a user to interact with a system.

[0543] "Trends and patterns" refer to regularities and tendencies extracted from data collected over a long period of time, and are used to predict and optimize the future.

[0544] "Terminal" refers to an electronic device that functions as a relay point for transmitting data collected from IoT devices to a server.

[0545] "Server" refers to a central management device that collects and analyzes data, and generates and sends control instructions.

[0546] This invention provides an "IoT AI system" that works in conjunction with multiple IoT devices to achieve optimal operation and improve energy efficiency. This system is primarily composed of the following modules: a data collection module, a data analysis module, a control module, a user interface module, and a big data analysis module.

[0547] First, let's explain the data collection module. The server periodically collects operating status data from each IoT device. This data includes temperature settings, power usage, and operating hours. Environmental data is also simultaneously collected through outdoor temperature sensors and weather forecast APIs. The terminals send the operating status data collected from the IoT devices to the server. As a concrete example, the server receives data from an air conditioner that "the current room temperature is 25°C, the set temperature is 22°C," and obtains data from the weather forecast API that "the current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C."

[0548] Next, we will explain the data analysis module. The server uses the collected data to run machine learning algorithms and calculate the optimal device operating conditions. In this process, the server learns the efficient operating patterns of each IoT device and derives the settings that are most suitable for the user's lifestyle and environmental conditions. As a concrete example, the server concludes that "it is optimal to change the air conditioner's temperature setting to 23°C."

[0549] Next, we will explain the control module. The server generates and sends control instructions to each IoT device based on the analysis results. The terminal receives the control instructions from the server and changes the device settings accordingly. As a concrete example, the server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends this instruction to the air conditioner. The air conditioner changes the set temperature to 23°C.

[0550] Next, we will explain the user interface module. The server generates a dashboard to display the analysis results and control instructions in an easy-to-understand manner to the user. This dashboard can be accessed by the user via a smartphone or PC. The user can use the dashboard to check the current device status and control settings and change them as necessary. As a concrete example, the user receives a notification on their smartphone saying, "The air conditioner's set temperature has been changed to 23°C," and confirms this.

[0551] Finally, we will explain the big data analysis module. The server collects daily data and accumulates the analysis results over the long term, saving them as big data. This big data is used to further analyze trends and patterns and create future prediction models. As a concrete example, the server accumulates household energy consumption data on an annual basis, analyzes seasonal usage patterns, and suggests that "it would be more energy-efficient to not lower the temperature setting any further in winter."

[0552] For example, when a user goes out during the day in summer, the system analyzes the outside temperature, the temperature inside the house, and the weather forecast, and automatically adjusts the air conditioner's temperature setting. Even if the outside temperature is high and the house is predicted to heat up quickly, the system maintains the optimal setting and maximizes energy efficiency by returning the room to an appropriate temperature before the user returns home. Long-term data analysis also identifies energy consumption trends in specific regions and seasons, and performs optimization based on predictions. In this way, the IoT AI system improves user convenience and reduces energy waste. As a result, it is possible to reduce environmental impact and aim to create a sustainable living environment.

[0553] An example of a prompt for a generative AI model is, "Please explain what specific processing the IoT AI system performs at each step. As an example, please show a process that includes how to optimize the air conditioner's temperature setting."

[0554] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0555] Step 1:

[0556] Starting Data Collection

[0557] The server periodically sends data collection requests to each IoT device. These requests include instructions for operational status data such as temperature setting, power usage, and operating time. The terminal receives these requests and collects the necessary data from each IoT device. As a specific example, the terminal obtains data from an air conditioner that "the current room temperature is 25°C, and the set temperature is 22°C." The input is a data collection request from the server, and the output is operational status data from each IoT device.

[0558] Step 2:

[0559] Obtaining environmental data

[0560] The server obtains environmental data through an external temperature sensor or weather forecast API. Specifically, it obtains the current outside temperature and weather information. For example, the server uses a weather forecast API to obtain data such as "The current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C." The input is a request from an external sensor or API, and the output is environmental data.

[0561] Step 3:

[0562] Data transmission

[0563] The device sends the collected operational status data to the server. The server stores the environmental data obtained from the weather forecast API. Specifically, the device sends data to the server such as "current room temperature is 25°C, set temperature is 22°C." The input is the operational status data from the device, and the output is the operational status data and environmental data stored on the server.

[0564] Step 4:

[0565] Data integration

[0566] The server integrates the operational status data and environmental data from each IoT device. The input is the operational status data and environmental data, and the output is an integrated data set.

[0567] Step 5:

[0568] Running machine learning algorithms

[0569] The server uses the integrated data to run a machine learning algorithm and calculate the optimal device operating conditions. This process involves data preprocessing, feature extraction, and model training. For example, the server may conclude that "setting the air conditioner temperature to 23°C is optimal." The input is the integrated data set, and the output is the optimal operating conditions.

[0570] Step 6:

[0571] Control instruction generation

[0572] The server generates control instructions for each IoT device based on the analysis results. Specifically, the server generates an instruction to "change the air conditioner's set temperature to 23°C." The input is the optimal operating conditions, and the output is the generated control instruction.

[0573] Step 7:

[0574] Sending control instructions

[0575] The server sends the generated control instructions to each IoT device. The terminal receives the control instructions from the server and changes the device settings accordingly. Specifically, the server sends the instruction "change the air conditioner's set temperature to 23°C," and the air conditioner changes its set temperature. The input is the generated control instruction, and the output is the setting change of the IoT device that received the control instruction.

[0576] Step 8:

[0577] Generate a dashboard

[0578] The server generates a dashboard to display the analysis results and control instructions in an easy-to-understand manner for the user. The inputs are the analysis results and control instructions, and the output is the generated dashboard.

[0579] Step 9:

[0580] Accessing the User Interface

[0581] Users access the dashboard using a smartphone or PC to check the current device status and control settings. Specifically, the user receives a notification on their smartphone saying, "The air conditioner temperature setting has been changed to 23°C," and confirms it. The input is the dashboard URL and app, and the output is status information that the user can check.

[0582] Step 10:

[0583] Accumulation and analysis of big data

[0584] The server collects daily data and accumulates the analysis results over the long term, saving them as big data. Data accumulated over a long period of time is analyzed on a large scale to identify trends and patterns. Specifically, the server accumulates household energy consumption data on an annual basis and suggests, "It would be more energy-efficient to not lower the temperature setting any further in winter." The input is the daily collected data and analysis results, and the output is the accumulated big data and its analysis results.

[0585] (Application example 1)

[0586] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0587] Modern self-driving vehicles operate using numerous information and communication technology devices, but it is difficult to efficiently manage the operating status of these devices and external environmental data, and calculate optimal operating conditions in real time. Furthermore, systems that automatically set optimal routes and speeds taking into account traffic information and weather conditions have not yet been fully realized. As a result, there are problems with wasted energy consumption and reduced operating efficiency.

[0588] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0589] In this invention, the server includes means for collecting operation status data from various information and communication technology devices, means for acquiring external environment data, means for analyzing the collected data using a machine learning algorithm and calculating optimal device operating conditions, means for sending control instructions to the various information and communication technology devices based on the analysis results, means for providing a user interface for displaying the analysis results and the content of the control instructions to the user, means for analyzing data accumulated over a long period of time on a large scale and identifying trends and patterns, means for collecting and analyzing vehicle status and traffic information to calculate optimal driving routes and speed settings, and means for sending control instructions to the autonomously driven vehicle based on the analysis results, thereby optimizing the operating conditions of the autonomously driven vehicle and enabling improved energy efficiency and operating efficiency.

[0590] "Information and communication technology equipment" means electronic devices for data collection, communication, control, etc.

[0591] "Operation status data" refers to information relating to the operating status of the device, such as its usage status, power consumption, temperature, and usage time.

[0592] "External environment data" is information about surrounding conditions such as weather, temperature, humidity, and traffic conditions.

[0593] A "machine learning algorithm" is a mathematical method for learning patterns and relationships from data and making predictions and classifications.

[0594] A "control instruction" is an instruction to change or adjust the operation of a device.

[0595] A "user interface" is the screen or method of operation that allows a user to interact with a system.

[0596] "Trends and patterns" are tendencies and regularities that emerge from data collected over a long period of time.

[0597] An "autonomous vehicle" is a vehicle that operates autonomously without the need for human operation.

[0598] A "travel route" is a specific route that a vehicle will take to reach its destination.

[0599] "Speed ​​setting" is a parameter that determines the vehicle's traveling speed.

[0600] This invention relates to an "Autonomous Vehicle AI Optimization Commander" system that works in conjunction with multiple information and communication technology devices to optimize operation and improve energy efficiency. This enables vehicle operation to be optimized and energy efficiency to be improved. The system is primarily composed of the following modules: a data collection module, a data analysis module, a control module, a user interface module, and a big data analysis module.

[0601] Explanation of the specific processing of the system

[0602] 1. Data Collection Module

[0603] The server periodically collects operational status data from the autonomous vehicle, including remaining battery power, driving speed, distance to destination, etc. It also simultaneously obtains environmental data from traffic information APIs and weather forecast APIs.

[0604] The terminal transmits operation-related data, including traffic and weather information, to the server.

[0605] Example: A server receives data from an autonomous vehicle, such as "Current battery remaining is 80%, driving speed is 60km / h, distance to destination is 20km." At the same time, it obtains data from a traffic information API that "Congestion is occurring" and from a weather forecast API that "Current weather is rain."

[0606] 2. Data Analysis Module

[0607] The server analyzes the collected data using machine learning algorithms to calculate the optimal vehicle route and speed. In this process, the server learns the optimal driving pattern based on data obtained from various information and communication technology devices, and derives the settings that are most suitable for real-time environmental conditions.

[0608] Example: The server performs an analysis and concludes, "Change driving speed to 55 km / h and re-route to avoid traffic jams."

[0609] 3. Control Module

[0610] The server generates control instructions for the autonomous vehicle based on the analysis results, which reflect the optimal driving route and speed settings.

[0611] The terminal receives control instructions from the server and changes the settings of the autonomous vehicle accordingly.

[0612] Example: The server generates an instruction to "change driving speed to 55 km / h and re-plan the route" and sends this instruction to an autonomous vehicle. The autonomous vehicle changes its driving speed to 55 km / h and re-plans the route.

[0613] 4. User Interface Module

[0614] The server generates a dashboard that displays the analysis results and control instructions in an easy-to-understand format for users, and users can access this dashboard via their smartphones or PCs.

[0615] The dashboard allows users to view the current vehicle status and control settings and change them if necessary.

[0616] Example: A user receives a notification on their smartphone that says "Your driving speed has been changed to 55km / h" and confirms it.

[0617] 5. Big Data Analysis Module

[0618] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[0619] Example: A server accumulates energy consumption data of autonomous vehicles on a yearly basis, analyzes seasonal usage patterns, and suggests that "it would be more energy-efficient to not reduce the speed any further in winter."

[0620] Prompt Sentence Examples

[0621] As a concrete example, the following prompt sentences can be input to a generative AI model:

[0622] "Vehicle ID 12345's current speed is 60 km / h, battery remaining is 80%, distance to destination is 20 km, there is traffic congestion information, and it is raining. Please predict the optimal driving speed under these conditions."

[0623] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0624] Step 1:

[0625] Data collection

[0626] The server collects operational status data from the autonomous vehicles (remaining battery level, driving speed, distance to destination, etc.) and simultaneously obtains environmental data (traffic conditions, weather information) from traffic information APIs and weather forecast APIs.

[0627] Input: Data from autonomous vehicles, traffic information API, weather forecast API

[0628] Output: Collected data (remaining battery level, driving speed, distance to destination, traffic conditions, weather information)

[0629] Specific operation: The server periodically connects to the autonomous vehicle to obtain operating status data, and at the same time calls the API to obtain external environment data.

[0630] Step 2:

[0631] Data analysis

[0632] The server then inputs the collected data into a machine learning algorithm to calculate the optimal route and speed, taking into account the vehicle's remaining battery power, current speed, distance to destination, traffic conditions, weather information, and more.

[0633] Input: Collected data (battery level, driving speed, distance to destination, traffic conditions, weather information)

[0634] Output: Optimal driving path and speed settings

[0635] Specific operation: The server uses a machine learning model (e.g., a Linear Regression model) to analyze the input data and calculate the optimal driving route and speed.

[0636] Step 3:

[0637] Control instruction generation

[0638] The server generates control instructions for the autonomous vehicle based on the results of the data analysis, including instructions on optimal driving routes and speed settings.

[0639] Input: Data analysis results (optimal driving route and speed settings)

[0640] Output: Control instructions for the autonomous vehicle

[0641] Specific operation: The server executes a control algorithm based on the analysis results and generates control instructions for the autonomous vehicle.

[0642] Step 4:

[0643] Sending control instructions

[0644] The server then sends the generated control instructions to the autonomous vehicle, which then adjusts its route and speed based on these instructions.

[0645] Input: Generated control instructions

[0646] Output: The behavior of the autonomous vehicle based on the executed control instructions.

[0647] Specific operation: The server sends instructions to the autonomous vehicle to adjust its speed and route, and the autonomous vehicle carries out the instructions.

[0648] Step 5:

[0649] Display in the user interface

[0650] The server generates a user interface to display the analysis results and control instructions to the user, who can then check them on their smartphone or PC.

[0651] Input: Analysis results, control instructions

[0652] Output: What is displayed on the user interface

[0653] Specific operation: The server generates a dashboard and sends information to a smartphone or PC to display the analysis results and control instructions.

[0654] Step 6:

[0655] Big data accumulation and analysis

[0656] The server collects daily data and accumulates the analysis results over the long term, saving them as big data. This big data can then be used to analyze more detailed trends and patterns.

[0657] Input: Daily data collection results, analysis results

[0658] Output: Long-term trends and patterns

[0659] Specific operation: The server periodically analyzes the accumulated data and updates the future prediction model.

[0660] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0661] This invention relates to an "IoT AI Commander" system that works in conjunction with multiple IoT devices to achieve optimal operation and improve energy efficiency. This system aims to provide a more user-friendly and comfortable environment by combining it with an emotion engine that recognizes the user's emotions. The specific processing of the system is described below.

[0662] Explanation of the specific processing of the system

[0663] 1. Data Collection Module

[0664] The server periodically sends requests to collect operational status data from each IoT device, including the device's operating status, configuration information, power consumption, and operating time.

[0665] The terminal periodically sends operational status data of the IoT device to the server.

[0666] Example: An air conditioner sends data to the server saying, "The current room temperature is 25°C, and the set temperature is 22°C."

[0667] 2. Emotion Recognition by Emotion Engine

[0668] The server recognizes emotions by analyzing the user's facial expressions and tone of voice, using cameras and microphones to collect data in real time.

[0669] Example: If a user is in their living room and has a relaxed expression, the emotion engine will recognize that expression and determine that they are "relaxed."

[0670] 3. Data integration and preprocessing

[0671] The server integrates the operational status data of IoT devices, environmental data (outside temperature, weather forecast, etc.), and emotion data, and performs preprocessing, which includes data cleansing, missing value completion, and format standardization.

[0672] 4. Analysis using machine learning algorithms

[0673] The server then uses machine learning algorithms to analyze the preprocessed data, with the goal of calculating the optimal operating conditions for each IoT device.

[0674] Example: As a result of the analysis, the conclusion is reached that "it is optimal to change the air conditioner's set temperature to 23°C."

[0675] 5. Integrated analysis with emotion data

[0676] The server further adjusts the device's operating conditions based on the user's emotional data obtained by the emotion engine. If the user is feeling stressed, the server may make additional adjustments, such as changing the lighting to a warmer color and playing music.

[0677] Example: If the user is determined to be stressed, not only will the air conditioner temperature be changed to 23°C, but the color temperature of the lighting will be changed to 3000K (warm color) to create a relaxing environment.

[0678] 6. Generating and Sending Control Instructions

[0679] The server generates the necessary control instructions based on the analysis results and sends them to each IoT device.

[0680] The terminal receives control instructions from the server and changes the device settings accordingly.

[0681] Example: The server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends it to the air conditioner. The air conditioner changes the set temperature to 23°C.

[0682] 7. User Interface

[0683] The server generates a dashboard to display the analysis results and control instructions to users. Users can access the dashboard on their smartphones or PCs to check and change the current device status and control settings.

[0684] Users can monitor the operation of IoT devices based on the information displayed on the dashboard and manually change settings as needed.

[0685] Example: A user sees a notification on their smartphone that says "The air conditioner temperature setting has been changed to 23°C" and manually adjusts the temperature setting further if necessary.

[0686] 8. Big Data Analysis

[0687] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[0688] Example: Analyzing energy consumption data for each season (summer and winter) and proposing more efficient operation methods.

[0689] In this way, the IoT AI Commander improves user convenience and reduces energy waste while taking emotions into consideration, ultimately reducing the environmental impact and helping to create a sustainable living environment.

[0690] The processing flow will be explained below.

[0691] Step 1:

[0692] The server periodically sends requests to collect operational status data from each IoT device, prompting each device to send its current operating status, configuration information, power consumption, operating time, and other information.

[0693] Step 2:

[0694] The terminal receives a request from the server and transmits the operational status data of each IoT device to the server.

[0695] Example: An air conditioner sends information to the server that "the current room temperature is 25°C, and the set temperature is 22°C."

[0696] Step 3:

[0697] The server accesses external environmental data sources (e.g., weather forecast APIs or external sensors) to obtain environmental data such as outside temperature, humidity, and weather forecasts.

[0698] Example: The server retrieves data from a weather forecast API: "The current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C."

[0699] Step 4:

[0700] The server runs an emotion engine to obtain emotion data by analyzing the user's facial expressions and voice. Data is collected in real time via cameras and microphones.

[0701] Example: A camera and microphone analyze a user's facial expressions and tone of voice to determine whether they are relaxed or stressed.

[0702] Step 5:

[0703] The server integrates the collected operational status data, environmental data, and emotion data and performs preprocessing, which cleanses the data, fills in missing values, and standardizes the format.

[0704] Step 6:

[0705] The server uses machine learning algorithms to analyze the preprocessed data and calculate the optimal operating conditions for each IoT device.

[0706] Example: After analysis, you conclude that the optimal temperature setting for an air conditioner is 23°C.

[0707] Step 7:

[0708] The server optimizes the analysis results by integrating the emotional data, generating different control instructions depending on whether the user is relaxed or stressed.

[0709] Example: If the user is determined to be stressed, the system will instruct the user to change the air conditioner temperature setting to 23°C, change the lighting color temperature to warmer colors, and play relaxing music.

[0710] Step 8:

[0711] The server then sends the generated control instructions to each IoT device, which then changes their behavior in accordance with the new settings.

[0712] Step 9:

[0713] The terminal receives control instructions from the server and changes the settings of the corresponding IoT device.

[0714] Example: The air conditioner changes the set temperature to 23°C, the smart lighting changes the color temperature to 3000K (warm), and the music player starts playing relaxing music.

[0715] Step 10:

[0716] The server generates a dashboard to display the analysis results and control instructions to the user, which can be viewed on a smartphone or PC.

[0717] Step 11:

[0718] Users can use the dashboard to view current device status and control settings, and manually change settings if necessary.

[0719] Example: A user sees a notification on their smartphone that says "The air conditioner temperature setting has been changed to 23°C" and then changes the temperature setting to 25°C.

[0720] Step 12:

[0721] The server accumulates daily data over the long term and stores it as big data. This data is used to analyze trends and patterns and create predictive models for the future.

[0722] Example: Analyzing past data, understanding seasonal energy consumption patterns, and proposing optimal energy consumption plans.

[0723] The above is the specific processing flow of the "IoT AI Commander" system, which combines user emotional data. This system improves user convenience and comfort while reducing energy waste and realizing a sustainable living environment.

[0724] Example 2

[0725] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0726] Modern IoT devices collect large amounts of data, improving user convenience. However, analyzing the data and calculating optimal device operating conditions requires advanced technology. Furthermore, existing systems do not adequately consider user emotions and comfort, creating challenges in improving the user experience. Furthermore, there is a need for more methods to utilize collected data over the long term to optimize energy efficiency and realize sustainable living environments.

[0727] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0728] In this invention, the server includes means for collecting operation status data from various IoT devices, means for acquiring external environmental data, means for analyzing the collected data using a machine learning algorithm and calculating optimal device operating conditions, means for sending control instructions to the various IoT devices based on the analysis results, means for providing a user interface for displaying the analysis results and the control instructions to the user, means for analyzing data accumulated over a long period of time on a large scale and identifying trends and patterns, means for analyzing the user's facial expressions and tone of voice to collect emotion data, and means for adjusting the IoT device operating conditions based on the emotion data. This makes it possible to calculate optimal device operating conditions taking user emotions into consideration, improve energy efficiency, and realize a sustainable living environment.

[0729] "IoT devices" is a general term for devices that are connected to the Internet and can collect and control data by communicating with each other.

[0730] "Operational status data" refers to information about the operation of an IoT device, such as its operating status, settings, power consumption, and operating time.

[0731] "Environmental data" refers to information about the external environment, such as outside temperature, humidity, and weather forecasts.

[0732] "Machine learning algorithms" refers to technologies that include mathematical models and statistical methods for analyzing collected data and calculating optimal equipment operating conditions.

[0733] "Control instructions" refer to specific operational commands sent to IoT devices to achieve optimal device operating conditions.

[0734] "User interface" refers to the display screen and operating means that allow a user to interact with the system and check or change the system's status.

[0735] "Large-scale analysis" refers to techniques and methods that use large amounts of data accumulated over a long period of time to identify trends and patterns.

[0736] "Emotion data" refers to information that indicates the user's emotional state, obtained from the user's facial expression, tone of voice, etc.

[0737] "Trend" or "pattern" refers to a statistical tendency or recurring characteristic derived from collected data.

[0738] "Operating conditions" refers to the settings and operating modes required for optimal operation of IoT devices.

[0739] A "sustainable living environment" refers to a sustainable living space that improves energy efficiency, reduces waste, and reduces the environmental impact.

[0740] This invention relates to an "IoT AI Commander" system that aims to achieve optimal operation and improved energy efficiency by linking multiple IoT devices. This system aims to provide a more user-friendly and comfortable environment by combining it with an emotion engine that recognizes the user's emotions.

[0741] The system's main components are a server, a device, and a user. The server collects operational status data from various IoT devices and acquires external environmental data, then analyzes all collected data using a machine learning algorithm. Based on the analysis results, it sends control instructions to the IoT devices and displays them to the user through a user interface. In addition, an emotion engine analyzes the user's facial expressions and tone of voice, collects emotional data, and adjusts the operating conditions of the IoT devices. This increases user comfort and improves energy efficiency.

[0742] Hardware and Software Examples

[0743] Servers: High-performance API servers, database management systems (e.g., MySQL), machine learning frameworks (e.g., TensorFlow)

[0744] Terminal: IoT device (e.g. smart speaker, smart air conditioner, smart light)

[0745] Emotion engine: Image and speech recognition technologies (e.g., OpenCV, Google Speech-to-Text API)

[0746] Explanation of program processing

[0747] Data collection

[0748] The server periodically collects operational status data from various IoT devices. Specifically, it sends API requests and receives data such as operating status, configuration information, and power consumption from the IoT devices. The terminal is responsible for sending operational status data to the server and providing the specific operating status of air conditioners and lighting.

[0749] Example: An air conditioner sends the following data to the server: "The current room temperature is 25°C, the set temperature is 22°C, and the power consumption is 1.2kW."

[0750] Example prompt: "Please report your current air conditioning settings and operating status."

[0751] Collecting Emotional Data

[0752] The server analyzes the user's facial expressions and tone of voice in real time through a camera and microphone. The emotion engine then sends the collected emotional data to the server.

[0753] Example: The server analyzes video data from a camera to detect whether the user is laughing, and also determines from the tone of voice that the user is relaxed.

[0754] Example prompt: "Analyze the user's facial expression and tone of voice and report their current emotions."

[0755] Data preprocessing and integrated analysis

[0756] The server integrates the collected IoT device data and emotion data, cleansing the data, filling in missing values, and standardizing the format. It then analyzes the data using machine learning algorithms to calculate the optimal operating conditions for the IoT device. The analysis results and emotion data are integrated to generate the final control instructions.

[0757] Example: A machine learning model calculates the optimal temperature setting for an air conditioner as 23°C. If the user is relaxing, change the color temperature of the lights to a warmer color and play music.

[0758] Example prompt: "Analyze the data and calculate the optimal temperature setting for your air conditioner."

[0759] Generating and sending control instructions

[0760] The server generates specific control instructions based on the analysis results and sends them to the IoT device. The device receives the control instructions from the server and changes the device settings accordingly.

[0761] Example: The server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends it to the air conditioner. The air conditioner changes the set temperature to 23°C.

[0762] Example prompt: "Please tell the air conditioner to change the temperature setting to 23°C."

[0763] Providing a user interface

[0764] The server generates a dashboard to display the analysis results and control instructions to the user, providing a user interface. Users can access the dashboard on their smartphones or PCs to check and change the current device status and control settings.

[0765] Example: A user sees a notification on their smartphone that says "The air conditioner temperature setting has been changed to 23°C" and manually adjusts the temperature setting further if necessary.

[0766] Example prompt: "Show me the current air conditioner temperature setting and operating status."

[0767] Big Data Analysis

[0768] The server collects daily data and accumulates the analysis results over the long term, saving them as big data. This big data is used to analyze trends and patterns and create future prediction models.

[0769] Example: A server analyzes historical energy consumption data and proposes new algorithms to improve energy efficiency.

[0770] Example prompt: "Based on historical energy consumption data, create a proposal to improve the operational efficiency of your air conditioner."

[0771] Through the above process, this system optimizes energy usage while maintaining user comfort, providing a sustainable living environment.

[0772] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0773] Step 1: Collecting IoT device data

[0774] The server sends a request to each IoT device to collect operational status data.

[0775] Input: Data collection request sent from the server to the IoT device.

[0776] Output: Operational status data sent from IoT devices (e.g., room temperature, set temperature, power consumption).

[0777] Specific operation: The server sends a request to the air conditioner, and the air conditioner replies with the data "Current room temperature is 25°C, set temperature is 22°C, power consumption is 1.2kW."

[0778] Step 2: Obtaining environmental data

[0779] The server obtains external environmental data (e.g., outside temperature, weather forecast).

[0780] Input: Real-time environmental data that the server retrieves from external data sources.

[0781] Output: The captured environmental data.

[0782] Specific operation: The server accesses the weather API to obtain the current outside temperature and weather forecast.

[0783] Step 3: Collecting emotion data

[0784] The server collects the user's facial expressions and tone of voice in real time through a camera and microphone.

[0785] Input: Video data from the camera and audio data from the microphone.

[0786] Output: Emotion data indicating the user's emotions.

[0787] How it works: The server analyzes the camera footage to detect whether the user is smiling and also determines from the tone of their voice that they are relaxed.

[0788] Step 4: Preprocessing the data

[0789] The server integrates and preprocesses data from IoT devices, environmental data, and emotion data.

[0790] Input: Operational status data from IoT devices, external environmental data, and user emotion data.

[0791] Output: Preprocessed integrated data.

[0792] Specific operation: The server fills in missing values ​​of each data with the average value and unifies the data format.

[0793] Step 5: Machine learning analysis

[0794] The server uses machine learning algorithms to perform analysis on the pre-processed data.

[0795] Input: Preprocessed integrated data.

[0796] Output: Optimal operating conditions for IoT devices.

[0797] What it does: A machine learning model calculates the optimal temperature setting for an air conditioner: 23°C. The algorithm then derives settings that minimize energy consumption and maximize user comfort.

[0798] Step 6: Integrated analysis with emotion data

[0799] The server readjusts the analysis results by taking into account the emotion data obtained from the emotion engine.

[0800] Input: Analysis results from machine learning algorithms, emotion data.

[0801] Output: Optimal operating conditions taking emotions into account.

[0802] Specific behavior: If the user is determined to be relaxed, the air conditioner temperature setting is changed to 23°C and the lighting color temperature is changed to 3000K (warm color).

[0803] Step 7: Generate and send control instructions

[0804] The server generates control instructions based on the analysis results and sends them to each IoT device.

[0805] Input: Optimal operating conditions taking emotions into account.

[0806] Output: Specific control instructions for each IoT device.

[0807] Specific operation: The server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends it to the air conditioner. The air conditioner changes the set temperature to 23°C.

[0808] Step 8: Update the User Interface

[0809] The server displays the analysis results and the contents of the control instructions on a dashboard and provides them to the user.

[0810] Input: Control instructions and analysis results.

[0811] Output: The dashboard content that is displayed to the user.

[0812] Specific behavior: The user sees a notification on their smartphone that says "The air conditioner temperature setting has been changed to 23°C" and adjusts it as needed.

[0813] Step 9: Analyze the big data

[0814] The server collects daily data and stores the analysis results over the long term, and performs large-scale analysis.

[0815] Input: Data accumulated over a long period of time.

[0816] Output: Analysis results that identify trends and patterns.

[0817] Specific operation: The server analyzes past energy consumption data and proposes new algorithms to improve energy efficiency.

[0818] Through these steps, the system optimizes energy consumption and realizes sustainable living while maintaining user comfort.

[0819] (Application example 2)

[0820] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0821] In existing IoT environments, it is difficult to set optimal device operation settings that take user emotions into account, which means that users cannot always enjoy comfort and convenience.In addition, the effective use of collected data and the optimization of long-term energy consumption are not sufficiently implemented, making it difficult to achieve a sustainable living environment.

[0822] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting operation status data from various sensing devices, means for acquiring external environmental information, means for analyzing the collected data and user emotion data using a machine learning algorithm and calculating optimal device operating conditions, means for sending control instructions to various devices based on the analysis results, means for providing a user interface for displaying the analysis results and the content of the control instructions to the user, means for analyzing data accumulated over a long period of time on a large scale and identifying trends and patterns, means for recognizing emotions by analyzing the user's facial expressions and tone of voice, and means for adjusting device operation settings based on the emotion data. This makes it possible to provide a comfortable environment that is in tune with the user's emotions, while optimizing energy consumption and realizing a sustainable living environment.

[0823] A "sensing device" is a device that detects information about the state of the environment or an object and collects that data.

[0824] "Environmental information" refers to data about the external environment, such as temperature, humidity, weather, and illuminance.

[0825] "Device" refers to various equipment that collects operational status data and is controlled in conjunction with external systems.

[0826] "Emotion data" is information about the user's psychological state obtained by analyzing the user's facial expressions and tone of voice.

[0827] A "machine learning algorithm" is a computational method for building patterns and predictive models based on collected data and calculating optimal operating conditions.

[0828] A "user interface" is a system that allows users to visually check and operate analysis results and control instructions.

[0829] "Trend" refers to a tendency or pattern of fluctuation that can be found by analyzing collected data over a long period of time.

[0830] "Data collection means" refers to the function for collecting necessary data from various sensing devices.

[0831] "External information acquisition means" refers to a function for acquiring data related to the external environment.

[0832] "Analysis means" refers to the function of analyzing collected data using machine learning algorithms to derive optimal operating conditions.

[0833] "Control instruction sending means" refers to a function for sending instructions to various devices based on the analysis results.

[0834] "Emotion recognition means" refers to a function for recognizing emotions by analyzing the user's facial expressions and tone of voice.

[0835] "Operation setting adjustment means" refers to a function for adjusting the operation settings of a device based on emotion data.

[0836] A system for implementing the present invention is configured as follows.

[0837] Specific system configuration

[0838] 1. Data Collection Module

[0839] The server periodically collects operational status data from various sensing devices, including the operating status, configuration information, power consumption, and operating time of each device.

[0840] The terminal periodically sends operational status data of the IoT device to the server.

[0841] Example: A lighting system sends data to a server saying "Current brightness is 500 lumens, set brightness is 400 lumens."

[0842] 2. Environmental Information Acquisition Module

[0843] The server acquires external environmental information (temperature, humidity, weather information, etc.).

[0844] Example: Obtain information from a weather data provider such as "The current outside temperature is 28°C and the humidity is 70%."

[0845] 3. Emotion Recognition Module

[0846] The server analyzes the user's facial expressions and tone of voice to recognize their emotions, using cameras and microphones to collect data in real time.

[0847] Example: A camera in a store captures the user's facial expression and determines that they are "relaxed."

[0848] 4. Analysis Module

[0849] The server uses machine learning algorithms to analyze the collected data and calculate the optimal operating conditions for the device.

[0850] Example: Analysis results in the conclusion that "adjusting the lighting brightness to 420 lumens is optimal."

[0851] 5. Control instruction transmission module

[0852] The server generates the necessary control instructions based on the analysis results and sends them to the various devices.

[0853] The terminal receives control instructions from the server and changes the device settings accordingly.

[0854] Example: The server generates an instruction to "change the lighting brightness to 420 lumens" and sends it to the lighting system. The lighting system changes the set brightness to 420 lumens.

[0855] 6. User Interface

[0856] The server provides an interface for displaying the analysis results and control instructions to the user. Users can access the interface on their smartphones or PCs to check and change the current device status and settings.

[0857] Based on the information displayed in the interface, the user can monitor the device's operation and manually change settings if necessary.

[0858] Example: A user sees a notification on their smartphone that says "Light setting brightness changed to 420 lumens" and manually adjusts the setting if necessary.

[0859] 7. Big Data Analysis Module

[0860] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[0861] Example: Analyzing seasonal energy consumption data and proposing more efficient operation methods.

[0862] 8. Operational Settings Adjustment Module

[0863] The server adjusts the device's operational settings based on the user's emotional data. If the user is feeling stressed, additional adjustments are made, such as changing the lighting to warmer colors and playing music.

[0864] Example: If the user is determined to be stressed, the color temperature of the lighting is changed to 3000K (warm color) to create a relaxing environment.

[0865] Specific examples

[0866] 1. While customers are browsing products in a store, smart glasses analyze their facial expressions and, if they are smiling, change the background music to a brighter tone.

[0867] 2. If customers seem stressed, reset the store's air conditioning to a comfortable temperature and change the lighting to a warmer color.

[0868] Prompt Sentence Examples

[0869] "Please provide a blueprint for an application that uses smart glasses and IoT devices to optimize the environment in a physical store according to the customer's emotions. As a specific example, please include a scenario in which cheerful music is played for smiling customers, and the air conditioning and lighting are adjusted for stressed customers."

[0870] The above is a detailed example of how the "IoT AI Commander" can be applied in a brick-and-mortar store. By using this system, it is possible to provide a comfortable environment that is sensitive to the user's emotions, while optimizing energy consumption and realizing a sustainable living environment.

[0871] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0872] Step 1:

[0873] Data collection

[0874] The server collects operational status data from various sensing devices. Input data includes the operating status, setting information, power consumption, and operating time of each device. This data is collected and stored in the server's database. Specifically, the lighting system sends data such as "current brightness is 500 lumens, set brightness is 400 lumens."

[0875] Step 2:

[0876] Obtaining environmental information

[0877] The server obtains external environmental information (temperature, humidity, weather information, etc.) from various APIs. Input data includes weather data and data from environmental monitors, which are collected and used for analysis. Specifically, it obtains information such as "The current outside temperature is 28°C and humidity is 70%" from a weather data provider.

[0878] Step 3:

[0879] Collecting Emotional Data

[0880] The device collects the user's facial expressions and tone of voice through a camera and microphone, and converts them into emotional data using an emotion recognition model (using, for example, TensorFlow and Keras). The input data is video and audio data, and the output is the user's emotional state (relaxed, stressed, etc.). Specifically, the device captures the user's facial expressions with a camera inside the store and determines that the user is "relaxed."

[0881] Step 4:

[0882] Data Preprocessing

[0883] The server integrates the collected operational status data, environmental information, and emotion data, and performs preprocessing such as data cleansing, missing value completion, and data format unification. The input data is the data collected in steps 1 to 3, and the output is a clean, unified, and analyzable dataset.

[0884] Step 5:

[0885] Analysis and Optimization

[0886] The server analyzes the preprocessed data using machine learning algorithms (e.g., random forest, regression, etc.) to calculate the optimal device operating conditions. The input data is the preprocessed data, and the output is specific operating instructions (e.g., adjust the lighting brightness to 420 lumens).

[0887] Step 6:

[0888] Generating and sending control instructions

[0889] The server generates control instructions based on the analysis results and sends them to various devices. The input data are optimized operation instructions, and the output is specific operation instructions for each device. Specifically, an instruction such as "change the lighting brightness to 420 lumens" is generated and sent to the lighting system.

[0890] Step 7:

[0891] Providing a user interface

[0892] The server provides an interface for displaying the analysis results and control instructions to the user. The input data are the analysis results and control instructions, and the output is a dashboard and notifications that the user can access. The user can check the current device status and settings from their smartphone or PC and manually adjust them as necessary. Specifically, the user receives a notification on their smartphone that "the lighting setting brightness has been changed to 420 lumens."

[0893] Step 8:

[0894] Big Data Analysis

[0895] The server collects daily data and stores the analysis results over the long term, and performs big data analysis. The input data is data collected over a long period of time, and the output is the analysis results on trends and patterns. This data is used to create future prediction models that are useful for proposing energy efficiency improvements. Specifically, it analyzes energy consumption data by season and proposes more efficient operating methods.

[0896] Step 9:

[0897] Adjusting Operation Settings

[0898] The server adjusts the device's operational settings based on the user's emotional data. The input data is the result of emotion recognition, and the output is the emotion-based operational settings. Specifically, if the user is feeling stressed, the server adjusts the lighting color temperature to 3000K (warm color) to create a relaxing environment.

[0899] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0900] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0901] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0902] [Third embodiment]

[0903] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0904] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0905] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0906] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0907] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0908] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0909] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0910] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0911] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0912] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0913] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0914] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0915] This invention relates to an "IoT AI Commander" system that works in conjunction with multiple IoT devices to achieve optimal operation and improve energy efficiency. This system is primarily composed of the following modules: a data collection module, a data analysis module, a control module, a user interface module, and a big data analysis module.

[0916] Explanation of the specific processing of the system

[0917] 1. Data Collection Module

[0918] The server periodically collects operational status data from each IoT device, including temperature settings, power consumption, and operating time. Environmental data from external temperature sensors and weather forecast APIs is also collected at the same time.

[0919] The terminal periodically transmits the operating status of each IoT device to the server.

[0920] Example: The server receives data from the air conditioner, such as "The current room temperature is 25°C, and the set temperature is 22°C." At the same time, it obtains data from the weather forecast API, such as "The current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C."

[0921] 2. Data Analysis Module

[0922] The server analyzes the collected data using machine learning algorithms to calculate the optimal operating conditions for the devices. In this process, it learns the efficient operating patterns of each IoT device and derives the settings that are best suited to the user's lifestyle and environmental conditions.

[0923] Example: The server performs an analysis and concludes that "it would be best to change the air conditioner's temperature setting to 23°C."

[0924] 3. Control Module

[0925] The server generates control instructions for each IoT device based on the analysis results. These control instructions are designed to reflect the optimal operating settings for each device.

[0926] The terminal receives control instructions from the server and changes the device settings accordingly.

[0927] Example: The server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends this instruction to the air conditioner. The air conditioner changes the set temperature to 23°C.

[0928] 4. User Interface Module

[0929] The server generates a dashboard that displays the analysis results and control instructions in an easy-to-understand format for users, and users can access this dashboard via their smartphones or PCs.

[0930] The dashboard allows users to view current device status and control settings and change them if necessary.

[0931] Example: A user receives a notification on their smartphone that says, "The air conditioner temperature setting has been changed to 23°C" and confirms it.

[0932] 5. Big Data Analysis Module

[0933] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[0934] Example: A server accumulates energy consumption data for the home on a yearly basis, analyzes seasonal usage patterns, and suggests that "it would be more energy efficient to not lower the set temperature any further in winter."

[0935] Example

[0936] For example, when a user leaves the house during the day in summer, the system analyzes the outside temperature, the temperature inside the house, and the weather forecast to automatically adjust the air conditioner's temperature setting. Even if the outside temperature is high and the house is predicted to heat up quickly, the system will maintain the optimal setting and maximize energy efficiency by returning the room to an appropriate temperature before the user returns. Long-term data analysis also identifies energy consumption trends in specific regions and seasons, allowing for predictive optimization.

[0937] In this way, the IoT AI Commander improves user convenience and reduces energy waste, ultimately reducing the environmental impact and helping to create a sustainable living environment.

[0938] The processing flow will be explained below.

[0939] Step 1:

[0940] The server periodically sends requests to collect operational status data from each IoT device, which triggers each IoT device to send its current operating status and configuration information to the server.

[0941] Step 2:

[0942] In response to the received request, the terminal sends the operating status data of each IoT device (temperature setting, power consumption, operating time, etc.) to the server.

[0943] Example: An air conditioner sends data to the server saying, "The current room temperature is 25°C, and the set temperature is 22°C."

[0944] Step 3:

[0945] The server accesses external environmental sensors and weather forecast APIs to obtain environmental data such as outside temperature, humidity, and weather forecasts.

[0946] Example: The server retrieves data from a weather forecast API: "The current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C."

[0947] Step 4:

[0948] The server preprocesses the collected operational status data and environmental data, which includes data cleanup, filling in errors and missing values, and standardizing the format.

[0949] Step 5:

[0950] The server uses machine learning algorithms to perform analysis based on the preprocessed data, which then calculates the optimal operating conditions for each IoT device.

[0951] Example: As a result of the analysis, the conclusion is reached that "it is optimal to change the air conditioner's set temperature to 23°C."

[0952] Step 6:

[0953] The server generates specific control instructions based on the analysis results, which are designed to reflect optimal operating settings for each IoT device.

[0954] Example: The server generates the instruction "Change the air conditioner temperature setting to 23°C."

[0955] Step 7:

[0956] The server then sends the generated control instructions to each IoT device, which then changes their operating settings in accordance with the control instructions.

[0957] Step 8:

[0958] The terminal executes the control instructions received from the server and changes the settings of the corresponding IoT device.

[0959] Example: An air conditioner receives an instruction to change the set temperature to 23°C.

[0960] Step 9:

[0961] The server generates a dashboard to display the analysis results and control instructions to the user, who can access the dashboard via a smartphone or PC.

[0962] Step 10:

[0963] The dashboard allows users to view current device status and control settings and change them if necessary.

[0964] Example: A user sees a notification on their smartphone that says, "The air conditioner temperature setting has been changed to 23°C."

[0965] Step 11:

[0966] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[0967] Step 12:

[0968] Based on the results of big data analysis, the server makes new predictions and optimizations, providing even more accurate control instructions.

[0969] Example: Based on winter usage data, we suggest that "it would be more energy efficient to not lower the set temperature any further."

[0970] Example 1

[0971] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0972] In systems using conventional IoT devices, efficient operation of each device and improvement of energy efficiency have not been sufficiently achieved. Furthermore, in many systems, it is difficult to effectively analyze collected data and derive optimal operating conditions. This poses a challenge in optimizing energy consumption in homes and businesses and providing a sustainable living environment.

[0973] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0974] In this invention, the server includes: means for collecting operational status data from various IoT devices; means for acquiring external environmental data; means for analyzing the collected data using a machine learning algorithm to calculate optimal device operating conditions; means for transmitting control instructions to various IoT devices based on the analysis results; means for providing a user interface for displaying the analysis results and the control instructions to the user; means for analyzing data accumulated over a long period of time on a large scale and identifying trends and patterns; means including a terminal for transmitting data collected from each IoT device to the server; and means for the server to generate and transmit control instructions to each IoT device. This enables integrated analysis of various data and rapid determination of optimal operating conditions. This significantly improves energy efficiency and realizes a sustainable living environment.

[0975] An "IoT device" is an electronic device that can connect to the Internet and collect, send, and receive data.

[0976] "Operational status data" refers to information about the operating status and behavior of IoT devices, including data such as temperature settings, power consumption, and operating time.

[0977] "Environmental data" refers to information about the state of the environment in which an IoT device is located, including, for example, outside temperature, humidity, and weather information.

[0978] A "machine learning algorithm" refers to a mathematical method or model that analyzes large amounts of data to learn patterns and rules and make predictions and classifications.

[0979] "Control instructions" refer to commands and configuration change instructions sent from a server to an IoT device, and are used to optimize the device's operation.

[0980] "User interface" refers to the software and hardware components that provide the visual and operational means for a user to interact with a system.

[0981] "Trends and patterns" refer to regularities and tendencies extracted from data collected over a long period of time, and are used to predict and optimize the future.

[0982] "Terminal" refers to an electronic device that functions as a relay point for transmitting data collected from IoT devices to a server.

[0983] "Server" refers to a central management device that collects and analyzes data, and generates and sends control instructions.

[0984] This invention provides an "IoT AI system" that works in conjunction with multiple IoT devices to achieve optimal operation and improve energy efficiency. This system is primarily composed of the following modules: a data collection module, a data analysis module, a control module, a user interface module, and a big data analysis module.

[0985] First, let's explain the data collection module. The server periodically collects operating status data from each IoT device. This data includes temperature settings, power usage, and operating hours. Environmental data is also simultaneously collected through outdoor temperature sensors and weather forecast APIs. The terminals send the operating status data collected from the IoT devices to the server. As a concrete example, the server receives data from an air conditioner that "the current room temperature is 25°C, the set temperature is 22°C," and obtains data from the weather forecast API that "the current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C."

[0986] Next, we will explain the data analysis module. The server uses the collected data to run machine learning algorithms and calculate the optimal device operating conditions. In this process, the server learns the efficient operating patterns of each IoT device and derives the settings that are most suitable for the user's lifestyle and environmental conditions. As a concrete example, the server concludes that "it is optimal to change the air conditioner's temperature setting to 23°C."

[0987] Next, we will explain the control module. The server generates and sends control instructions to each IoT device based on the analysis results. The terminal receives the control instructions from the server and changes the device settings accordingly. As a concrete example, the server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends this instruction to the air conditioner. The air conditioner changes the set temperature to 23°C.

[0988] Next, we will explain the user interface module. The server generates a dashboard to display the analysis results and control instructions in an easy-to-understand manner to the user. This dashboard can be accessed by the user via a smartphone or PC. The user can use the dashboard to check the current device status and control settings and change them as necessary. As a concrete example, the user receives a notification on their smartphone saying, "The air conditioner's set temperature has been changed to 23°C," and confirms this.

[0989] Finally, we will explain the big data analysis module. The server collects daily data and accumulates the analysis results over the long term, saving them as big data. This big data is used to further analyze trends and patterns and create future prediction models. As a concrete example, the server accumulates household energy consumption data on an annual basis, analyzes seasonal usage patterns, and suggests that "it would be more energy-efficient to not lower the temperature setting any further in winter."

[0990] For example, when a user goes out during the day in summer, the system analyzes the outside temperature, the temperature inside the house, and the weather forecast, and automatically adjusts the air conditioner's temperature setting. Even if the outside temperature is high and the house is predicted to heat up quickly, the system maintains the optimal setting and maximizes energy efficiency by returning the room to an appropriate temperature before the user returns home. Long-term data analysis also identifies energy consumption trends in specific regions and seasons, and performs optimization based on predictions. In this way, the IoT AI system improves user convenience and reduces energy waste. As a result, it is possible to reduce environmental impact and aim to create a sustainable living environment.

[0991] An example of a prompt for a generative AI model is, "Please explain what specific processing the IoT AI system performs at each step. As an example, please show a process that includes how to optimize the air conditioner's temperature setting."

[0992] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0993] Step 1:

[0994] Starting Data Collection

[0995] The server periodically sends data collection requests to each IoT device. These requests include instructions for operational status data such as temperature setting, power usage, and operating time. The terminal receives these requests and collects the necessary data from each IoT device. As a specific example, the terminal obtains data from an air conditioner that "the current room temperature is 25°C, and the set temperature is 22°C." The input is a data collection request from the server, and the output is operational status data from each IoT device.

[0996] Step 2:

[0997] Obtaining environmental data

[0998] The server obtains environmental data through an external temperature sensor or weather forecast API. Specifically, it obtains the current outside temperature and weather information. For example, the server uses a weather forecast API to obtain data such as "The current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C." The input is a request from an external sensor or API, and the output is environmental data.

[0999] Step 3:

[1000] Data transmission

[1001] The device sends the collected operational status data to the server. The server stores the environmental data obtained from the weather forecast API. Specifically, the device sends data to the server such as "current room temperature is 25°C, set temperature is 22°C." The input is the operational status data from the device, and the output is the operational status data and environmental data stored on the server.

[1002] Step 4:

[1003] Data integration

[1004] The server integrates the operational status data and environmental data from each IoT device. The input is the operational status data and environmental data, and the output is an integrated data set.

[1005] Step 5:

[1006] Running machine learning algorithms

[1007] The server uses the integrated data to run a machine learning algorithm and calculate the optimal device operating conditions. This process involves data preprocessing, feature extraction, and model training. For example, the server may conclude that "setting the air conditioner temperature to 23°C is optimal." The input is the integrated data set, and the output is the optimal operating conditions.

[1008] Step 6:

[1009] Control instruction generation

[1010] The server generates control instructions for each IoT device based on the analysis results. Specifically, the server generates an instruction to "change the air conditioner's set temperature to 23°C." The input is the optimal operating conditions, and the output is the generated control instruction.

[1011] Step 7:

[1012] Sending control instructions

[1013] The server sends the generated control instructions to each IoT device. The terminal receives the control instructions from the server and changes the device settings accordingly. Specifically, the server sends the instruction "change the air conditioner's set temperature to 23°C," and the air conditioner changes its set temperature. The input is the generated control instruction, and the output is the setting change of the IoT device that received the control instruction.

[1014] Step 8:

[1015] Generate a dashboard

[1016] The server generates a dashboard to display the analysis results and control instructions in an easy-to-understand manner for the user. The inputs are the analysis results and control instructions, and the output is the generated dashboard.

[1017] Step 9:

[1018] Accessing the User Interface

[1019] Users access the dashboard using a smartphone or PC to check the current device status and control settings. Specifically, the user receives a notification on their smartphone saying, "The air conditioner temperature setting has been changed to 23°C," and confirms it. The input is the dashboard URL and app, and the output is status information that the user can check.

[1020] Step 10:

[1021] Accumulation and analysis of big data

[1022] The server collects daily data and accumulates the analysis results over the long term, saving them as big data. Data accumulated over a long period of time is analyzed on a large scale to identify trends and patterns. Specifically, the server accumulates household energy consumption data on an annual basis and suggests, "It would be more energy-efficient to not lower the temperature setting any further in winter." The input is the daily collected data and analysis results, and the output is the accumulated big data and its analysis results.

[1023] (Application example 1)

[1024] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1025] Modern self-driving vehicles operate using numerous information and communication technology devices, but it is difficult to efficiently manage the operating status of these devices and external environmental data, and calculate optimal operating conditions in real time. Furthermore, systems that automatically set optimal routes and speeds taking into account traffic information and weather conditions have not yet been fully realized. As a result, there are problems with wasted energy consumption and reduced operating efficiency.

[1026] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1027] In this invention, the server includes means for collecting operation status data from various information and communication technology devices, means for acquiring external environment data, means for analyzing the collected data using a machine learning algorithm and calculating optimal device operating conditions, means for sending control instructions to the various information and communication technology devices based on the analysis results, means for providing a user interface for displaying the analysis results and the content of the control instructions to the user, means for analyzing data accumulated over a long period of time on a large scale and identifying trends and patterns, means for collecting and analyzing vehicle status and traffic information to calculate optimal driving routes and speed settings, and means for sending control instructions to the autonomously driven vehicle based on the analysis results, thereby optimizing the operating conditions of the autonomously driven vehicle and enabling improved energy efficiency and operating efficiency.

[1028] "Information and communication technology equipment" means electronic devices for data collection, communication, control, etc.

[1029] "Operation status data" refers to information relating to the operating status of the device, such as its usage status, power consumption, temperature, and usage time.

[1030] "External environment data" is information about surrounding conditions such as weather, temperature, humidity, and traffic conditions.

[1031] A "machine learning algorithm" is a mathematical method for learning patterns and relationships from data and making predictions and classifications.

[1032] A "control instruction" is an instruction to change or adjust the operation of a device.

[1033] A "user interface" is the screen or method of operation that allows a user to interact with a system.

[1034] "Trends and patterns" are tendencies and regularities that emerge from data collected over a long period of time.

[1035] An "autonomous vehicle" is a vehicle that operates autonomously without the need for human operation.

[1036] A "travel route" is a specific route that a vehicle will take to reach its destination.

[1037] "Speed ​​setting" is a parameter that determines the vehicle's traveling speed.

[1038] This invention relates to an "Autonomous Vehicle AI Optimization Commander" system that works in conjunction with multiple information and communication technology devices to optimize operation and improve energy efficiency. This enables vehicle operation to be optimized and energy efficiency to be improved. The system is primarily composed of the following modules: a data collection module, a data analysis module, a control module, a user interface module, and a big data analysis module.

[1039] Explanation of the specific processing of the system

[1040] 1. Data Collection Module

[1041] The server periodically collects operational status data from the autonomous vehicle, including remaining battery power, driving speed, distance to destination, etc. It also simultaneously obtains environmental data from traffic information APIs and weather forecast APIs.

[1042] The terminal transmits operation-related data, including traffic and weather information, to the server.

[1043] Example: A server receives data from an autonomous vehicle, such as "Current battery remaining is 80%, driving speed is 60km / h, distance to destination is 20km." At the same time, it obtains data from a traffic information API that "Congestion is occurring" and from a weather forecast API that "Current weather is rain."

[1044] 2. Data Analysis Module

[1045] The server analyzes the collected data using machine learning algorithms to calculate the optimal vehicle route and speed. In this process, the server learns the optimal driving pattern based on data obtained from various information and communication technology devices, and derives the settings that are most suitable for real-time environmental conditions.

[1046] Example: The server performs an analysis and concludes, "Change driving speed to 55 km / h and re-route to avoid traffic jams."

[1047] 3. Control Module

[1048] The server generates control instructions for the autonomous vehicle based on the analysis results, which reflect the optimal driving route and speed settings.

[1049] The terminal receives control instructions from the server and changes the settings of the autonomous vehicle accordingly.

[1050] Example: The server generates an instruction to "change driving speed to 55 km / h and re-plan the route" and sends this instruction to an autonomous vehicle. The autonomous vehicle changes its driving speed to 55 km / h and re-plans the route.

[1051] 4. User Interface Module

[1052] The server generates a dashboard that displays the analysis results and control instructions in an easy-to-understand format for users, and users can access this dashboard via their smartphones or PCs.

[1053] The dashboard allows users to view the current vehicle status and control settings and change them if necessary.

[1054] Example: A user receives a notification on their smartphone that says "Your driving speed has been changed to 55km / h" and confirms it.

[1055] 5. Big Data Analysis Module

[1056] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[1057] Example: A server accumulates energy consumption data of autonomous vehicles on a yearly basis, analyzes seasonal usage patterns, and suggests that "it would be more energy-efficient to not reduce the speed any further in winter."

[1058] Prompt Sentence Examples

[1059] As a concrete example, the following prompt sentences can be input to a generative AI model:

[1060] "Vehicle ID 12345's current speed is 60 km / h, battery remaining is 80%, distance to destination is 20 km, there is traffic congestion information, and it is raining. Please predict the optimal driving speed under these conditions."

[1061] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1062] Step 1:

[1063] Data collection

[1064] The server collects operational status data from the autonomous vehicles (remaining battery level, driving speed, distance to destination, etc.) and simultaneously obtains environmental data (traffic conditions, weather information) from traffic information APIs and weather forecast APIs.

[1065] Input: Data from autonomous vehicles, traffic information API, weather forecast API

[1066] Output: Collected data (remaining battery level, driving speed, distance to destination, traffic conditions, weather information)

[1067] Specific operation: The server periodically connects to the autonomous vehicle to obtain operating status data, and at the same time calls the API to obtain external environment data.

[1068] Step 2:

[1069] Data analysis

[1070] The server then inputs the collected data into a machine learning algorithm to calculate the optimal route and speed, taking into account the vehicle's remaining battery power, current speed, distance to destination, traffic conditions, weather information, and more.

[1071] Input: Collected data (battery level, driving speed, distance to destination, traffic conditions, weather information)

[1072] Output: Optimal driving path and speed settings

[1073] Specific operation: The server uses a machine learning model (e.g., a Linear Regression model) to analyze the input data and calculate the optimal driving route and speed.

[1074] Step 3:

[1075] Control instruction generation

[1076] The server generates control instructions for the autonomous vehicle based on the results of the data analysis, including instructions on optimal driving routes and speed settings.

[1077] Input: Data analysis results (optimal driving route and speed settings)

[1078] Output: Control instructions for the autonomous vehicle

[1079] Specific operation: The server executes a control algorithm based on the analysis results and generates control instructions for the autonomous vehicle.

[1080] Step 4:

[1081] Sending control instructions

[1082] The server then sends the generated control instructions to the autonomous vehicle, which then adjusts its route and speed based on these instructions.

[1083] Input: Generated control instructions

[1084] Output: The behavior of the autonomous vehicle based on the executed control instructions.

[1085] Specific operation: The server sends instructions to the autonomous vehicle to adjust its speed and route, and the autonomous vehicle carries out the instructions.

[1086] Step 5:

[1087] Display in the user interface

[1088] The server generates a user interface to display the analysis results and control instructions to the user, who can then check them on their smartphone or PC.

[1089] Input: Analysis results, control instructions

[1090] Output: What is displayed on the user interface

[1091] Specific operation: The server generates a dashboard and sends information to a smartphone or PC to display the analysis results and control instructions.

[1092] Step 6:

[1093] Big data accumulation and analysis

[1094] The server collects daily data and accumulates the analysis results over the long term, saving them as big data. This big data can then be used to analyze more detailed trends and patterns.

[1095] Input: Daily data collection results, analysis results

[1096] Output: Long-term trends and patterns

[1097] Specific operation: The server periodically analyzes the accumulated data and updates the future prediction model.

[1098] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1099] This invention relates to an "IoT AI Commander" system that works in conjunction with multiple IoT devices to achieve optimal operation and improve energy efficiency. This system aims to provide a more user-friendly and comfortable environment by combining it with an emotion engine that recognizes the user's emotions. The specific processing of the system is described below.

[1100] Explanation of the specific processing of the system

[1101] 1. Data Collection Module

[1102] The server periodically sends requests to collect operational status data from each IoT device, including the device's operating status, configuration information, power consumption, and operating time.

[1103] The terminal periodically sends operational status data of the IoT device to the server.

[1104] Example: An air conditioner sends data to the server saying, "The current room temperature is 25°C, and the set temperature is 22°C."

[1105] 2. Emotion Recognition by Emotion Engine

[1106] The server recognizes emotions by analyzing the user's facial expressions and tone of voice, using cameras and microphones to collect data in real time.

[1107] Example: If a user is in their living room and has a relaxed expression, the emotion engine will recognize that expression and determine that they are "relaxed."

[1108] 3. Data integration and preprocessing

[1109] The server integrates the operational status data of IoT devices, environmental data (outside temperature, weather forecast, etc.), and emotion data, and performs preprocessing, which includes data cleansing, missing value completion, and format standardization.

[1110] 4. Analysis using machine learning algorithms

[1111] The server then uses machine learning algorithms to analyze the preprocessed data, with the goal of calculating the optimal operating conditions for each IoT device.

[1112] Example: As a result of the analysis, the conclusion is reached that "it is optimal to change the air conditioner's set temperature to 23°C."

[1113] 5. Integrated analysis with emotion data

[1114] The server further adjusts the device's operating conditions based on the user's emotional data obtained by the emotion engine. If the user is feeling stressed, the server may make additional adjustments, such as changing the lighting to a warmer color and playing music.

[1115] Example: If the user is determined to be stressed, not only will the air conditioner temperature be changed to 23°C, but the color temperature of the lighting will be changed to 3000K (warm color) to create a relaxing environment.

[1116] 6. Generating and Sending Control Instructions

[1117] The server generates the necessary control instructions based on the analysis results and sends them to each IoT device.

[1118] The terminal receives control instructions from the server and changes the device settings accordingly.

[1119] Example: The server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends it to the air conditioner. The air conditioner changes the set temperature to 23°C.

[1120] 7. User Interface

[1121] The server generates a dashboard to display the analysis results and control instructions to users. Users can access the dashboard on their smartphones or PCs to check and change the current device status and control settings.

[1122] Users can monitor the operation of IoT devices based on the information displayed on the dashboard and manually change settings as needed.

[1123] Example: A user sees a notification on their smartphone that says "The air conditioner temperature setting has been changed to 23°C" and manually adjusts the temperature setting further if necessary.

[1124] 8. Big Data Analysis

[1125] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[1126] Example: Analyzing energy consumption data for each season (summer and winter) and proposing more efficient operation methods.

[1127] In this way, the IoT AI Commander improves user convenience and reduces energy waste while taking emotions into consideration, ultimately reducing the environmental impact and helping to create a sustainable living environment.

[1128] The processing flow will be explained below.

[1129] Step 1:

[1130] The server periodically sends requests to collect operational status data from each IoT device, prompting each device to send its current operating status, configuration information, power consumption, operating time, and other information.

[1131] Step 2:

[1132] The terminal receives a request from the server and transmits the operational status data of each IoT device to the server.

[1133] Example: An air conditioner sends information to the server that "the current room temperature is 25°C, and the set temperature is 22°C."

[1134] Step 3:

[1135] The server accesses external environmental data sources (e.g., weather forecast APIs or external sensors) to obtain environmental data such as outside temperature, humidity, and weather forecasts.

[1136] Example: The server retrieves data from a weather forecast API: "The current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C."

[1137] Step 4:

[1138] The server runs an emotion engine to obtain emotion data by analyzing the user's facial expressions and voice. Data is collected in real time via cameras and microphones.

[1139] Example: A camera and microphone analyze a user's facial expressions and tone of voice to determine whether they are relaxed or stressed.

[1140] Step 5:

[1141] The server integrates the collected operational status data, environmental data, and emotion data and performs preprocessing, which cleanses the data, fills in missing values, and standardizes the format.

[1142] Step 6:

[1143] The server uses machine learning algorithms to analyze the preprocessed data and calculate the optimal operating conditions for each IoT device.

[1144] Example: After analysis, you conclude that the optimal temperature setting for an air conditioner is 23°C.

[1145] Step 7:

[1146] The server optimizes the analysis results by integrating the emotional data, generating different control instructions depending on whether the user is relaxed or stressed.

[1147] Example: If the user is determined to be stressed, the system will instruct the user to change the air conditioner temperature setting to 23°C, change the lighting color temperature to warmer colors, and play relaxing music.

[1148] Step 8:

[1149] The server then sends the generated control instructions to each IoT device, which then changes their behavior in accordance with the new settings.

[1150] Step 9:

[1151] The terminal receives control instructions from the server and changes the settings of the corresponding IoT device.

[1152] Example: The air conditioner changes the set temperature to 23°C, the smart lighting changes the color temperature to 3000K (warm), and the music player starts playing relaxing music.

[1153] Step 10:

[1154] The server generates a dashboard to display the analysis results and control instructions to the user, which can be viewed on a smartphone or PC.

[1155] Step 11:

[1156] Users can use the dashboard to view current device status and control settings, and manually change settings if necessary.

[1157] Example: A user sees a notification on their smartphone that says "The air conditioner temperature setting has been changed to 23°C" and then changes the temperature setting to 25°C.

[1158] Step 12:

[1159] The server accumulates daily data over the long term and stores it as big data. This data is used to analyze trends and patterns and create predictive models for the future.

[1160] Example: Analyzing past data, understanding seasonal energy consumption patterns, and proposing optimal energy consumption plans.

[1161] The above is the specific processing flow of the "IoT AI Commander" system, which combines user emotional data. This system improves user convenience and comfort while reducing energy waste and realizing a sustainable living environment.

[1162] Example 2

[1163] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1164] Modern IoT devices collect large amounts of data, improving user convenience. However, analyzing the data and calculating optimal device operating conditions requires advanced technology. Furthermore, existing systems do not adequately consider user emotions and comfort, creating challenges in improving the user experience. Furthermore, there is a need for more methods to utilize collected data over the long term to optimize energy efficiency and realize sustainable living environments.

[1165] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1166] In this invention, the server includes means for collecting operation status data from various IoT devices, means for acquiring external environmental data, means for analyzing the collected data using a machine learning algorithm and calculating optimal device operating conditions, means for sending control instructions to the various IoT devices based on the analysis results, means for providing a user interface for displaying the analysis results and the control instructions to the user, means for analyzing data accumulated over a long period of time on a large scale and identifying trends and patterns, means for analyzing the user's facial expressions and tone of voice to collect emotion data, and means for adjusting the IoT device operating conditions based on the emotion data. This makes it possible to calculate optimal device operating conditions taking user emotions into consideration, improve energy efficiency, and realize a sustainable living environment.

[1167] "IoT devices" is a general term for devices that are connected to the Internet and can collect and control data by communicating with each other.

[1168] "Operational status data" refers to information about the operation of an IoT device, such as its operating status, settings, power consumption, and operating time.

[1169] "Environmental data" refers to information about the external environment, such as outside temperature, humidity, and weather forecasts.

[1170] "Machine learning algorithms" refers to technologies that include mathematical models and statistical methods for analyzing collected data and calculating optimal equipment operating conditions.

[1171] "Control instructions" refer to specific operational commands sent to IoT devices to achieve optimal device operating conditions.

[1172] "User interface" refers to the display screen and operating means that allow a user to interact with the system and check or change the system's status.

[1173] "Large-scale analysis" refers to techniques and methods that use large amounts of data accumulated over a long period of time to identify trends and patterns.

[1174] "Emotion data" refers to information that indicates the user's emotional state, obtained from the user's facial expression, tone of voice, etc.

[1175] "Trend" or "pattern" refers to a statistical tendency or recurring characteristic derived from collected data.

[1176] "Operating conditions" refers to the settings and operating modes required for optimal operation of IoT devices.

[1177] A "sustainable living environment" refers to a sustainable living space that improves energy efficiency, reduces waste, and reduces the environmental impact.

[1178] This invention relates to an "IoT AI Commander" system that aims to achieve optimal operation and improved energy efficiency by linking multiple IoT devices. This system aims to provide a more user-friendly and comfortable environment by combining it with an emotion engine that recognizes the user's emotions.

[1179] The system's main components are a server, a device, and a user. The server collects operational status data from various IoT devices and acquires external environmental data, then analyzes all collected data using a machine learning algorithm. Based on the analysis results, it sends control instructions to the IoT devices and displays them to the user through a user interface. In addition, an emotion engine analyzes the user's facial expressions and tone of voice, collects emotional data, and adjusts the operating conditions of the IoT devices. This increases user comfort and improves energy efficiency.

[1180] Hardware and Software Examples

[1181] Servers: High-performance API servers, database management systems (e.g., MySQL), machine learning frameworks (e.g., TensorFlow)

[1182] Terminal: IoT device (e.g. smart speaker, smart air conditioner, smart light)

[1183] Emotion engine: Image and speech recognition technologies (e.g., OpenCV, Google Speech-to-Text API)

[1184] Explanation of program processing

[1185] Data collection

[1186] The server periodically collects operational status data from various IoT devices. Specifically, it sends API requests and receives data such as operating status, configuration information, and power consumption from the IoT devices. The terminal is responsible for sending operational status data to the server and providing the specific operating status of air conditioners and lighting.

[1187] Example: An air conditioner sends the following data to the server: "The current room temperature is 25°C, the set temperature is 22°C, and the power consumption is 1.2kW."

[1188] Example prompt: "Please report your current air conditioning settings and operating status."

[1189] Collecting Emotional Data

[1190] The server analyzes the user's facial expressions and tone of voice in real time through a camera and microphone. The emotion engine then sends the collected emotional data to the server.

[1191] Example: The server analyzes video data from a camera to detect whether the user is laughing, and also determines from the tone of voice that the user is relaxed.

[1192] Example prompt: "Analyze the user's facial expression and tone of voice and report their current emotions."

[1193] Data preprocessing and integrated analysis

[1194] The server integrates the collected IoT device data and emotion data, cleansing the data, filling in missing values, and standardizing the format. It then analyzes the data using machine learning algorithms to calculate the optimal operating conditions for the IoT device. The analysis results and emotion data are integrated to generate the final control instructions.

[1195] Example: A machine learning model calculates the optimal temperature setting for an air conditioner as 23°C. If the user is relaxing, change the color temperature of the lights to a warmer color and play music.

[1196] Example prompt: "Analyze the data and calculate the optimal temperature setting for your air conditioner."

[1197] Generating and sending control instructions

[1198] The server generates specific control instructions based on the analysis results and sends them to the IoT device. The device receives the control instructions from the server and changes the device settings accordingly.

[1199] Example: The server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends it to the air conditioner. The air conditioner changes the set temperature to 23°C.

[1200] Example prompt: "Please tell the air conditioner to change the temperature setting to 23°C."

[1201] Providing a user interface

[1202] The server generates a dashboard to display the analysis results and control instructions to the user, providing a user interface. Users can access the dashboard on their smartphones or PCs to check and change the current device status and control settings.

[1203] Example: A user sees a notification on their smartphone that says "The air conditioner temperature setting has been changed to 23°C" and manually adjusts the temperature setting further if necessary.

[1204] Example prompt: "Show me the current air conditioner temperature setting and operating status."

[1205] Big Data Analysis

[1206] The server collects daily data and accumulates the analysis results over the long term, saving them as big data. This big data is used to analyze trends and patterns and create future prediction models.

[1207] Example: A server analyzes historical energy consumption data and proposes new algorithms to improve energy efficiency.

[1208] Example prompt: "Based on historical energy consumption data, create a proposal to improve the operational efficiency of your air conditioner."

[1209] Through the above process, this system optimizes energy usage while maintaining user comfort, providing a sustainable living environment.

[1210] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1211] Step 1: Collecting IoT device data

[1212] The server sends a request to each IoT device to collect operational status data.

[1213] Input: Data collection request sent from the server to the IoT device.

[1214] Output: Operational status data sent from IoT devices (e.g., room temperature, set temperature, power consumption).

[1215] Specific operation: The server sends a request to the air conditioner, and the air conditioner replies with the data "Current room temperature is 25°C, set temperature is 22°C, power consumption is 1.2kW."

[1216] Step 2: Obtaining environmental data

[1217] The server obtains external environmental data (e.g., outside temperature, weather forecast).

[1218] Input: Real-time environmental data that the server retrieves from external data sources.

[1219] Output: The captured environmental data.

[1220] Specific operation: The server accesses the weather API to obtain the current outside temperature and weather forecast.

[1221] Step 3: Collecting emotion data

[1222] The server collects the user's facial expressions and tone of voice in real time through a camera and microphone.

[1223] Input: Video data from the camera and audio data from the microphone.

[1224] Output: Emotion data indicating the user's emotions.

[1225] How it works: The server analyzes the camera footage to detect whether the user is smiling and also determines from the tone of their voice that they are relaxed.

[1226] Step 4: Preprocessing the data

[1227] The server integrates and preprocesses data from IoT devices, environmental data, and emotion data.

[1228] Input: Operational status data from IoT devices, external environmental data, and user emotion data.

[1229] Output: Preprocessed integrated data.

[1230] Specific operation: The server fills in missing values ​​of each data with the average value and unifies the data format.

[1231] Step 5: Machine learning analysis

[1232] The server uses machine learning algorithms to perform analysis on the pre-processed data.

[1233] Input: Preprocessed integrated data.

[1234] Output: Optimal operating conditions for IoT devices.

[1235] What it does: A machine learning model calculates the optimal temperature setting for an air conditioner: 23°C. The algorithm then derives settings that minimize energy consumption and maximize user comfort.

[1236] Step 6: Integrated analysis with emotion data

[1237] The server readjusts the analysis results by taking into account the emotion data obtained from the emotion engine.

[1238] Input: Analysis results from machine learning algorithms, emotion data.

[1239] Output: Optimal operating conditions taking emotions into account.

[1240] Specific behavior: If the user is determined to be relaxed, the air conditioner temperature setting is changed to 23°C and the lighting color temperature is changed to 3000K (warm color).

[1241] Step 7: Generate and send control instructions

[1242] The server generates control instructions based on the analysis results and sends them to each IoT device.

[1243] Input: Optimal operating conditions taking emotions into account.

[1244] Output: Specific control instructions for each IoT device.

[1245] Specific operation: The server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends it to the air conditioner. The air conditioner changes the set temperature to 23°C.

[1246] Step 8: Update the User Interface

[1247] The server displays the analysis results and the contents of the control instructions on a dashboard and provides them to the user.

[1248] Input: Control instructions and analysis results.

[1249] Output: The dashboard content that is displayed to the user.

[1250] Specific behavior: The user sees a notification on their smartphone that says "The air conditioner temperature setting has been changed to 23°C" and adjusts it as needed.

[1251] Step 9: Analyze the big data

[1252] The server collects daily data and stores the analysis results over the long term, and performs large-scale analysis.

[1253] Input: Data accumulated over a long period of time.

[1254] Output: Analysis results that identify trends and patterns.

[1255] Specific operation: The server analyzes past energy consumption data and proposes new algorithms to improve energy efficiency.

[1256] Through these steps, the system optimizes energy consumption and realizes sustainable living while maintaining user comfort.

[1257] (Application example 2)

[1258] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1259] In existing IoT environments, it is difficult to set optimal device operation settings that take user emotions into account, which means that users cannot always enjoy comfort and convenience.In addition, the effective use of collected data and the optimization of long-term energy consumption are not sufficiently implemented, making it difficult to achieve a sustainable living environment.

[1260] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting operation status data from various sensing devices, means for acquiring external environmental information, means for analyzing the collected data and user emotion data using a machine learning algorithm and calculating optimal device operating conditions, means for sending control instructions to various devices based on the analysis results, means for providing a user interface for displaying the analysis results and the content of the control instructions to the user, means for analyzing data accumulated over a long period of time on a large scale and identifying trends and patterns, means for recognizing emotions by analyzing the user's facial expressions and tone of voice, and means for adjusting device operation settings based on the emotion data. This makes it possible to provide a comfortable environment that is in tune with the user's emotions, while optimizing energy consumption and realizing a sustainable living environment.

[1261] A "sensing device" is a device that detects information about the state of the environment or an object and collects that data.

[1262] "Environmental information" refers to data about the external environment, such as temperature, humidity, weather, and illuminance.

[1263] "Device" refers to various equipment that collects operational status data and is controlled in conjunction with external systems.

[1264] "Emotion data" is information about the user's psychological state obtained by analyzing the user's facial expressions and tone of voice.

[1265] A "machine learning algorithm" is a computational method for building patterns and predictive models based on collected data and calculating optimal operating conditions.

[1266] A "user interface" is a system that allows users to visually check and operate analysis results and control instructions.

[1267] "Trend" refers to a tendency or pattern of fluctuation that can be found by analyzing collected data over a long period of time.

[1268] "Data collection means" refers to the function for collecting necessary data from various sensing devices.

[1269] "External information acquisition means" refers to a function for acquiring data related to the external environment.

[1270] "Analysis means" refers to the function of analyzing collected data using machine learning algorithms to derive optimal operating conditions.

[1271] "Control instruction sending means" refers to a function for sending instructions to various devices based on the analysis results.

[1272] "Emotion recognition means" refers to a function for recognizing emotions by analyzing the user's facial expressions and tone of voice.

[1273] "Operation setting adjustment means" refers to a function for adjusting the operation settings of a device based on emotion data.

[1274] A system for implementing the present invention is configured as follows.

[1275] Specific system configuration

[1276] 1. Data Collection Module

[1277] The server periodically collects operational status data from various sensing devices, including the operating status, configuration information, power consumption, and operating time of each device.

[1278] The terminal periodically sends operational status data of the IoT device to the server.

[1279] Example: A lighting system sends data to a server saying "Current brightness is 500 lumens, set brightness is 400 lumens."

[1280] 2. Environmental Information Acquisition Module

[1281] The server acquires external environmental information (temperature, humidity, weather information, etc.).

[1282] Example: Obtain information from a weather data provider such as "The current outside temperature is 28°C and the humidity is 70%."

[1283] 3. Emotion Recognition Module

[1284] The server analyzes the user's facial expressions and tone of voice to recognize their emotions, using cameras and microphones to collect data in real time.

[1285] Example: A camera in a store captures the user's facial expression and determines that they are "relaxed."

[1286] 4. Analysis Module

[1287] The server uses machine learning algorithms to analyze the collected data and calculate the optimal operating conditions for the device.

[1288] Example: Analysis results in the conclusion that "adjusting the lighting brightness to 420 lumens is optimal."

[1289] 5. Control instruction transmission module

[1290] The server generates the necessary control instructions based on the analysis results and sends them to the various devices.

[1291] The terminal receives control instructions from the server and changes the device settings accordingly.

[1292] Example: The server generates an instruction to "change the lighting brightness to 420 lumens" and sends it to the lighting system. The lighting system changes the set brightness to 420 lumens.

[1293] 6. User Interface

[1294] The server provides an interface for displaying the analysis results and control instructions to the user. Users can access the interface on their smartphones or PCs to check and change the current device status and settings.

[1295] Based on the information displayed in the interface, the user can monitor the device's operation and manually change settings if necessary.

[1296] Example: A user sees a notification on their smartphone that says "Light setting brightness changed to 420 lumens" and manually adjusts the setting if necessary.

[1297] 7. Big Data Analysis Module

[1298] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[1299] Example: Analyzing seasonal energy consumption data and proposing more efficient operation methods.

[1300] 8. Operational Settings Adjustment Module

[1301] The server adjusts the device's operational settings based on the user's emotional data. If the user is feeling stressed, additional adjustments are made, such as changing the lighting to warmer colors and playing music.

[1302] Example: If the user is determined to be stressed, the color temperature of the lighting is changed to 3000K (warm color) to create a relaxing environment.

[1303] Specific examples

[1304] 1. While customers are browsing products in a store, smart glasses analyze their facial expressions and, if they are smiling, change the background music to a brighter tone.

[1305] 2. If customers seem stressed, reset the store's air conditioning to a comfortable temperature and change the lighting to a warmer color.

[1306] Prompt Sentence Examples

[1307] "Please provide a blueprint for an application that uses smart glasses and IoT devices to optimize the environment in a physical store according to the customer's emotions. As a specific example, please include a scenario in which cheerful music is played for smiling customers, and the air conditioning and lighting are adjusted for stressed customers."

[1308] The above is a detailed example of how the "IoT AI Commander" can be applied in a brick-and-mortar store. By using this system, it is possible to provide a comfortable environment that is sensitive to the user's emotions, while optimizing energy consumption and realizing a sustainable living environment.

[1309] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1310] Step 1:

[1311] Data collection

[1312] The server collects operational status data from various sensing devices. Input data includes the operating status, setting information, power consumption, and operating time of each device. This data is collected and stored in the server's database. Specifically, the lighting system sends data such as "current brightness is 500 lumens, set brightness is 400 lumens."

[1313] Step 2:

[1314] Obtaining environmental information

[1315] The server obtains external environmental information (temperature, humidity, weather information, etc.) from various APIs. Input data includes weather data and data from environmental monitors, which are collected and used for analysis. Specifically, it obtains information such as "The current outside temperature is 28°C and humidity is 70%" from a weather data provider.

[1316] Step 3:

[1317] Collecting Emotional Data

[1318] The device collects the user's facial expressions and tone of voice through a camera and microphone, and converts them into emotional data using an emotion recognition model (using, for example, TensorFlow and Keras). The input data is video and audio data, and the output is the user's emotional state (relaxed, stressed, etc.). Specifically, the device captures the user's facial expressions with a camera inside the store and determines that the user is "relaxed."

[1319] Step 4:

[1320] Data Preprocessing

[1321] The server integrates the collected operational status data, environmental information, and emotion data, and performs preprocessing such as data cleansing, missing value completion, and data format unification. The input data is the data collected in steps 1 to 3, and the output is a clean, unified, and analyzable dataset.

[1322] Step 5:

[1323] Analysis and Optimization

[1324] The server analyzes the preprocessed data using machine learning algorithms (e.g., random forest, regression, etc.) to calculate the optimal device operating conditions. The input data is the preprocessed data, and the output is specific operating instructions (e.g., adjust the lighting brightness to 420 lumens).

[1325] Step 6:

[1326] Generating and sending control instructions

[1327] The server generates control instructions based on the analysis results and sends them to various devices. The input data are optimized operation instructions, and the output is specific operation instructions for each device. Specifically, an instruction such as "change the lighting brightness to 420 lumens" is generated and sent to the lighting system.

[1328] Step 7:

[1329] Providing a user interface

[1330] The server provides an interface for displaying the analysis results and control instructions to the user. The input data are the analysis results and control instructions, and the output is a dashboard and notifications that the user can access. The user can check the current device status and settings from their smartphone or PC and manually adjust them as necessary. Specifically, the user receives a notification on their smartphone that "the lighting setting brightness has been changed to 420 lumens."

[1331] Step 8:

[1332] Big Data Analysis

[1333] The server collects daily data and stores the analysis results over the long term, and performs big data analysis. The input data is data collected over a long period of time, and the output is the analysis results on trends and patterns. This data is used to create future prediction models that are useful for proposing energy efficiency improvements. Specifically, it analyzes energy consumption data by season and proposes more efficient operating methods.

[1334] Step 9:

[1335] Adjusting Operation Settings

[1336] The server adjusts the device's operational settings based on the user's emotional data. The input data is the result of emotion recognition, and the output is the emotion-based operational settings. Specifically, if the user is feeling stressed, the server adjusts the lighting color temperature to 3000K (warm color) to create a relaxing environment.

[1337] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1338] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1339] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1340] [Fourth embodiment]

[1341] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1342] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1343] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1344] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1345] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1346] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1347] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1348] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1349] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1350] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1351] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1352] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1353] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1354] This invention relates to an "IoT AI Commander" system that works in conjunction with multiple IoT devices to achieve optimal operation and improve energy efficiency. This system is primarily composed of the following modules: a data collection module, a data analysis module, a control module, a user interface module, and a big data analysis module.

[1355] Explanation of the specific processing of the system

[1356] 1. Data Collection Module

[1357] The server periodically collects operational status data from each IoT device, including temperature settings, power consumption, and operating time. Environmental data from external temperature sensors and weather forecast APIs is also collected at the same time.

[1358] The terminal periodically transmits the operating status of each IoT device to the server.

[1359] Example: The server receives data from the air conditioner, such as "The current room temperature is 25°C, and the set temperature is 22°C." At the same time, it obtains data from the weather forecast API, such as "The current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C."

[1360] 2. Data Analysis Module

[1361] The server analyzes the collected data using machine learning algorithms to calculate the optimal operating conditions for the devices. In this process, it learns the efficient operating patterns of each IoT device and derives the settings that are best suited to the user's lifestyle and environmental conditions.

[1362] Example: The server performs an analysis and concludes that "it would be best to change the air conditioner's temperature setting to 23°C."

[1363] 3. Control Module

[1364] The server generates control instructions for each IoT device based on the analysis results. These control instructions are designed to reflect the optimal operating settings for each device.

[1365] The terminal receives control instructions from the server and changes the device settings accordingly.

[1366] Example: The server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends this instruction to the air conditioner. The air conditioner changes the set temperature to 23°C.

[1367] 4. User Interface Module

[1368] The server generates a dashboard that displays the analysis results and control instructions in an easy-to-understand format for users, and users can access this dashboard via their smartphones or PCs.

[1369] The dashboard allows users to view current device status and control settings and change them if necessary.

[1370] Example: A user receives a notification on their smartphone that says, "The air conditioner temperature setting has been changed to 23°C" and confirms it.

[1371] 5. Big Data Analysis Module

[1372] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[1373] Example: A server accumulates energy consumption data for the home on a yearly basis, analyzes seasonal usage patterns, and suggests that "it would be more energy efficient to not lower the set temperature any further in winter."

[1374] Example

[1375] For example, when a user leaves the house during the day in summer, the system analyzes the outside temperature, the temperature inside the house, and the weather forecast to automatically adjust the air conditioner's temperature setting. Even if the outside temperature is high and the house is predicted to heat up quickly, the system will maintain the optimal setting and maximize energy efficiency by returning the room to an appropriate temperature before the user returns. Long-term data analysis also identifies energy consumption trends in specific regions and seasons, allowing for predictive optimization.

[1376] In this way, the IoT AI Commander improves user convenience and reduces energy waste, ultimately reducing the environmental impact and helping to create a sustainable living environment.

[1377] The processing flow will be explained below.

[1378] Step 1:

[1379] The server periodically sends requests to collect operational status data from each IoT device, which triggers each IoT device to send its current operating status and configuration information to the server.

[1380] Step 2:

[1381] In response to the received request, the terminal sends the operating status data of each IoT device (temperature setting, power consumption, operating time, etc.) to the server.

[1382] Example: An air conditioner sends data to the server saying, "The current room temperature is 25°C, and the set temperature is 22°C."

[1383] Step 3:

[1384] The server accesses external environmental sensors and weather forecast APIs to obtain environmental data such as outside temperature, humidity, and weather forecasts.

[1385] Example: The server retrieves data from a weather forecast API: "The current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C."

[1386] Step 4:

[1387] The server preprocesses the collected operational status data and environmental data, which includes data cleanup, filling in errors and missing values, and standardizing the format.

[1388] Step 5:

[1389] The server uses machine learning algorithms to perform analysis based on the preprocessed data, which then calculates the optimal operating conditions for each IoT device.

[1390] Example: As a result of the analysis, the conclusion is reached that "it is optimal to change the air conditioner's set temperature to 23°C."

[1391] Step 6:

[1392] The server generates specific control instructions based on the analysis results, which are designed to reflect optimal operating settings for each IoT device.

[1393] Example: The server generates the instruction "Change the air conditioner temperature setting to 23°C."

[1394] Step 7:

[1395] The server then sends the generated control instructions to each IoT device, which then changes their operating settings in accordance with the control instructions.

[1396] Step 8:

[1397] The terminal executes the control instructions received from the server and changes the settings of the corresponding IoT device.

[1398] Example: An air conditioner receives an instruction to change the set temperature to 23°C.

[1399] Step 9:

[1400] The server generates a dashboard to display the analysis results and control instructions to the user, who can access the dashboard via a smartphone or PC.

[1401] Step 10:

[1402] The dashboard allows users to view current device status and control settings and change them if necessary.

[1403] Example: A user sees a notification on their smartphone that says, "The air conditioner temperature setting has been changed to 23°C."

[1404] Step 11:

[1405] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[1406] Step 12:

[1407] Based on the results of big data analysis, the server makes new predictions and optimizations, providing even more accurate control instructions.

[1408] Example: Based on winter usage data, we suggest that "it would be more energy efficient to not lower the set temperature any further."

[1409] Example 1

[1410] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1411] In systems using conventional IoT devices, efficient operation of each device and improvement of energy efficiency have not been sufficiently achieved. Furthermore, in many systems, it is difficult to effectively analyze collected data and derive optimal operating conditions. This poses a challenge in optimizing energy consumption in homes and businesses and providing a sustainable living environment.

[1412] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1413] In this invention, the server includes: means for collecting operational status data from various IoT devices; means for acquiring external environmental data; means for analyzing the collected data using a machine learning algorithm to calculate optimal device operating conditions; means for transmitting control instructions to various IoT devices based on the analysis results; means for providing a user interface for displaying the analysis results and the control instructions to the user; means for analyzing data accumulated over a long period of time on a large scale and identifying trends and patterns; means including a terminal for transmitting data collected from each IoT device to the server; and means for the server to generate and transmit control instructions to each IoT device. This enables integrated analysis of various data and rapid determination of optimal operating conditions. This significantly improves energy efficiency and realizes a sustainable living environment.

[1414] An "IoT device" is an electronic device that can connect to the Internet and collect, send, and receive data.

[1415] "Operational status data" refers to information about the operating status and behavior of IoT devices, including data such as temperature settings, power consumption, and operating time.

[1416] "Environmental data" refers to information about the state of the environment in which an IoT device is located, including, for example, outside temperature, humidity, and weather information.

[1417] A "machine learning algorithm" refers to a mathematical method or model that analyzes large amounts of data to learn patterns and rules and make predictions and classifications.

[1418] "Control instructions" refer to commands and configuration change instructions sent from a server to an IoT device, and are used to optimize the device's operation.

[1419] "User interface" refers to the software and hardware components that provide the visual and operational means for a user to interact with a system.

[1420] "Trends and patterns" refer to regularities and tendencies extracted from data collected over a long period of time, and are used to predict and optimize the future.

[1421] "Terminal" refers to an electronic device that functions as a relay point for transmitting data collected from IoT devices to a server.

[1422] "Server" refers to a central management device that collects and analyzes data, and generates and sends control instructions.

[1423] This invention provides an "IoT AI system" that works in conjunction with multiple IoT devices to achieve optimal operation and improve energy efficiency. This system is primarily composed of the following modules: a data collection module, a data analysis module, a control module, a user interface module, and a big data analysis module.

[1424] First, let's explain the data collection module. The server periodically collects operating status data from each IoT device. This data includes temperature settings, power usage, and operating hours. Environmental data is also simultaneously collected through outdoor temperature sensors and weather forecast APIs. The terminals send the operating status data collected from the IoT devices to the server. As a concrete example, the server receives data from an air conditioner that "the current room temperature is 25°C, the set temperature is 22°C," and obtains data from the weather forecast API that "the current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C."

[1425] Next, we will explain the data analysis module. The server uses the collected data to run machine learning algorithms and calculate the optimal device operating conditions. In this process, the server learns the efficient operating patterns of each IoT device and derives the settings that are most suitable for the user's lifestyle and environmental conditions. As a concrete example, the server concludes that "it is optimal to change the air conditioner's temperature setting to 23°C."

[1426] Next, we will explain the control module. The server generates and sends control instructions to each IoT device based on the analysis results. The terminal receives the control instructions from the server and changes the device settings accordingly. As a concrete example, the server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends this instruction to the air conditioner. The air conditioner changes the set temperature to 23°C.

[1427] Next, we will explain the user interface module. The server generates a dashboard to display the analysis results and control instructions in an easy-to-understand manner to the user. This dashboard can be accessed by the user via a smartphone or PC. The user can use the dashboard to check the current device status and control settings and change them as necessary. As a concrete example, the user receives a notification on their smartphone saying, "The air conditioner's set temperature has been changed to 23°C," and confirms this.

[1428] Finally, we will explain the big data analysis module. The server collects daily data and accumulates the analysis results over the long term, saving them as big data. This big data is used to further analyze trends and patterns and create future prediction models. As a concrete example, the server accumulates household energy consumption data on an annual basis, analyzes seasonal usage patterns, and suggests that "it would be more energy-efficient to not lower the temperature setting any further in winter."

[1429] For example, when a user goes out during the day in summer, the system analyzes the outside temperature, the temperature inside the house, and the weather forecast, and automatically adjusts the air conditioner's temperature setting. Even if the outside temperature is high and the house is predicted to heat up quickly, the system maintains the optimal setting and maximizes energy efficiency by returning the room to an appropriate temperature before the user returns home. Long-term data analysis also identifies energy consumption trends in specific regions and seasons, and performs optimization based on predictions. In this way, the IoT AI system improves user convenience and reduces energy waste. As a result, it is possible to reduce environmental impact and aim to create a sustainable living environment.

[1430] An example of a prompt for a generative AI model is, "Please explain what specific processing the IoT AI system performs at each step. As an example, please show a process that includes how to optimize the air conditioner's temperature setting."

[1431] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1432] Step 1:

[1433] Starting Data Collection

[1434] The server periodically sends data collection requests to each IoT device. These requests include instructions for operational status data such as temperature setting, power usage, and operating time. The terminal receives these requests and collects the necessary data from each IoT device. As a specific example, the terminal obtains data from an air conditioner that "the current room temperature is 25°C, and the set temperature is 22°C." The input is a data collection request from the server, and the output is operational status data from each IoT device.

[1435] Step 2:

[1436] Obtaining environmental data

[1437] The server obtains environmental data through an external temperature sensor or weather forecast API. Specifically, it obtains the current outside temperature and weather information. For example, the server uses a weather forecast API to obtain data such as "The current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C." The input is a request from an external sensor or API, and the output is environmental data.

[1438] Step 3:

[1439] Data transmission

[1440] The device sends the collected operational status data to the server. The server stores the environmental data obtained from the weather forecast API. Specifically, the device sends data to the server such as "current room temperature is 25°C, set temperature is 22°C." The input is the operational status data from the device, and the output is the operational status data and environmental data stored on the server.

[1441] Step 4:

[1442] Data integration

[1443] The server integrates the operational status data and environmental data from each IoT device. The input is the operational status data and environmental data, and the output is an integrated data set.

[1444] Step 5:

[1445] Running machine learning algorithms

[1446] The server uses the integrated data to run a machine learning algorithm and calculate the optimal device operating conditions. This process involves data preprocessing, feature extraction, and model training. For example, the server may conclude that "setting the air conditioner temperature to 23°C is optimal." The input is the integrated data set, and the output is the optimal operating conditions.

[1447] Step 6:

[1448] Control instruction generation

[1449] The server generates control instructions for each IoT device based on the analysis results. Specifically, the server generates an instruction to "change the air conditioner's set temperature to 23°C." The input is the optimal operating conditions, and the output is the generated control instruction.

[1450] Step 7:

[1451] Sending control instructions

[1452] The server sends the generated control instructions to each IoT device. The terminal receives the control instructions from the server and changes the device settings accordingly. Specifically, the server sends the instruction "change the air conditioner's set temperature to 23°C," and the air conditioner changes its set temperature. The input is the generated control instruction, and the output is the setting change of the IoT device that received the control instruction.

[1453] Step 8:

[1454] Generate a dashboard

[1455] The server generates a dashboard to display the analysis results and control instructions in an easy-to-understand manner for the user. The inputs are the analysis results and control instructions, and the output is the generated dashboard.

[1456] Step 9:

[1457] Accessing the User Interface

[1458] Users access the dashboard using a smartphone or PC to check the current device status and control settings. Specifically, the user receives a notification on their smartphone saying, "The air conditioner temperature setting has been changed to 23°C," and confirms it. The input is the dashboard URL and app, and the output is status information that the user can check.

[1459] Step 10:

[1460] Accumulation and analysis of big data

[1461] The server collects daily data and accumulates the analysis results over the long term, saving them as big data. Data accumulated over a long period of time is analyzed on a large scale to identify trends and patterns. Specifically, the server accumulates household energy consumption data on an annual basis and suggests, "It would be more energy-efficient to not lower the temperature setting any further in winter." The input is the daily collected data and analysis results, and the output is the accumulated big data and its analysis results.

[1462] (Application example 1)

[1463] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1464] Modern self-driving vehicles operate using numerous information and communication technology devices, but it is difficult to efficiently manage the operating status of these devices and external environmental data, and calculate optimal operating conditions in real time. Furthermore, systems that automatically set optimal routes and speeds taking into account traffic information and weather conditions have not yet been fully realized. As a result, there are problems with wasted energy consumption and reduced operating efficiency.

[1465] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1466] In this invention, the server includes means for collecting operation status data from various information and communication technology devices, means for acquiring external environment data, means for analyzing the collected data using a machine learning algorithm and calculating optimal device operating conditions, means for sending control instructions to the various information and communication technology devices based on the analysis results, means for providing a user interface for displaying the analysis results and the content of the control instructions to the user, means for analyzing data accumulated over a long period of time on a large scale and identifying trends and patterns, means for collecting and analyzing vehicle status and traffic information to calculate optimal driving routes and speed settings, and means for sending control instructions to the autonomously driven vehicle based on the analysis results, thereby optimizing the operating conditions of the autonomously driven vehicle and enabling improved energy efficiency and operating efficiency.

[1467] "Information and communication technology equipment" means electronic devices for data collection, communication, control, etc.

[1468] "Operation status data" refers to information relating to the operating status of the device, such as its usage status, power consumption, temperature, and usage time.

[1469] "External environment data" is information about surrounding conditions such as weather, temperature, humidity, and traffic conditions.

[1470] A "machine learning algorithm" is a mathematical method for learning patterns and relationships from data and making predictions and classifications.

[1471] A "control instruction" is an instruction to change or adjust the operation of a device.

[1472] A "user interface" is the screen or method of operation that allows a user to interact with a system.

[1473] "Trends and patterns" are tendencies and regularities that emerge from data collected over a long period of time.

[1474] An "autonomous vehicle" is a vehicle that operates autonomously without the need for human operation.

[1475] A "travel route" is a specific route that a vehicle will take to reach its destination.

[1476] "Speed ​​setting" is a parameter that determines the vehicle's traveling speed.

[1477] This invention relates to an "Autonomous Vehicle AI Optimization Commander" system that works in conjunction with multiple information and communication technology devices to optimize operation and improve energy efficiency. This enables vehicle operation to be optimized and energy efficiency to be improved. The system is primarily composed of the following modules: a data collection module, a data analysis module, a control module, a user interface module, and a big data analysis module.

[1478] Explanation of the specific processing of the system

[1479] 1. Data Collection Module

[1480] The server periodically collects operational status data from the autonomous vehicle, including remaining battery power, driving speed, distance to destination, etc. It also simultaneously obtains environmental data from traffic information APIs and weather forecast APIs.

[1481] The terminal transmits operation-related data, including traffic and weather information, to the server.

[1482] Example: A server receives data from an autonomous vehicle, such as "Current battery remaining is 80%, driving speed is 60km / h, distance to destination is 20km." At the same time, it obtains data from a traffic information API that "Congestion is occurring" and from a weather forecast API that "Current weather is rain."

[1483] 2. Data Analysis Module

[1484] The server analyzes the collected data using machine learning algorithms to calculate the optimal vehicle route and speed. In this process, the server learns the optimal driving pattern based on data obtained from various information and communication technology devices, and derives the settings that are most suitable for real-time environmental conditions.

[1485] Example: The server performs an analysis and concludes, "Change driving speed to 55 km / h and re-route to avoid traffic jams."

[1486] 3. Control Module

[1487] The server generates control instructions for the autonomous vehicle based on the analysis results, which reflect the optimal driving route and speed settings.

[1488] The terminal receives control instructions from the server and changes the settings of the autonomous vehicle accordingly.

[1489] Example: The server generates an instruction to "change driving speed to 55 km / h and re-plan the route" and sends this instruction to an autonomous vehicle. The autonomous vehicle changes its driving speed to 55 km / h and re-plans the route.

[1490] 4. User Interface Module

[1491] The server generates a dashboard that displays the analysis results and control instructions in an easy-to-understand format for users, and users can access this dashboard via their smartphones or PCs.

[1492] The dashboard allows users to view the current vehicle status and control settings and change them if necessary.

[1493] Example: A user receives a notification on their smartphone that says "Your driving speed has been changed to 55km / h" and confirms it.

[1494] 5. Big Data Analysis Module

[1495] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[1496] Example: A server accumulates energy consumption data of autonomous vehicles on a yearly basis, analyzes seasonal usage patterns, and suggests that "it would be more energy-efficient to not reduce the speed any further in winter."

[1497] Prompt Sentence Examples

[1498] As a concrete example, the following prompt sentences can be input to a generative AI model:

[1499] "Vehicle ID 12345's current speed is 60 km / h, battery remaining is 80%, distance to destination is 20 km, there is traffic congestion information, and it is raining. Please predict the optimal driving speed under these conditions."

[1500] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1501] Step 1:

[1502] Data collection

[1503] The server collects operational status data from the autonomous vehicles (remaining battery level, driving speed, distance to destination, etc.) and simultaneously obtains environmental data (traffic conditions, weather information) from traffic information APIs and weather forecast APIs.

[1504] Input: Data from autonomous vehicles, traffic information API, weather forecast API

[1505] Output: Collected data (remaining battery level, driving speed, distance to destination, traffic conditions, weather information)

[1506] Specific operation: The server periodically connects to the autonomous vehicle to obtain operating status data, and at the same time calls the API to obtain external environment data.

[1507] Step 2:

[1508] Data analysis

[1509] The server then inputs the collected data into a machine learning algorithm to calculate the optimal route and speed, taking into account the vehicle's remaining battery power, current speed, distance to destination, traffic conditions, weather information, and more.

[1510] Input: Collected data (battery level, driving speed, distance to destination, traffic conditions, weather information)

[1511] Output: Optimal driving path and speed settings

[1512] Specific operation: The server uses a machine learning model (e.g., a Linear Regression model) to analyze the input data and calculate the optimal driving route and speed.

[1513] Step 3:

[1514] Control instruction generation

[1515] The server generates control instructions for the autonomous vehicle based on the results of the data analysis, including instructions on optimal driving routes and speed settings.

[1516] Input: Data analysis results (optimal driving route and speed settings)

[1517] Output: Control instructions for the autonomous vehicle

[1518] Specific operation: The server executes a control algorithm based on the analysis results and generates control instructions for the autonomous vehicle.

[1519] Step 4:

[1520] Sending control instructions

[1521] The server then sends the generated control instructions to the autonomous vehicle, which then adjusts its route and speed based on these instructions.

[1522] Input: Generated control instructions

[1523] Output: The behavior of the autonomous vehicle based on the executed control instructions.

[1524] Specific operation: The server sends instructions to the autonomous vehicle to adjust its speed and route, and the autonomous vehicle carries out the instructions.

[1525] Step 5:

[1526] Display in the user interface

[1527] The server generates a user interface to display the analysis results and control instructions to the user, who can then check them on their smartphone or PC.

[1528] Input: Analysis results, control instructions

[1529] Output: What is displayed on the user interface

[1530] Specific operation: The server generates a dashboard and sends information to a smartphone or PC to display the analysis results and control instructions.

[1531] Step 6:

[1532] Big data accumulation and analysis

[1533] The server collects daily data and accumulates the analysis results over the long term, saving them as big data. This big data can then be used to analyze more detailed trends and patterns.

[1534] Input: Daily data collection results, analysis results

[1535] Output: Long-term trends and patterns

[1536] Specific operation: The server periodically analyzes the accumulated data and updates the future prediction model.

[1537] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1538] This invention relates to an "IoT AI Commander" system that works in conjunction with multiple IoT devices to achieve optimal operation and improve energy efficiency. This system aims to provide a more user-friendly and comfortable environment by combining it with an emotion engine that recognizes the user's emotions. The specific processing of the system is described below.

[1539] Explanation of the specific processing of the system

[1540] 1. Data Collection Module

[1541] The server periodically sends requests to collect operational status data from each IoT device, including the device's operating status, configuration information, power consumption, and operating time.

[1542] The terminal periodically sends operational status data of the IoT device to the server.

[1543] Example: An air conditioner sends data to the server saying, "The current room temperature is 25°C, and the set temperature is 22°C."

[1544] 2. Emotion Recognition by Emotion Engine

[1545] The server recognizes emotions by analyzing the user's facial expressions and tone of voice, using cameras and microphones to collect data in real time.

[1546] Example: If a user is in their living room and has a relaxed expression, the emotion engine will recognize that expression and determine that they are "relaxed."

[1547] 3. Data integration and preprocessing

[1548] The server integrates the operational status data of IoT devices, environmental data (outside temperature, weather forecast, etc.), and emotion data, and performs preprocessing, which includes data cleansing, missing value completion, and format standardization.

[1549] 4. Analysis using machine learning algorithms

[1550] The server then uses machine learning algorithms to analyze the preprocessed data, with the goal of calculating the optimal operating conditions for each IoT device.

[1551] Example: As a result of the analysis, the conclusion is reached that "it is optimal to change the air conditioner's set temperature to 23°C."

[1552] 5. Integrated analysis with emotion data

[1553] The server further adjusts the device's operating conditions based on the user's emotional data obtained by the emotion engine. If the user is feeling stressed, the server may make additional adjustments, such as changing the lighting to a warmer color and playing music.

[1554] Example: If the user is determined to be stressed, not only will the air conditioner temperature be changed to 23°C, but the color temperature of the lighting will be changed to 3000K (warm color) to create a relaxing environment.

[1555] 6. Generating and Sending Control Instructions

[1556] The server generates the necessary control instructions based on the analysis results and sends them to each IoT device.

[1557] The terminal receives control instructions from the server and changes the device settings accordingly.

[1558] Example: The server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends it to the air conditioner. The air conditioner changes the set temperature to 23°C.

[1559] 7. User Interface

[1560] The server generates a dashboard to display the analysis results and control instructions to users. Users can access the dashboard on their smartphones or PCs to check and change the current device status and control settings.

[1561] Users can monitor the operation of IoT devices based on the information displayed on the dashboard and manually change settings as needed.

[1562] Example: A user sees a notification on their smartphone that says "The air conditioner temperature setting has been changed to 23°C" and manually adjusts the temperature setting further if necessary.

[1563] 8. Big Data Analysis

[1564] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[1565] Example: Analyzing energy consumption data for each season (summer and winter) and proposing more efficient operation methods.

[1566] In this way, the IoT AI Commander improves user convenience and reduces energy waste while taking emotions into consideration, ultimately reducing the environmental impact and helping to create a sustainable living environment.

[1567] The processing flow will be explained below.

[1568] Step 1:

[1569] The server periodically sends requests to collect operational status data from each IoT device, prompting each device to send its current operating status, configuration information, power consumption, operating time, and other information.

[1570] Step 2:

[1571] The terminal receives a request from the server and transmits the operational status data of each IoT device to the server.

[1572] Example: An air conditioner sends information to the server that "the current room temperature is 25°C, and the set temperature is 22°C."

[1573] Step 3:

[1574] The server accesses external environmental data sources (e.g., weather forecast APIs or external sensors) to obtain environmental data such as outside temperature, humidity, and weather forecasts.

[1575] Example: The server retrieves data from a weather forecast API: "The current outside temperature is 30°C, the weather is sunny, and today's maximum temperature is 32°C."

[1576] Step 4:

[1577] The server runs an emotion engine to obtain emotion data by analyzing the user's facial expressions and voice. Data is collected in real time via cameras and microphones.

[1578] Example: A camera and microphone analyze a user's facial expressions and tone of voice to determine whether they are relaxed or stressed.

[1579] Step 5:

[1580] The server integrates the collected operational status data, environmental data, and emotion data and performs preprocessing, which cleanses the data, fills in missing values, and standardizes the format.

[1581] Step 6:

[1582] The server uses machine learning algorithms to analyze the preprocessed data and calculate the optimal operating conditions for each IoT device.

[1583] Example: After analysis, you conclude that the optimal temperature setting for an air conditioner is 23°C.

[1584] Step 7:

[1585] The server optimizes the analysis results by integrating the emotional data, generating different control instructions depending on whether the user is relaxed or stressed.

[1586] Example: If the user is determined to be stressed, the system will instruct the user to change the air conditioner temperature setting to 23°C, change the lighting color temperature to warmer colors, and play relaxing music.

[1587] Step 8:

[1588] The server then sends the generated control instructions to each IoT device, which then changes their behavior in accordance with the new settings.

[1589] Step 9:

[1590] The terminal receives control instructions from the server and changes the settings of the corresponding IoT device.

[1591] Example: The air conditioner changes the set temperature to 23°C, the smart lighting changes the color temperature to 3000K (warm), and the music player starts playing relaxing music.

[1592] Step 10:

[1593] The server generates a dashboard to display the analysis results and control instructions to the user, which can be viewed on a smartphone or PC.

[1594] Step 11:

[1595] Users can use the dashboard to view current device status and control settings, and manually change settings if necessary.

[1596] Example: A user sees a notification on their smartphone that says "The air conditioner temperature setting has been changed to 23°C" and then changes the temperature setting to 25°C.

[1597] Step 12:

[1598] The server accumulates daily data over the long term and stores it as big data. This data is used to analyze trends and patterns and create predictive models for the future.

[1599] Example: Analyzing past data, understanding seasonal energy consumption patterns, and proposing optimal energy consumption plans.

[1600] The above is the specific processing flow of the "IoT AI Commander" system, which combines user emotional data. This system improves user convenience and comfort while reducing energy waste and realizing a sustainable living environment.

[1601] Example 2

[1602] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1603] Modern IoT devices collect large amounts of data, improving user convenience. However, analyzing the data and calculating optimal device operating conditions requires advanced technology. Furthermore, existing systems do not adequately consider user emotions and comfort, creating challenges in improving the user experience. Furthermore, there is a need for more methods to utilize collected data over the long term to optimize energy efficiency and realize sustainable living environments.

[1604] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1605] In this invention, the server includes means for collecting operation status data from various IoT devices, means for acquiring external environmental data, means for analyzing the collected data using a machine learning algorithm and calculating optimal device operating conditions, means for sending control instructions to the various IoT devices based on the analysis results, means for providing a user interface for displaying the analysis results and the control instructions to the user, means for analyzing data accumulated over a long period of time on a large scale and identifying trends and patterns, means for analyzing the user's facial expressions and tone of voice to collect emotion data, and means for adjusting the IoT device operating conditions based on the emotion data. This makes it possible to calculate optimal device operating conditions taking user emotions into consideration, improve energy efficiency, and realize a sustainable living environment.

[1606] "IoT devices" is a general term for devices that are connected to the Internet and can collect and control data by communicating with each other.

[1607] "Operational status data" refers to information about the operation of an IoT device, such as its operating status, settings, power consumption, and operating time.

[1608] "Environmental data" refers to information about the external environment, such as outside temperature, humidity, and weather forecasts.

[1609] "Machine learning algorithms" refers to technologies that include mathematical models and statistical methods for analyzing collected data and calculating optimal equipment operating conditions.

[1610] "Control instructions" refer to specific operational commands sent to IoT devices to achieve optimal device operating conditions.

[1611] "User interface" refers to the display screen and operating means that allow a user to interact with the system and check or change the system's status.

[1612] "Large-scale analysis" refers to techniques and methods that use large amounts of data accumulated over a long period of time to identify trends and patterns.

[1613] "Emotion data" refers to information that indicates the user's emotional state, obtained from the user's facial expression, tone of voice, etc.

[1614] "Trend" or "pattern" refers to a statistical tendency or recurring characteristic derived from collected data.

[1615] "Operating conditions" refers to the settings and operating modes required for optimal operation of IoT devices.

[1616] A "sustainable living environment" refers to a sustainable living space that improves energy efficiency, reduces waste, and reduces the environmental impact.

[1617] This invention relates to an "IoT AI Commander" system that aims to achieve optimal operation and improved energy efficiency by linking multiple IoT devices. This system aims to provide a more user-friendly and comfortable environment by combining it with an emotion engine that recognizes the user's emotions.

[1618] The system's main components are a server, a device, and a user. The server collects operational status data from various IoT devices and acquires external environmental data, then analyzes all collected data using a machine learning algorithm. Based on the analysis results, it sends control instructions to the IoT devices and displays them to the user through a user interface. In addition, an emotion engine analyzes the user's facial expressions and tone of voice, collects emotional data, and adjusts the operating conditions of the IoT devices. This increases user comfort and improves energy efficiency.

[1619] Hardware and Software Examples

[1620] Servers: High-performance API servers, database management systems (e.g., MySQL), machine learning frameworks (e.g., TensorFlow)

[1621] Terminal: IoT device (e.g. smart speaker, smart air conditioner, smart light)

[1622] Emotion engine: Image and speech recognition technologies (e.g., OpenCV, Google Speech-to-Text API)

[1623] Explanation of program processing

[1624] Data collection

[1625] The server periodically collects operational status data from various IoT devices. Specifically, it sends API requests and receives data such as operating status, configuration information, and power consumption from the IoT devices. The terminal is responsible for sending operational status data to the server and providing the specific operating status of air conditioners and lighting.

[1626] Example: An air conditioner sends the following data to the server: "The current room temperature is 25°C, the set temperature is 22°C, and the power consumption is 1.2kW."

[1627] Example prompt: "Please report your current air conditioning settings and operating status."

[1628] Collecting Emotional Data

[1629] The server analyzes the user's facial expressions and tone of voice in real time through a camera and microphone. The emotion engine then sends the collected emotional data to the server.

[1630] Example: The server analyzes video data from a camera to detect whether the user is laughing, and also determines from the tone of voice that the user is relaxed.

[1631] Example prompt: "Analyze the user's facial expression and tone of voice and report their current emotions."

[1632] Data preprocessing and integrated analysis

[1633] The server integrates the collected IoT device data and emotion data, cleansing the data, filling in missing values, and standardizing the format. It then analyzes the data using machine learning algorithms to calculate the optimal operating conditions for the IoT device. The analysis results and emotion data are integrated to generate the final control instructions.

[1634] Example: A machine learning model calculates the optimal temperature setting for an air conditioner as 23°C. If the user is relaxing, change the color temperature of the lights to a warmer color and play music.

[1635] Example prompt: "Analyze the data and calculate the optimal temperature setting for your air conditioner."

[1636] Generating and sending control instructions

[1637] The server generates specific control instructions based on the analysis results and sends them to the IoT device. The device receives the control instructions from the server and changes the device settings accordingly.

[1638] Example: The server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends it to the air conditioner. The air conditioner changes the set temperature to 23°C.

[1639] Example prompt: "Please tell the air conditioner to change the temperature setting to 23°C."

[1640] Providing a user interface

[1641] The server generates a dashboard to display the analysis results and control instructions to the user, providing a user interface. Users can access the dashboard on their smartphones or PCs to check and change the current device status and control settings.

[1642] Example: A user sees a notification on their smartphone that says "The air conditioner temperature setting has been changed to 23°C" and manually adjusts the temperature setting further if necessary.

[1643] Example prompt: "Show me the current air conditioner temperature setting and operating status."

[1644] Big Data Analysis

[1645] The server collects daily data and accumulates the analysis results over the long term, saving them as big data. This big data is used to analyze trends and patterns and create future prediction models.

[1646] Example: A server analyzes historical energy consumption data and proposes new algorithms to improve energy efficiency.

[1647] Example prompt: "Based on historical energy consumption data, create a proposal to improve the operational efficiency of your air conditioner."

[1648] Through the above process, this system optimizes energy usage while maintaining user comfort, providing a sustainable living environment.

[1649] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1650] Step 1: Collecting IoT device data

[1651] The server sends a request to each IoT device to collect operational status data.

[1652] Input: Data collection request sent from the server to the IoT device.

[1653] Output: Operational status data sent from IoT devices (e.g., room temperature, set temperature, power consumption).

[1654] Specific operation: The server sends a request to the air conditioner, and the air conditioner replies with the data "Current room temperature is 25°C, set temperature is 22°C, power consumption is 1.2kW."

[1655] Step 2: Obtaining environmental data

[1656] The server obtains external environmental data (e.g., outside temperature, weather forecast).

[1657] Input: Real-time environmental data that the server retrieves from external data sources.

[1658] Output: The captured environmental data.

[1659] Specific operation: The server accesses the weather API to obtain the current outside temperature and weather forecast.

[1660] Step 3: Collecting emotion data

[1661] The server collects the user's facial expressions and tone of voice in real time through a camera and microphone.

[1662] Input: Video data from the camera and audio data from the microphone.

[1663] Output: Emotion data indicating the user's emotions.

[1664] How it works: The server analyzes the camera footage to detect whether the user is smiling and also determines from the tone of their voice that they are relaxed.

[1665] Step 4: Preprocessing the data

[1666] The server integrates and preprocesses data from IoT devices, environmental data, and emotion data.

[1667] Input: Operational status data from IoT devices, external environmental data, and user emotion data.

[1668] Output: Preprocessed integrated data.

[1669] Specific operation: The server fills in missing values ​​of each data with the average value and unifies the data format.

[1670] Step 5: Machine learning analysis

[1671] The server uses machine learning algorithms to perform analysis on the pre-processed data.

[1672] Input: Preprocessed integrated data.

[1673] Output: Optimal operating conditions for IoT devices.

[1674] What it does: A machine learning model calculates the optimal temperature setting for an air conditioner: 23°C. The algorithm then derives settings that minimize energy consumption and maximize user comfort.

[1675] Step 6: Integrated analysis with emotion data

[1676] The server readjusts the analysis results by taking into account the emotion data obtained from the emotion engine.

[1677] Input: Analysis results from machine learning algorithms, emotion data.

[1678] Output: Optimal operating conditions taking emotions into account.

[1679] Specific behavior: If the user is determined to be relaxed, the air conditioner temperature setting is changed to 23°C and the lighting color temperature is changed to 3000K (warm color).

[1680] Step 7: Generate and send control instructions

[1681] The server generates control instructions based on the analysis results and sends them to each IoT device.

[1682] Input: Optimal operating conditions taking emotions into account.

[1683] Output: Specific control instructions for each IoT device.

[1684] Specific operation: The server generates an instruction to "change the air conditioner's set temperature to 23°C" and sends it to the air conditioner. The air conditioner changes the set temperature to 23°C.

[1685] Step 8: Update the User Interface

[1686] The server displays the analysis results and the contents of the control instructions on a dashboard and provides them to the user.

[1687] Input: Control instructions and analysis results.

[1688] Output: The dashboard content that is displayed to the user.

[1689] Specific behavior: The user sees a notification on their smartphone that says "The air conditioner temperature setting has been changed to 23°C" and adjusts it as needed.

[1690] Step 9: Analyze the big data

[1691] The server collects daily data and stores the analysis results over the long term, and performs large-scale analysis.

[1692] Input: Data accumulated over a long period of time.

[1693] Output: Analysis results that identify trends and patterns.

[1694] Specific operation: The server analyzes past energy consumption data and proposes new algorithms to improve energy efficiency.

[1695] Through these steps, the system optimizes energy consumption and realizes sustainable living while maintaining user comfort.

[1696] (Application example 2)

[1697] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1698] In existing IoT environments, it is difficult to set optimal device operation settings that take user emotions into account, which means that users cannot always enjoy comfort and convenience.In addition, the effective use of collected data and the optimization of long-term energy consumption are not sufficiently implemented, making it difficult to achieve a sustainable living environment.

[1699] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting operation status data from various sensing devices, means for acquiring external environmental information, means for analyzing the collected data and user emotion data using a machine learning algorithm and calculating optimal device operating conditions, means for sending control instructions to various devices based on the analysis results, means for providing a user interface for displaying the analysis results and the content of the control instructions to the user, means for analyzing data accumulated over a long period of time on a large scale and identifying trends and patterns, means for recognizing emotions by analyzing the user's facial expressions and tone of voice, and means for adjusting device operation settings based on the emotion data. This makes it possible to provide a comfortable environment that is in tune with the user's emotions, while optimizing energy consumption and realizing a sustainable living environment.

[1700] A "sensing device" is a device that detects information about the state of the environment or an object and collects that data.

[1701] "Environmental information" refers to data about the external environment, such as temperature, humidity, weather, and illuminance.

[1702] "Device" refers to various equipment that collects operational status data and is controlled in conjunction with external systems.

[1703] "Emotion data" is information about the user's psychological state obtained by analyzing the user's facial expressions and tone of voice.

[1704] A "machine learning algorithm" is a computational method for building patterns and predictive models based on collected data and calculating optimal operating conditions.

[1705] A "user interface" is a system that allows users to visually check and operate analysis results and control instructions.

[1706] "Trend" refers to a tendency or pattern of fluctuation that can be found by analyzing collected data over a long period of time.

[1707] "Data collection means" refers to the function for collecting necessary data from various sensing devices.

[1708] "External information acquisition means" refers to a function for acquiring data related to the external environment.

[1709] "Analysis means" refers to the function of analyzing collected data using machine learning algorithms to derive optimal operating conditions.

[1710] "Control instruction sending means" refers to a function for sending instructions to various devices based on the analysis results.

[1711] "Emotion recognition means" refers to a function for recognizing emotions by analyzing the user's facial expressions and tone of voice.

[1712] "Operation setting adjustment means" refers to a function for adjusting the operation settings of a device based on emotion data.

[1713] A system for implementing the present invention is configured as follows.

[1714] Specific system configuration

[1715] 1. Data Collection Module

[1716] The server periodically collects operational status data from various sensing devices, including the operating status, configuration information, power consumption, and operating time of each device.

[1717] The terminal periodically sends operational status data of the IoT device to the server.

[1718] Example: A lighting system sends data to a server saying "Current brightness is 500 lumens, set brightness is 400 lumens."

[1719] 2. Environmental Information Acquisition Module

[1720] The server acquires external environmental information (temperature, humidity, weather information, etc.).

[1721] Example: Obtain information from a weather data provider such as "The current outside temperature is 28°C and the humidity is 70%."

[1722] 3. Emotion Recognition Module

[1723] The server analyzes the user's facial expressions and tone of voice to recognize their emotions, using cameras and microphones to collect data in real time.

[1724] Example: A camera in a store captures the user's facial expression and determines that they are "relaxed."

[1725] 4. Analysis Module

[1726] The server uses machine learning algorithms to analyze the collected data and calculate the optimal operating conditions for the device.

[1727] Example: Analysis results in the conclusion that "adjusting the lighting brightness to 420 lumens is optimal."

[1728] 5. Control instruction transmission module

[1729] The server generates the necessary control instructions based on the analysis results and sends them to the various devices.

[1730] The terminal receives control instructions from the server and changes the device settings accordingly.

[1731] Example: The server generates an instruction to "change the lighting brightness to 420 lumens" and sends it to the lighting system. The lighting system changes the set brightness to 420 lumens.

[1732] 6. User Interface

[1733] The server provides an interface for displaying the analysis results and control instructions to the user. Users can access the interface on their smartphones or PCs to check and change the current device status and settings.

[1734] Based on the information displayed in the interface, the user can monitor the device's operation and manually change settings if necessary.

[1735] Example: A user sees a notification on their smartphone that says "Light setting brightness changed to 420 lumens" and manually adjusts the setting if necessary.

[1736] 7. Big Data Analysis Module

[1737] The server collects and analyzes data daily and stores it as big data over the long term. This big data is then used to analyze further trends and patterns and create future prediction models.

[1738] Example: Analyzing seasonal energy consumption data and proposing more efficient operation methods.

[1739] 8. Operational Settings Adjustment Module

[1740] The server adjusts the device's operational settings based on the user's emotional data. If the user is feeling stressed, additional adjustments are made, such as changing the lighting to warmer colors and playing music.

[1741] Example: If the user is determined to be stressed, the color temperature of the lighting is changed to 3000K (warm color) to create a relaxing environment.

[1742] Specific examples

[1743] 1. While customers are browsing products in a store, smart glasses analyze their facial expressions and, if they are smiling, change the background music to a brighter tone.

[1744] 2. If customers seem stressed, reset the store's air conditioning to a comfortable temperature and change the lighting to a warmer color.

[1745] Prompt Sentence Examples

[1746] "Please provide a blueprint for an application that uses smart glasses and IoT devices to optimize the environment in a physical store according to the customer's emotions. As a specific example, please include a scenario in which cheerful music is played for smiling customers, and the air conditioning and lighting are adjusted for stressed customers."

[1747] The above is a detailed example of how the "IoT AI Commander" can be applied in a brick-and-mortar store. By using this system, it is possible to provide a comfortable environment that is sensitive to the user's emotions, while optimizing energy consumption and realizing a sustainable living environment.

[1748] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1749] Step 1:

[1750] Data collection

[1751] The server collects operational status data from various sensing devices. Input data includes the operating status, setting information, power consumption, and operating time of each device. This data is collected and stored in the server's database. Specifically, the lighting system sends data such as "current brightness is 500 lumens, set brightness is 400 lumens."

[1752] Step 2:

[1753] Obtaining environmental information

[1754] The server obtains external environmental information (temperature, humidity, weather information, etc.) from various APIs. Input data includes weather data and data from environmental monitors, which are collected and used for analysis. Specifically, it obtains information such as "The current outside temperature is 28°C and humidity is 70%" from a weather data provider.

[1755] Step 3:

[1756] Collecting Emotional Data

[1757] The device collects the user's facial expressions and tone of voice through a camera and microphone, and converts them into emotional data using an emotion recognition model (using, for example, TensorFlow and Keras). The input data is video and audio data, and the output is the user's emotional state (relaxed, stressed, etc.). Specifically, the device captures the user's facial expressions with a camera inside the store and determines that the user is "relaxed."

[1758] Step 4:

[1759] Data Preprocessing

[1760] The server integrates the collected operational status data, environmental information, and emotion data, and performs preprocessing such as data cleansing, missing value completion, and data format unification. The input data is the data collected in steps 1 to 3, and the output is a clean, unified, and analyzable dataset.

[1761] Step 5:

[1762] Analysis and Optimization

[1763] The server analyzes the preprocessed data using machine learning algorithms (e.g., random forest, regression, etc.) to calculate the optimal device operating conditions. The input data is the preprocessed data, and the output is specific operating instructions (e.g., adjust the lighting brightness to 420 lumens).

[1764] Step 6:

[1765] Generating and sending control instructions

[1766] The server generates control instructions based on the analysis results and sends them to various devices. The input data are optimized operation instructions, and the output is specific operation instructions for each device. Specifically, an instruction such as "change the lighting brightness to 420 lumens" is generated and sent to the lighting system.

[1767] Step 7:

[1768] Providing a user interface

[1769] The server provides an interface for displaying the analysis results and control instructions to the user. The input data are the analysis results and control instructions, and the output is a dashboard and notifications that the user can access. The user can check the current device status and settings from their smartphone or PC and manually adjust them as necessary. Specifically, the user receives a notification on their smartphone that "the lighting setting brightness has been changed to 420 lumens."

[1770] Step 8:

[1771] Big Data Analysis

[1772] The server collects daily data and stores the analysis results over the long term, and performs big data analysis. The input data is data collected over a long period of time, and the output is the analysis results on trends and patterns. This data is used to create future prediction models that are useful for proposing energy efficiency improvements. Specifically, it analyzes energy consumption data by season and proposes more efficient operating methods.

[1773] Step 9:

[1774] Adjusting Operation Settings

[1775] The server adjusts the device's operational settings based on the user's emotional data. The input data is the result of emotion recognition, and the output is the emotion-based operational settings. Specifically, if the user is feeling stressed, the server adjusts the lighting color temperature to 3000K (warm color) to create a relaxing environment.

[1776] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1777] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemin...

Claims

1. A means of collecting operational status data from various IoT devices, a means for acquiring external environmental data; A means for analyzing the collected data using machine learning algorithms to calculate optimal operating conditions for the equipment; A means for sending control instructions to various IoT devices based on the analysis results; means for providing a user interface for displaying the analysis results and the contents of the control instructions to a user; A means of analyzing large amounts of data accumulated over long periods of time to identify trends and patterns; A system including:

2. 10. The system of claim 1, further comprising means for real-time data collection, analysis and control.

3. 2. The system according to claim 1, further comprising means for optimizing energy consumption in the home based on the collected data and analysis results, thereby realizing a sustainable living environment.

Images