system

Abstract

We provide the system. [Solution] Means for collecting household energy consumption data, A means for analyzing usage patterns based on the aforementioned data, A means for generating proposals to optimize energy efficiency based on the aforementioned analysis results, A means for notifying the user terminal of the aforementioned proposal, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As the interest in an environmentally friendly lifestyle increases, there is a problem that many consumers do not know how to efficiently use energy in the home. This problem arises from the inability to grasp the waste of energy consumption in real time and the lack of knowledge of the optimal consumption method. In addition, due to the sparse participation in the local community, there is also a social problem that the opportunities for participating in eco activities and exchanging information are limited.

Means for Solving the Problems

[0005] This invention provides a system that monitors household energy consumption in real time, analyzes usage patterns, generates suggestions for optimizing energy efficiency, and notifies the user. Furthermore, it aims to strengthen connections with local communities and promote information exchange by collecting local eco-activities and event information and providing customized information based on the user's interests and lifestyle. This makes it possible to improve environmental awareness at the individual and community level.

[0006] "Household energy consumption data" refers to information about the amount of electricity, gas, and other energy used in a household. This information includes detailed data on the energy consumed by each device and piece of equipment.

[0007] "Usage patterns" refer to a collection of data that shows trends and habits in energy consumption over a specific period. This clearly shows when energy is used most and which devices consume the most energy.

[0008] "Suggestions for optimizing energy efficiency" refer to recommendations that outline specific actions and schedules for adjusting energy usage times and amounts in order to achieve energy savings.

[0009] "Notifying a user's device" refers to the act of transmitting information to a device owned within the home or by an individual. This includes providing information to the user in real time via electronic devices such as smartphones and personal computers.

[0010] "Local event information" refers to information about events and activities held within a specific geographical area. This includes information about gatherings, workshops, and seminars related to eco-activities and environmental protection.

[0011] "Automatic control" refers to technology that automatically operates household devices based on templates or programmed patterns. This includes a variety of settings to optimize energy consumption. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0013] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), etc.

[0016] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disk (e.g., hard disk), or magnetic tape, etc.

[0018] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] Possible embodiments for implementing the present invention include an energy management system and a local community information exchange system. The specific implementation methods for each system will be described below.

[0034] First, to manage energy consumption within the home, the server collects real-time energy consumption data from smart meters installed in each home. This data includes detailed usage information, such as which devices are consuming how much energy at what time of day.

[0035] The collected data is stored in a server database, and an AI agent analyzes this data using machine learning techniques. Based on the analysis, the server identifies energy usage patterns and generates suggestions for optimizing energy efficiency. For example, it can suggest operating air conditioners, which consume a lot of energy at night, during peak solar power generation hours in the daytime.

[0036] Next, the generated suggestions are sent from the server to the user's terminal. The user receives these notifications through their terminal and can select the specified action. The user can also enable the system's automatic control function, allowing device settings to be automatically changed at specified times.

[0037] Furthermore, to strengthen ties with local communities, the server regularly collects information on eco-events and activities held within the region. This includes information from local governments and eco-organizations. The collected information is customized based on the user's profile and interests and provided to their device. Users can view recommended event information on their device and have the opportunity to participate in events that interest them.

[0038] For example, if the server gathers information that "an eco-fair is being held nearby this weekend," it will prioritize notifying users with young children, taking into account that the event is family-friendly. In this way, users can deepen their interaction with local residents through participation in events.

[0039] As described above, the present invention aims to optimize energy use and promote participation in local communities, and the entire system is configured to work together organically to support the user's eco-lifestyle.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The server collects real-time energy usage data from smart meters installed in homes. This data includes usage time, consumption, and type of each device.

[0043] Step 2:

[0044] The server stores and accumulates the collected data in a database. This records usage history from the past to the present, laying the foundation for analysis.

[0045] Step 3:

[0046] The server uses machine learning algorithms to analyze information in the database and identify energy usage patterns within the home. This analysis detects peak energy consumption and wasteful consumption.

[0047] Step 4:

[0048] Based on the analysis results, the AI ​​agent generates suggestions for optimizing energy efficiency. These suggestions include recommendations for optimal device usage time and energy-saving mode settings.

[0049] Step 5:

[0050] The server notifies the user's device of the generated suggestions. These notifications include specific actions, such as, "Please adjust your air conditioner usage from nighttime to daytime solar power generation hours."

[0051] Step 6:

[0052] The user reviews the suggestions received through the terminal and either manually changes the settings or enables the automatic control function to allow the system to automatically adjust the device settings.

[0053] Step 7:

[0054] On the other hand, the server collects information on local eco-events, obtaining data from local governments and partner organizations. This information includes details such as the location, date and time, and target audience.

[0055] Step 8:

[0056] The AI ​​agent customizes the collected event information based on the user's profile and selects events of interest.

[0057] Step 9:

[0058] The server notifies the user's terminal of customized event information and sends a message such as, "Why not bring your family to the eco-fair being held nearby this weekend?"

[0059] Step 10:

[0060] Users check event information on their devices, decide to participate in events that interest them, and gain opportunities to interact with local communities through participation.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] This invention aims to effectively manage daily energy consumption within the home, improve energy efficiency, and promote participation in the local community. To achieve this, a system is needed that utilizes household energy consumption data and local activity information to automatically generate personalized suggestions tailored to individual user needs and appropriately notify users of these suggestions. Furthermore, a challenge remains in realizing these functions while minimizing the burden on users.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes means for collecting energy consumption data within the home, means for analyzing usage patterns based on the data, and means for generating suggestions to optimize energy efficiency based on the analysis results. This enables the generation of efficient suggestions tailored to the energy usage situation within the home and the establishment of effective relationships with the local community.

[0066] "Household energy consumption data" refers to data that measures and records the amount of energy, such as electricity and gas, consumed by various electrical appliances and equipment in a household over a certain period of time.

[0067] A "usage pattern" is a pattern extracted from energy consumption data that shows the trends and regularities in energy consumption within a specific period.

[0068] A "proposal to optimize energy efficiency" is a proposal that outlines specific action guidelines and improvement measures aimed at reducing energy consumption while maintaining necessary convenience, based on usage patterns.

[0069] A "user terminal" is an electronic device owned by an individual and used for energy management and receiving local information, and includes smartphones and tablets.

[0070] "Local activity information" refers to information about events and activities held in a specific area, and is provided with the aim of promoting community engagement.

[0071] "Customization" refers to optimizing the information and recommendations provided based on the user's interests and past participation history, and tailoring them to individual needs.

[0072] A "generative AI model for performing machine learning" is a model based on artificial intelligence technology that analyzes energy consumption and local activity patterns and is used to generate future usage predictions and optimization suggestions.

[0073] A "prompt statement" is a series of sentences that constitute the input information necessary to create suggestions and recommendations for a generative AI model, and is used to improve the accuracy and efficiency of the model.

[0074] Embodiments of the present invention will now be described. The present invention constructs a system for effectively managing household energy consumption and enhancing engagement with the local community. The details are described below.

[0075] First, the server collects energy consumption data in real time from smart meters installed in each home. This uses specific communication protocols (e.g., HTTP or MQTT) to clearly understand energy usage. The data is stored in a database management system (e.g., MySQL® or PostgreSQL).

[0076] Next, an AI agent running on the server analyzes the collected data using machine learning techniques (specifically, generative AI models such as TENSORFLOW® and PyTorch). At this stage, an algorithm that learns past patterns and predicts future consumption is applied, generating suggestions for efficiency improvements. These suggestions may include adjusting the usage time of air conditioners and lighting, or utilizing energy-saving modes.

[0077] The generated suggestions are sent from the server to the user's device (e.g., smartphone or tablet) via push notification. The user can then review the suggestions on their device and select the appropriate action. Furthermore, automated control is also achieved using IoT device control protocols (e.g., Zigbee, Z-Wave).

[0078] The server also regularly collects information on local eco-events and activities, customizes it based on the user's profile, and provides it to the device. For example, it collects information such as "an eco-fair is being held nearby this weekend" and sends notifications to families with young children encouraging them to participate.

[0079] For example, if a user is interested in learning how to save energy, the server will send a suggestion to the device saying, "You can save energy by slightly increasing the refrigerator setting." Another example of a prompt for the generating AI model is, "Analyze the user's energy usage patterns and suggest an eco-friendly event suitable for this weekend."

[0080] The system, configured in this way, aims to evolve users' lifestyles in an eco-friendly manner while simultaneously supporting increased engagement between users and their local communities.

[0081] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0082] Step 1:

[0083] The server collects energy consumption data in real time from smart meters installed in homes. It receives energy consumption information from each smart meter as input and generates structured energy consumption data for storage in a database as output. This operation involves data retrieval via communication protocols (e.g., HTTP or MQTT).

[0084] Step 2:

[0085] The server stores the collected energy consumption data in a database. It accepts the structured data formed in step 1 as input, and the data is permanently stored in a database management system (e.g., MySQL or PostgreSQL) as output. This step prepares the data for subsequent analysis processes.

[0086] Step 3:

[0087] The server's AI agent retrieves energy consumption data stored in a database and performs pattern recognition using a generative AI model. The input is historical energy consumption data, and the output is an analysis showing patterns and trends in energy use. Machine learning frameworks such as TensorFlow and PyTorch are used in this step.

[0088] Step 4:

[0089] The server generates suggestions to optimize energy efficiency based on the analysis results. Using pattern data generated by the AI ​​model as input, it outputs specific energy-saving suggestions for the user. At this stage, suggestions such as "Adjust air conditioner usage to peak hours during the day" are formed as prompts.

[0090] Step 5:

[0091] The server notifies the user's device of the generated suggestions. The input is the generated suggestion data, and the output is a notification on the user's smartphone or tablet. Using a push notification service (e.g., Firebase Cloud Messaging), the suggestions are displayed on the device in real time.

[0092] Step 6:

[0093] The user reviews the suggestions received through the device and selects an action as needed. The input is the suggestion information displayed on the device, and the output is the user's selection. Furthermore, if the user agrees, configuration changes for controlling the IoT device are automatically made.

[0094] Step 7:

[0095] The server collects local activity information and customizes it based on user interests. It takes event information from local governments and activity groups as input, and outputs event notifications optimized for each individual user. In addition to data collection from information sources, matching is performed using user profiles.

[0096] Step 8:

[0097] Users participate in local events of interest based on customized information presented on their devices. The input is the customized event information displayed on the device, and the output is the user's decision to participate in the events and their engagement. Through this process, connections with the local community are strengthened.

[0098] (Application Example 1)

[0099] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0100] In modern cities, there is a need to optimize household energy consumption and strengthen engagement with the local community. However, current systems do not adequately manage energy in real time or promote appropriate participation in local events. As a result, energy waste and lack of participation in community activities occur. A system is needed to efficiently solve these problems.

[0101] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0102] In this invention, the server includes a device for collecting household energy consumption data, a device for analyzing usage patterns based on the data, and a device for generating suggestions to optimize energy efficiency based on the analysis results. This makes it possible to reduce wasted energy consumption and notify the user of optimal energy usage suggestions in real time.

[0103] "Household energy consumption data" refers to information that shows how energy such as electricity and gas is used within a home.

[0104] "Usage patterns" refer to the results of an analysis of trends and characteristics of energy use over a specific period.

[0105] "Suggestions for optimizing energy efficiency" refer to providing specific advice and instructions for reducing energy consumption or using energy more efficiently.

[0106] A "user information terminal" is a device used to receive notifications and suggestions from the energy management system, such as a smartphone or tablet.

[0107] "Local activity information" refers to information about events and community activities that take place within a specific area.

[0108] A "device that sends push notifications based on user interests" refers to a system that transmits relevant information to the user's information terminal in real time, based on the user's interests and preferences.

[0109] The embodiments for carrying out the invention are shown below.

[0110] This invention employs a system in which a server collects energy consumption data in real time from smart meters installed in homes. The smart meters record detailed energy usage in each home and transmit this data to the server. This data is stored using an internal database system, such as MongoDB, on the server. The collected data is analyzed using machine learning software such as TensorFlow to identify patterns in energy use.

[0111] The server generates suggestions for optimizing energy efficiency based on the analysis results. These suggestions are then pushed to the user's information terminal using a notification service such as Firebase. This allows the user to receive concrete actions for energy saving in real time.

[0112] Furthermore, the server has the ability to collect information on activities taking place within the region via the internet and to filter and customize it based on the user's interests. Activity information selected according to the user's interests is then pushed to the user's smartphone or tablet.

[0113] Specific examples include suggestions to refrain from using air conditioners during peak energy consumption hours, and notifications encouraging participation in eco-fairs held on weekends. When using a generative AI model, it is possible to generate specific suggestions by using the following prompt statements.

[0114] "Generate suggestions for optimizing energy efficiency based on the user's household energy consumption data for the past month."

[0115] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0116] Step 1:

[0117] The server receives real-time energy consumption data from smart meters installed in homes. This data includes energy usage by time of day and usage status of each device. The server records the acquired data in its database system. The server structures this data and stores it in a large database to facilitate subsequent analysis processes.

[0118] Step 2:

[0119] The server uses collected energy consumption data as input and analyzes it with machine learning models utilizing TensorFlow, etc. Specifically, it identifies consumption patterns and generates suggestions for efficient energy use. This analysis process detects outliers and compares them to normal conditions to discover patterns that can improve energy efficiency.

[0120] Step 3:

[0121] The server sends energy efficiency suggestions, generated based on the analysis results, to the user's device via a notification service such as Firebase. Users can reduce energy waste by receiving these notifications and following the suggested methods. The notifications include specific advice, such as which devices to use and when, for maximum efficiency.

[0122] Step 4:

[0123] The server collects local activity information from the internet. This includes local event calendars and information provided by local governments. The collected information is then filtered based on the user's profile and past participation history, and the appropriate information is selected.

[0124] Step 5:

[0125] Information on selected local activities is customized to the user's interests and delivered via push notifications. Through these notifications, users can participate in local events that interest them and deepen their engagement with the community. This customization utilizes the results of an analysis of the user's past interests and tendencies.

[0126] Step 6:

[0127] When users participate in a local event, they can provide feedback using their device. The server collects this feedback information and uses it as data to make more accurate suggestions for future events. This feedback data is used to analyze the popularity of the event and the satisfaction level of the participants.

[0128] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0129] One possible embodiment of the present invention is a system that combines an emotion engine with a system that optimizes household energy consumption and promotes participation in local events. This system utilizes various devices installed in the home and a user terminal, and by using the emotion engine, provides more personalized suggestions.

[0130] First, the server collects energy consumption data from smart meters and smart devices in the home and stores it in a database. This data includes the usage time, energy consumption, and type of energy used for each device. Based on this data, the server identifies energy usage patterns and generates suggestions for optimization.

[0131] Next, the system's built-in emotion engine recognizes the user's emotional state in real time. Emotion recognition is based on facial expression analysis using a camera, voice tone analysis, and user interaction logs.

[0132] The server incorporates emotional data recognized by the emotion engine and adjusts energy optimization suggestions to suit the user's psychological state. For example, if the system determines that the user is stressed, it can suggest lighting settings that promote relaxation.

[0133] The user terminal will be notified of the adjusted suggestions. The user can review the notification and manually configure the suggested energy settings and action instructions, or select automatic control mode and let the system handle it.

[0134] Furthermore, the server collects local event information and uses an emotion engine to select events that match the user's interests and mood. This information is notified to the user's device and accompanied by specific messages to encourage participation. For example, it might say, "If you're feeling refreshed, why not join a yoga workshop that's sure to have a relaxing effect?"

[0135] For example, if a user is feeling irritated at home, the emotion engine will detect this, and the server will suggest lowering the lighting and playing calming music. This suggestion will be notified to the device, and if the user agrees, the settings will be automatically applied. Information about local art therapy events will also be appropriately notified.

[0136] Thus, the present invention is a system that combines efficient energy management with the provision of services based on the user's emotional state, thereby improving the user experience and promoting active engagement with the local community.

[0137] The following describes the processing flow.

[0138] Step 1:

[0139] The server collects real-time energy consumption data from smart meters and smart appliances within the home. This data includes the usage time and power consumption of each device.

[0140] Step 2:

[0141] The server stores the collected data in a database. This allows for the accumulation of past usage history, which can then be used for later analysis.

[0142] Step 3:

[0143] The server uses machine learning algorithms to analyze energy consumption information stored in the database and identify usage patterns. This analysis reveals peak consumption and wasteful consumption.

[0144] Step 4:

[0145] The system's built-in emotion engine analyzes the user's facial expressions and voice tone to recognize emotions in real time. User interaction logs are also used for emotion recognition.

[0146] Step 5:

[0147] The server adjusts suggestions to optimize energy efficiency based on the user's emotional state, as recognized by the emotion engine. For example, if the user is stressed, it might suggest changing the lighting to a softer, more relaxing light.

[0148] Step 6:

[0149] The server notifies the user's terminal of the adjusted suggestions. The notification includes the suggested energy settings and activity instructions.

[0150] Step 7:

[0151] The user checks the notification on their device and can choose to manually change device settings according to the suggestion or leave it to the system to take automatic control.

[0152] Step 8:

[0153] The server regularly collects information on events held within the region, including details on eco-related activities and local events.

[0154] Step 9:

[0155] The emotion engine selects events that are likely to be of interest to the user based on their current emotions. The selected event information is tailored to the user's profile and emotions.

[0156] Step 10:

[0157] The server notifies the user's device of event information that matches their interests and mood, along with a message encouraging them to participate. The user can check the notification and register to participate in events they like.

[0158] (Example 2)

[0159] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0160] In modern homes, a variety of devices exist, energy consumption is increasing, and efficient resource management is required. Furthermore, creating a comfortable living environment that suits individual emotions and actively participating in community activities are also important issues. However, existing systems often fail to adequately provide optimized solutions that consider these individual needs. Therefore, the present invention aims to realize a system that improves the efficiency of resource management within the home and provides personalized suggestions that respond to the user's emotional state.

[0161] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0162] In this invention, the server includes means for collecting resource consumption information within the home, means for recognizing emotional states using an emotion engine, and means for improving the quality of information and suggestions using a generative AI model. This enables efficient resource management and the generation of comfortable suggestions that respond to the user's emotions.

[0163] "Resource consumption information" refers to data on all resources used within a household, specifically including information on the amount of electricity, water, and gas used.

[0164] The "emotion engine" is a function that analyzes and recognizes the user's emotional state from factors such as voice tone and facial expressions.

[0165] A "generative AI model" refers to an algorithm that uses artificial intelligence technology to improve the accuracy of data analysis and proposal generation.

[0166] A "means for adjusting suggestions" refers to a mechanism for appropriately modifying the content and method of generated suggestions, taking into account dynamic factors such as the user's emotional state.

[0167] A "terminal" is a device used to notify users of energy consumption suggestions and local activity information, and includes smartphones and tablets.

[0168] "Automatic control" refers to a system or device operating autonomously based on pre-set rules, without requiring manual operation by the user.

[0169] "Local activity information" refers to information about events and activities held within a specific region, and is intended to encourage active participation from users.

[0170] "Personalized experiences" refer to services and information tailored to the user's emotional state and interests, meaning that different values ​​are provided for each user.

[0171] This system is designed to optimize resource consumption within the home and provide suggestions tailored to the user's emotions. Specifically, the server, terminals, and users each play their respective roles, facilitating the smooth collection, analysis, and notification of information.

[0172] The server collects resource consumption information from multiple sensors and network-connected devices installed within the home. This utilizes smart meters and other smart devices, storing the collected data in a central database. Furthermore, the server performs data analysis using software libraries such as Python's Pandas and SciPy. This identifies resource usage patterns and generates optimization suggestions to improve energy efficiency.

[0173] Regarding emotion recognition, the server uses an emotion engine to analyze the user's voice tone and facial expression data. This analysis incorporates an interface that utilizes a camera and microphone. As a result of the analysis, the server can understand the user's emotional state in real time. The server combines this emotion data with resource consumption information to provide specific and effective suggestions tailored to the user.

[0174] The terminal's role is to inform the user of suggestions and notifications sent from the server. Devices such as smartphones and tablets are used, allowing users to review suggestions and choose how to respond. For example, it's possible to suggest relaxing lighting settings or music playback to a user experiencing stress. If the user approves the suggestion, the terminal automatically continues the settings.

[0175] Furthermore, the server collects local activity information and provides personalized information based on emotions and interests. This allows users to participate in local events that match their mood and interests, deepening their connection with the community. For example, prompts such as, "If you're feeling refreshed, why not join a yoga workshop that can help you relax?" are used to encourage participation.

[0176] This system utilizes generative AI models to improve the quality of suggestions and make users' lives more comfortable and efficient.

[0177] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0178] Step 1:

[0179] The server collects resource consumption information from smart meters and smart devices within the home. It receives usage data from each device as input. The server receives this data and stores it in a database. The collected data includes time-series consumption and usage time for each device. Specifically, it accesses each device at regular intervals to retrieve information and aggregates it at the central server. The output is organized resource consumption information.

[0180] Step 2:

[0181] The server performs data analysis based on the collected resource consumption information. The resource consumption information collected in Step 1 is used as input. Trends and outliers in the data are analyzed using Python's Pandas and SciPy libraries. For example, historical data is analyzed to identify patterns of sharp increases in energy consumption during specific time periods. The output reports energy usage patterns and potential optimization opportunities.

[0182] Step 3:

[0183] The server uses an emotion engine to recognize the user's emotional state. It acquires real-time data from the camera and microphone as input. This data is analyzed by an emotion recognition model to identify the user's emotional state (e.g., stress, relaxation). Specifically, facial expression recognition and voice tone analysis are used. The output provides the user's current emotional state.

[0184] Step 4:

[0185] The server generates suggestions based on energy usage patterns and emotional states. It uses the outputs from steps 2 and 3 as input. Leveraging a generative AI model, it creates an optimal energy consumption plan that considers the user's health and comfort. For example, for a stressed user, it can suggest relaxing lighting settings. The output includes specific action suggestions and proposed setting changes.

[0186] Step 5:

[0187] The terminal notifies the user of the suggestion from the server. It receives the suggestion generated in step 4 as input. The terminal presents the notification to the user visually and audibly, prompting them to review the suggestion. Specific actions include displaying a notification on the smartphone screen and providing an audio notification. The output ensures that the suggestion is reliably communicated to the user.

[0188] Step 6:

[0189] The user responds to the suggestion displayed on the device. As input, they receive a notification in step 5. The user reviews the suggestion and decides whether to perform it manually based on their choice or allow automated control. Specifically, they can perform the action from the smartphone app or press the "Approve" button to leave it to the system. As output, an action occurs according to the user's choice.

[0190] Step 7:

[0191] The server automatically controls and adjusts the home environment based on user selections. It also collects local activity information and provides personalized event information based on the user's emotional state and interests. Inputs include the user selections from step 6 and local activity information. Based on the selections, the server changes lighting and music settings and notifies the device of appropriate local events. Outputs include the adjusted home settings and local event notifications.

[0192] (Application Example 2)

[0193] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0194] In modern home and urban energy management, achieving both efficient energy consumption and improved quality of life for residents simultaneously is a challenging task. In particular, the emotional state of individual household residents significantly influences energy consumption patterns and their willingness to participate in community activities, but current technology struggles to effectively consider this and optimize the entire system. Furthermore, while personalized information provision based on the emotions and interests of individual residents is needed in community life, there are limitations to achieving this.

[0195] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0196] In this invention, the server includes means for collecting household energy consumption data, means for analyzing energy usage patterns, and means for detecting emotional states and adjusting optimization suggestions. This enables energy-efficient lifestyle suggestions based on the emotional states of individual residents. Furthermore, it can customize local event information based on emotional states to increase motivation to participate.

[0197] "Means of collecting household energy consumption data" refers to devices and technologies that collect the amount of energy consumed and usage patterns through various sensors and smart meters installed in the home.

[0198] "Means of analyzing usage patterns" refers to technologies that analyze trends and characteristics of energy use for specific devices or time periods based on collected energy consumption data.

[0199] "Means for generating proposals to optimize energy efficiency" refers to technologies that utilize the results of usage pattern analysis to design specific advice and strategies for reducing and improving energy consumption.

[0200] "Means of notifying user terminals" refers to technologies that notify users within their home using devices such as smartphones and computers of the generated suggestions.

[0201] "Means for detecting emotional states" refers to technologies that use cameras and voice analysis techniques to identify a user's emotional state from their facial expressions and tone of voice.

[0202] "Means for adjusting optimization suggestions" refers to technologies that flexibly modify the content of energy efficiency suggestions, taking into account the user's emotional state.

[0203] "Means of collecting local event information" refers to technologies for gathering information about events and activities held in a geographically specific area.

[0204] "Means of customizing information" refers to technologies that adjust collected event information to be optimal for individual users based on their interests and emotional state.

[0205] This invention is a system that optimizes household energy consumption and improves engagement with the local community based on the emotional state of residents. The server collects energy consumption data in real time from smart meters and various sensors installed in the home. Using this data, software with a dedicated algorithm is used to analyze energy usage patterns.

[0206] Next, the server uses emotion recognition technology to detect the user's emotional state. This technology is implemented through facial expression analysis using a camera and voice tone analysis. This allows the server to determine the user's stress level and relaxation level.

[0207] Based on the emotional data and usage pattern analysis, the server generates suggestions to optimize energy efficiency. If the user is experiencing stress, for example, it might suggest relaxing settings such as lowering the lighting.

[0208] Furthermore, the server collects local event information and provides customized suggestions to encourage participation in events tailored to the user's emotional state. For example, it can send a specific message to the user such as, "There's a relaxing yoga class available. Would you like to join?"

[0209] These suggestions and notifications are delivered to the user's device via smartphone or computer. The user can review the notification and either manually follow the suggestion or let the system handle the settings in automatic control mode.

[0210] As a concrete example, if a user is feeling irritated at home, the server can detect this and provide information about nearby art therapy events, along with suggestions to gently adjust the lighting. This functionality is implemented using Python and emotion recognition modules such as EmotionEngine.

[0211] An example of a prompt might be, "In a smart city, consider ways to optimize energy consumption and encourage event participation based on user emotions."

[0212] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0213] Step 1:

[0214] The server collects energy consumption data in real time from smart meters and sensors installed in homes. This input data includes power consumption, usage time, and consumption patterns for each device. The data is initially processed and stored in a database, which provides the foundational information for subsequent analysis.

[0215] Step 2:

[0216] The server applies algorithms to the collected energy consumption data to analyze energy usage patterns. Time-series analysis is performed as a data processing step to extract consumption trends for each time period. The output includes usage characteristics by day of the week and peak consumption patterns. Based on these results, areas for improvement in energy management are identified.

[0217] Step 3:

[0218] The server uses a camera and microphone to detect the user's emotional state in real time. It uses facial expression data from the camera and tone data from the audio as input. An emotion recognition algorithm determines the user's emotional state, such as whether they are stressed or relaxed, and records the result as output.

[0219] Step 4:

[0220] The server integrates analysis results of energy consumption patterns with the user's emotional state to generate specific suggestions for optimizing energy efficiency. Using analysis results and emotional data as input, it creates adjusted energy efficiency suggestions as output. For example, if the user is stressed, it might suggest lighting settings that promote relaxation.

[0221] Step 5:

[0222] The server collects local event information and generates customized event participation suggestions based on the user's emotional state. Here, local event information is taken as input, events are filtered considering the emotional state and interests, and output as recommended events for the user.

[0223] Step 6:

[0224] The terminal notifies the user of energy optimization suggestions and event participation suggestions sent from the server. The notifications include specific action suggestions and invitations to participate in events. The user reviews the notifications and either manually implements the suggestions or selects automatic control mode to let the terminal handle it.

[0225] Step 7:

[0226] Users make decisions regarding energy settings and event participation based on suggestions through their device. The device feeds the user's decisions back to the server, which then uses this feedback to make future suggestions more personalized. This establishes a continuous optimization cycle.

[0227] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0228] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0229] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0230] [Second Embodiment]

[0231] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0232] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0233] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0234] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0235] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0236] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0237] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0238] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0239] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0240] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0241] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0242] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0243] Possible embodiments for implementing the present invention include an energy management system and a local community information exchange system. The specific implementation methods for each system will be described below.

[0244] First, to manage energy consumption within the home, the server collects real-time energy consumption data from smart meters installed in each home. This data includes detailed usage information, such as which devices are consuming how much energy at what time of day.

[0245] The collected data is stored in a server database, and an AI agent analyzes this data using machine learning techniques. Based on the analysis, the server identifies energy usage patterns and generates suggestions for optimizing energy efficiency. For example, it can suggest operating air conditioners, which consume a lot of energy at night, during peak solar power generation hours in the daytime.

[0246] Next, the generated suggestions are sent from the server to the user's terminal. The user receives these notifications through their terminal and can select the specified action. The user can also enable the system's automatic control function, allowing device settings to be automatically changed at specified times.

[0247] Furthermore, to strengthen ties with local communities, the server regularly collects information on eco-events and activities held within the region. This includes information from local governments and eco-organizations. The collected information is customized based on the user's profile and interests and provided to their device. Users can view recommended event information on their device and have the opportunity to participate in events that interest them.

[0248] For example, if the server gathers information that "an eco-fair is being held nearby this weekend," it will prioritize notifying users with young children, taking into account that the event is family-friendly. In this way, users can deepen their interaction with local residents through participation in events.

[0249] As described above, the present invention aims to optimize energy use and promote participation in local communities, and the entire system is configured to work together organically to support the user's eco-lifestyle.

[0250] The following describes the processing flow.

[0251] Step 1:

[0252] The server collects real-time energy usage data from smart meters installed in homes. This data includes usage time, consumption, and type of each device.

[0253] Step 2:

[0254] The server stores and accumulates the collected data in a database. This records usage history from the past to the present, laying the foundation for analysis.

[0255] Step 3:

[0256] The server uses machine learning algorithms to analyze information in the database and identify energy usage patterns within the home. This analysis detects peak energy consumption and wasteful consumption.

[0257] Step 4:

[0258] Based on the analysis results, the AI ​​agent generates suggestions for optimizing energy efficiency. These suggestions include recommendations for optimal device usage time and energy-saving mode settings.

[0259] Step 5:

[0260] The server notifies the user's device of the generated suggestions. These notifications include specific actions, such as, "Please adjust your air conditioner usage from nighttime to daytime solar power generation hours."

[0261] Step 6:

[0262] The user reviews the suggestions received through the terminal and either manually changes the settings or enables the automatic control function to allow the system to automatically adjust the device settings.

[0263] Step 7:

[0264] On the other hand, the server collects information on local eco-events, obtaining data from local governments and partner organizations. This information includes details such as the location, date and time, and target audience.

[0265] Step 8:

[0266] The AI ​​agent customizes the collected event information based on the user's profile and selects events of interest.

[0267] Step 9:

[0268] The server notifies the user's terminal of customized event information and sends a message such as, "Why not bring your family to the eco-fair being held nearby this weekend?"

[0269] Step 10:

[0270] Users check event information on their devices, decide to participate in events that interest them, and gain opportunities to interact with local communities through participation.

[0271] (Example 1)

[0272] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0273] This invention aims to effectively manage daily energy consumption within the home, improve energy efficiency, and promote participation in the local community. To achieve this, a system is needed that utilizes household energy consumption data and local activity information to automatically generate personalized suggestions tailored to individual user needs and appropriately notify users of these suggestions. Furthermore, a challenge remains in realizing these functions while minimizing the burden on users.

[0274] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0275] In this invention, the server includes means for collecting energy consumption data within the home, means for analyzing usage patterns based on the data, and means for generating suggestions to optimize energy efficiency based on the analysis results. This enables the generation of efficient suggestions tailored to the energy usage situation within the home and the establishment of effective relationships with the local community.

[0276] "Household energy consumption data" refers to data that measures and records the amount of energy, such as electricity and gas, consumed by various electrical appliances and equipment in a household over a certain period of time.

[0277] A "usage pattern" is a pattern extracted from energy consumption data that shows the trends and regularities in energy consumption within a specific period.

[0278] A "proposal to optimize energy efficiency" is a proposal that outlines specific action guidelines and improvement measures aimed at reducing energy consumption while maintaining necessary convenience, based on usage patterns.

[0279] A "user terminal" is an electronic device owned by an individual and used for energy management and receiving local information, and includes smartphones and tablets.

[0280] "Local activity information" refers to information about events and activities held in a specific area, and is provided with the aim of promoting community engagement.

[0281] "Customization" refers to optimizing the information and recommendations provided based on the user's interests and past participation history, and tailoring them to individual needs.

[0282] A "generative AI model for performing machine learning" is a model based on artificial intelligence technology that analyzes energy consumption and local activity patterns and is used to generate future usage predictions and optimization suggestions.

[0283] A "prompt sentence" is a series of sentences that constitute the input information necessary to create proposals and recommendations for a generative AI model, and is used to improve the accuracy and efficiency of the model.

[0284] Embodiments of the present invention will be described. The present invention constructs a system for effectively managing home energy consumption and enhancing engagement with the local community. The details are described below.

[0285] First, the server collects energy consumption data in real time from smart meters installed in each home. For this, a specific communication protocol (e.g., HTTP or MQTT) is used to clearly understand the energy usage situation. The data is stored in a database management system (e.g., MySQL or PostgreSQL).

[0286] Next, an AI agent operating on the server analyzes the data collected using machine learning techniques (specifically, generative AI models such as TensorFlow or PyTorch). At this stage, an algorithm that learns past patterns and predicts future consumption is applied, and proposals for efficiency improvement are generated. The generated proposals may specifically include adjusting the usage time of air conditioners and lighting, or utilizing energy-saving modes.

[0287] The generated proposals are sent from the server to the user's terminal (e.g., smartphone or tablet) via push notifications. On the terminal, the user can view the proposals and select appropriate actions. Furthermore, automatic control is also realized by utilizing the control protocols of IoT devices (e.g., Zigbee, Z-Wave).

[0288] In addition, the server regularly collects information on eco-events and activities within the region, customizes it based on the user's profile, and provides it to the terminal. For example, information such as "An eco-fair will be held in the neighborhood this weekend" is collected, and a notification is sent to encourage families with children to participate.

[0289] For example, if a user is interested in learning how to save energy, the server will send a suggestion to the device saying, "You can save energy by slightly increasing the refrigerator setting." Another example of a prompt for the generating AI model is, "Analyze the user's energy usage patterns and suggest an eco-friendly event suitable for this weekend."

[0290] The system, configured in this way, aims to evolve users' lifestyles in an eco-friendly manner while simultaneously supporting increased engagement between users and their local communities.

[0291] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0292] Step 1:

[0293] The server collects energy consumption data in real time from smart meters installed in homes. It receives energy consumption information from each smart meter as input and generates structured energy consumption data for storage in a database as output. This operation involves data retrieval via communication protocols (e.g., HTTP or MQTT).

[0294] Step 2:

[0295] The server stores the collected energy consumption data in a database. It accepts the structured data formed in step 1 as input, and the data is permanently stored in a database management system (e.g., MySQL or PostgreSQL) as output. This step prepares the data for subsequent analysis processes.

[0296] Step 3:

[0297] The server's AI agent retrieves energy consumption data stored in a database and performs pattern recognition using a generative AI model. The input is historical energy consumption data, and the output is an analysis showing patterns and trends in energy use. Machine learning frameworks such as TensorFlow and PyTorch are used in this step.

[0298] Step 4:

[0299] The server generates suggestions to optimize energy efficiency based on the analysis results. Using pattern data generated by the AI ​​model as input, it outputs specific energy-saving suggestions for the user. At this stage, suggestions such as "Adjust air conditioner usage to peak hours during the day" are formed as prompts.

[0300] Step 5:

[0301] The server notifies the user's device of the generated suggestions. The input is the generated suggestion data, and the output is a notification on the user's smartphone or tablet. Using a push notification service (e.g., Firebase Cloud Messaging), the suggestions are displayed on the device in real time.

[0302] Step 6:

[0303] The user reviews the suggestions received through the device and selects an action as needed. The input is the suggestion information displayed on the device, and the output is the user's selection. Furthermore, if the user agrees, configuration changes for controlling the IoT device are automatically made.

[0304] Step 7:

[0305] The server collects regional activity information and customizes it based on the interests of users. It obtains event information from local governments and activity groups as input, and the output is event notifications optimized for individual users. In addition to data collection from information sources, matching using user profiles is performed.

[0306] Step 8:

[0307] Based on the customized information presented on the terminal, the user participates in regional events of interest. The input is the customized event information displayed on the terminal, and the output is the user's decision-making and engagement regarding event participation. Through this process, the connection with the local community is strengthened.

[0308] (Application Example 1)

[0309] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0310] In modern cities, it is necessary to optimize energy consumption within households and strengthen the connection with the local community. However, in the current system, real-time energy management and the proper promotion of participation in regional events are not sufficiently carried out. As a result, there is waste of energy and insufficient participation in community activities. A system for efficiently solving these problems is required.

[0311] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following respective means.

[0312] In this invention, the server includes a device for collecting energy consumption data within a household, a device for analyzing usage patterns based on the data, and a device for generating proposals to optimize energy efficiency based on the analysis results. Thereby, it becomes possible to reduce waste of energy consumption and notify the user of optimal energy usage proposals in real time.

[0313] "Household energy consumption data" refers to information that shows how energy such as electricity and gas is used within a home.

[0314] "Usage patterns" refer to the results of an analysis of trends and characteristics of energy use over a specific period.

[0315] "Suggestions for optimizing energy efficiency" refer to providing specific advice and instructions for reducing energy consumption or using energy more efficiently.

[0316] A "user information terminal" is a device used to receive notifications and suggestions from the energy management system, such as a smartphone or tablet.

[0317] "Local activity information" refers to information about events and community activities that take place within a specific area.

[0318] A "device that sends push notifications based on user interests" refers to a system that transmits relevant information to the user's information terminal in real time, based on the user's interests and preferences.

[0319] The embodiments for carrying out the invention are shown below.

[0320] This invention employs a system in which a server collects energy consumption data in real time from smart meters installed in homes. The smart meters record detailed energy usage in each home and transmit this data to the server. This data is stored using an internal database system, such as MongoDB, on the server. The collected data is analyzed using machine learning software such as TensorFlow to identify patterns in energy use.

[0321] The server generates suggestions for optimizing energy efficiency based on the analysis results. These suggestions are then pushed to the user's information terminal using a notification service such as Firebase. This allows the user to receive concrete actions for energy saving in real time.

[0322] Furthermore, the server has the ability to collect information on activities taking place within the region via the internet and to filter and customize it based on the user's interests. Activity information selected according to the user's interests is then pushed to the user's smartphone or tablet.

[0323] Specific examples include suggestions to refrain from using air conditioners during peak energy consumption hours, and notifications encouraging participation in eco-fairs held on weekends. When using a generative AI model, it is possible to generate specific suggestions by using the following prompt statements.

[0324] "Generate suggestions for optimizing energy efficiency based on the user's household energy consumption data for the past month."

[0325] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0326] Step 1:

[0327] The server receives real-time energy consumption data from smart meters installed in homes. This data includes energy usage by time of day and usage status of each device. The server records the acquired data in its database system. The server structures this data and stores it in a large database to facilitate subsequent analysis processes.

[0328] Step 2:

[0329] The server uses collected energy consumption data as input and analyzes it with machine learning models utilizing TensorFlow, etc. Specifically, it identifies consumption patterns and generates suggestions for efficient energy use. This analysis process detects outliers and compares them to normal conditions to discover patterns that can improve energy efficiency.

[0330] Step 3:

[0331] The server sends energy efficiency suggestions, generated based on the analysis results, to the user's device via a notification service such as Firebase. Users can reduce energy waste by receiving these notifications and following the suggested methods. The notifications include specific advice, such as which devices to use and when, for maximum efficiency.

[0332] Step 4:

[0333] The server collects local activity information from the internet. This includes local event calendars and information provided by local governments. The collected information is then filtered based on the user's profile and past participation history, and the appropriate information is selected.

[0334] Step 5:

[0335] Information on selected local activities is customized to the user's interests and delivered via push notifications. Through these notifications, users can participate in local events that interest them and deepen their engagement with the community. This customization utilizes the results of an analysis of the user's past interests and tendencies.

[0336] Step 6:

[0337] When users participate in a local event, they can provide feedback using their device. The server collects this feedback information and uses it as data to make more accurate suggestions for future events. This feedback data is used to analyze the popularity of the event and the satisfaction level of the participants.

[0338] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0339] One possible embodiment of the present invention is a system that combines an emotion engine with a system that optimizes household energy consumption and promotes participation in local events. This system utilizes various devices installed in the home and a user terminal, and by using the emotion engine, provides more personalized suggestions.

[0340] First, the server collects energy consumption data from smart meters and smart devices in the home and stores it in a database. This data includes the usage time, energy consumption, and type of energy used for each device. Based on this data, the server identifies energy usage patterns and generates suggestions for optimization.

[0341] Next, the system's built-in emotion engine recognizes the user's emotional state in real time. Emotion recognition is based on facial expression analysis using a camera, voice tone analysis, and user interaction logs.

[0342] The server incorporates emotional data recognized by the emotion engine and adjusts energy optimization suggestions to suit the user's psychological state. For example, if the system determines that the user is stressed, it can suggest lighting settings that promote relaxation.

[0343] The user terminal will be notified of the adjusted suggestions. The user can review the notification and manually configure the suggested energy settings and action instructions, or select automatic control mode and let the system handle it.

[0344] Furthermore, the server collects local event information and uses an emotion engine to select events that match the user's interests and mood. This information is notified to the user's device and accompanied by specific messages to encourage participation. For example, it might say, "If you're feeling refreshed, why not join a yoga workshop that's sure to have a relaxing effect?"

[0345] For example, if a user is feeling irritated at home, the emotion engine will detect this, and the server will suggest lowering the lighting and playing calming music. This suggestion will be notified to the device, and if the user agrees, the settings will be automatically applied. Information about local art therapy events will also be appropriately notified.

[0346] Thus, the present invention is a system that combines efficient energy management with the provision of services based on the user's emotional state, thereby improving the user experience and promoting active engagement with the local community.

[0347] The following describes the processing flow.

[0348] Step 1:

[0349] The server collects real-time energy consumption data from smart meters and smart appliances within the home. This data includes the usage time and power consumption of each device.

[0350] Step 2:

[0351] The server stores the collected data in a database. This allows for the accumulation of past usage history, which can then be used for later analysis.

[0352] Step 3:

[0353] The server uses machine learning algorithms to analyze energy consumption information stored in the database and identify usage patterns. This analysis reveals peak consumption and wasteful consumption.

[0354] Step 4:

[0355] The system's built-in emotion engine analyzes the user's facial expressions and voice tone to recognize emotions in real time. User interaction logs are also used for emotion recognition.

[0356] Step 5:

[0357] The server adjusts suggestions to optimize energy efficiency based on the user's emotional state, as recognized by the emotion engine. For example, if the user is stressed, it might suggest changing the lighting to a softer, more relaxing light.

[0358] Step 6:

[0359] The server notifies the user's terminal of the adjusted suggestions. The notification includes the suggested energy settings and activity instructions.

[0360] Step 7:

[0361] The user checks the notification on their device and can choose to manually change device settings according to the suggestion or leave it to the system to take automatic control.

[0362] Step 8:

[0363] The server regularly collects information on events held within the region, including details on eco-related activities and local events.

[0364] Step 9:

[0365] The emotion engine selects events that are likely to be of interest to the user based on their current emotions. The selected event information is tailored to the user's profile and emotions.

[0366] Step 10:

[0367] The server notifies the user's device of event information that matches their interests and mood, along with a message encouraging them to participate. The user can check the notification and register to participate in events they like.

[0368] (Example 2)

[0369] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0370] In modern homes, a variety of devices exist, energy consumption is increasing, and efficient resource management is required. Furthermore, creating a comfortable living environment that suits individual emotions and actively participating in community activities are also important issues. However, existing systems often fail to adequately provide optimized solutions that consider these individual needs. Therefore, the present invention aims to realize a system that improves the efficiency of resource management within the home and provides personalized suggestions that respond to the user's emotional state.

[0371] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0372] In this invention, the server includes means for collecting resource consumption information within the home, means for recognizing emotional states using an emotion engine, and means for improving the quality of information and suggestions using a generative AI model. This enables efficient resource management and the generation of comfortable suggestions that respond to the user's emotions.

[0373] "Resource consumption information" refers to data on all resources used within a household, specifically including information on the amount of electricity, water, and gas used.

[0374] The "emotion engine" is a function that analyzes and recognizes the user's emotional state from factors such as voice tone and facial expressions.

[0375] A "generative AI model" refers to an algorithm that uses artificial intelligence technology to improve the accuracy of data analysis and proposal generation.

[0376] A "means for adjusting suggestions" refers to a mechanism for appropriately modifying the content and method of generated suggestions, taking into account dynamic factors such as the user's emotional state.

[0377] A "terminal" is a device used to notify users of energy consumption suggestions and local activity information, and includes smartphones and tablets.

[0378] "Automatic control" refers to a system or device operating autonomously based on pre-set rules, without requiring manual operation by the user.

[0379] "Local activity information" refers to information about events and activities held within a specific region, and is intended to encourage active participation from users.

[0380] "Personalized experiences" refer to services and information tailored to the user's emotional state and interests, meaning that different values ​​are provided for each user.

[0381] This system is designed to optimize resource consumption within the home and provide suggestions tailored to the user's emotions. Specifically, the server, terminals, and users each play their respective roles, facilitating the smooth collection, analysis, and notification of information.

[0382] The server collects resource consumption information from multiple sensors and network-connected devices installed within the home. This utilizes smart meters and other smart devices, storing the collected data in a central database. Furthermore, the server performs data analysis using software libraries such as Python's Pandas and SciPy. This identifies resource usage patterns and generates optimization suggestions to improve energy efficiency.

[0383] Regarding emotion recognition, the server uses an emotion engine to analyze the user's voice tone and facial expression data. This analysis incorporates an interface that utilizes a camera and microphone. As a result of the analysis, the server can understand the user's emotional state in real time. The server combines this emotion data with resource consumption information to provide specific and effective suggestions tailored to the user.

[0384] The terminal's role is to inform the user of suggestions and notifications sent from the server. Devices such as smartphones and tablets are used, allowing users to review suggestions and choose how to respond. For example, it's possible to suggest relaxing lighting settings or music playback to a user experiencing stress. If the user approves the suggestion, the terminal automatically continues the settings.

[0385] Furthermore, the server collects local activity information and provides personalized information based on emotions and interests. This allows users to participate in local events that match their mood and interests, deepening their connection with the community. For example, prompts such as, "If you're feeling refreshed, why not join a yoga workshop that can help you relax?" are used to encourage participation.

[0386] This system utilizes generative AI models to improve the quality of suggestions and make users' lives more comfortable and efficient.

[0387] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0388] Step 1:

[0389] The server collects resource consumption information from smart meters and smart devices within the home. It receives usage data from each device as input. The server receives this data and stores it in a database. The collected data includes time-series consumption and usage time for each device. Specifically, it accesses each device at regular intervals to retrieve information and aggregates it at the central server. The output is organized resource consumption information.

[0390] Step 2:

[0391] The server performs data analysis based on the collected resource consumption information. The resource consumption information collected in Step 1 is used as input. Trends and outliers in the data are analyzed using Python's Pandas and SciPy libraries. For example, historical data is analyzed to identify patterns of sharp increases in energy consumption during specific time periods. The output reports energy usage patterns and potential optimization opportunities.

[0392] Step 3:

[0393] The server uses an emotion engine to recognize the user's emotional state. It acquires real-time data from the camera and microphone as input. This data is analyzed by an emotion recognition model to identify the user's emotional state (e.g., stress, relaxation). Specifically, facial expression recognition and voice tone analysis are used. The output provides the user's current emotional state.

[0394] Step 4:

[0395] The server generates suggestions based on energy usage patterns and emotional states. It uses the outputs from steps 2 and 3 as input. Leveraging a generative AI model, it creates an optimal energy consumption plan that considers the user's health and comfort. For example, for a stressed user, it can suggest relaxing lighting settings. The output includes specific action suggestions and proposed setting changes.

[0396] Step 5:

[0397] The terminal notifies the user of the suggestion from the server. It receives the suggestion generated in step 4 as input. The terminal presents the notification to the user visually and audibly, prompting them to review the suggestion. Specific actions include displaying a notification on the smartphone screen and providing an audio notification. The output ensures that the suggestion is reliably communicated to the user.

[0398] Step 6:

[0399] The user responds to the suggestion displayed on the device. As input, they receive a notification in step 5. The user reviews the suggestion and decides whether to perform it manually based on their choice or allow automated control. Specifically, they can perform the action from the smartphone app or press the "Approve" button to leave it to the system. As output, an action occurs according to the user's choice.

[0400] Step 7:

[0401] The server automatically controls and adjusts the home environment based on user selections. It also collects local activity information and provides personalized event information based on the user's emotional state and interests. Inputs include the user selections from step 6 and local activity information. Based on the selections, the server changes lighting and music settings and notifies the device of appropriate local events. Outputs include the adjusted home settings and local event notifications.

[0402] (Application Example 2)

[0403] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0404] In modern home and urban energy management, achieving both efficient energy consumption and improved quality of life for residents simultaneously is a challenging task. In particular, the emotional state of individual household residents significantly influences energy consumption patterns and their willingness to participate in community activities, but current technology struggles to effectively consider this and optimize the entire system. Furthermore, while personalized information provision based on the emotions and interests of individual residents is needed in community life, there are limitations to achieving this.

[0405] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0406] In this invention, the server includes means for collecting household energy consumption data, means for analyzing energy usage patterns, and means for detecting emotional states and adjusting optimization suggestions. This enables energy-efficient lifestyle suggestions based on the emotional states of individual residents. Furthermore, it can customize local event information based on emotional states to increase motivation to participate.

[0407] "Means of collecting household energy consumption data" refers to devices and technologies that collect the amount of energy consumed and usage patterns through various sensors and smart meters installed in the home.

[0408] "Means of analyzing usage patterns" refers to technologies that analyze trends and characteristics of energy use for specific devices or time periods based on collected energy consumption data.

[0409] "Means for generating proposals to optimize energy efficiency" refers to technologies that utilize the results of usage pattern analysis to design specific advice and strategies for reducing and improving energy consumption.

[0410] "Means of notifying user terminals" refers to technologies that notify users within their home using devices such as smartphones and computers of the generated suggestions.

[0411] "Means for detecting emotional states" refers to technologies that use cameras and voice analysis techniques to identify a user's emotional state from their facial expressions and tone of voice.

[0412] "Means for adjusting optimization suggestions" refers to technologies that flexibly modify the content of energy efficiency suggestions, taking into account the user's emotional state.

[0413] "Means of collecting local event information" refers to technologies for gathering information about events and activities held in a geographically specific area.

[0414] "Means of customizing information" refers to technologies that adjust collected event information to be optimal for individual users based on their interests and emotional state.

[0415] This invention is a system that optimizes household energy consumption and improves engagement with the local community based on the emotional state of residents. The server collects energy consumption data in real time from smart meters and various sensors installed in the home. Using this data, software with a dedicated algorithm is used to analyze energy usage patterns.

[0416] Next, the server uses emotion recognition technology to detect the user's emotional state. This technology is implemented through facial expression analysis using a camera and voice tone analysis. This allows the server to determine the user's stress level and relaxation level.

[0417] Based on the emotional data and usage pattern analysis, the server generates suggestions to optimize energy efficiency. If the user is experiencing stress, for example, it might suggest relaxing settings such as lowering the lighting.

[0418] Furthermore, the server collects local event information and provides customized suggestions to encourage participation in events tailored to the user's emotional state. For example, it can send a specific message to the user such as, "There's a relaxing yoga class available. Would you like to join?"

[0419] These suggestions and notifications are delivered to the user's device via smartphone or computer. The user can review the notification and either manually follow the suggestion or let the system handle the settings in automatic control mode.

[0420] As a concrete example, if a user is feeling irritated at home, the server can detect this and provide information about nearby art therapy events, along with suggestions to gently adjust the lighting. This functionality is implemented using Python and emotion recognition modules such as EmotionEngine.

[0421] An example of a prompt might be, "In a smart city, consider ways to optimize energy consumption and encourage event participation based on user emotions."

[0422] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0423] Step 1:

[0424] The server collects energy consumption data in real time from smart meters and sensors installed in homes. This input data includes power consumption, usage time, and consumption patterns for each device. The data is initially processed and stored in a database, which provides the foundational information for subsequent analysis.

[0425] Step 2:

[0426] The server applies algorithms to the collected energy consumption data to analyze energy usage patterns. Time-series analysis is performed as a data processing step to extract consumption trends for each time period. The output includes usage characteristics by day of the week and peak consumption patterns. Based on these results, areas for improvement in energy management are identified.

[0427] Step 3:

[0428] The server uses a camera and microphone to detect the user's emotional state in real time. It uses facial expression data from the camera and tone data from the audio as input. An emotion recognition algorithm determines the user's emotional state, such as whether they are stressed or relaxed, and records the result as output.

[0429] Step 4:

[0430] The server integrates analysis results of energy consumption patterns with the user's emotional state to generate specific suggestions for optimizing energy efficiency. Using analysis results and emotional data as input, it creates adjusted energy efficiency suggestions as output. For example, if the user is stressed, it might suggest lighting settings that promote relaxation.

[0431] Step 5:

[0432] The server collects local event information and generates customized event participation suggestions based on the user's emotional state. Here, local event information is taken as input, events are filtered considering the emotional state and interests, and output as recommended events for the user.

[0433] Step 6:

[0434] The terminal notifies the user of energy optimization suggestions and event participation suggestions sent from the server. The notifications include specific action suggestions and invitations to participate in events. The user reviews the notifications and either manually implements the suggestions or selects automatic control mode to let the terminal handle it.

[0435] Step 7:

[0436] Users make decisions regarding energy settings and event participation based on suggestions through their device. The device feeds the user's decisions back to the server, which then uses this feedback to make future suggestions more personalized. This establishes a continuous optimization cycle.

[0437] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0438] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0439] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0440] [Third Embodiment]

[0441] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0442] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0443] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0444] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0445] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0446] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0447] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0448] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0449] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0450] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0451] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0452] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0453] Possible embodiments for implementing the present invention include an energy management system and a local community information exchange system. The specific implementation methods for each system will be described below.

[0454] First, to manage energy consumption within the home, the server collects real-time energy consumption data from smart meters installed in each home. This data includes detailed usage information, such as which devices are consuming how much energy at what time of day.

[0455] The collected data is stored in a server database, and an AI agent analyzes this data using machine learning techniques. Based on the analysis, the server identifies energy usage patterns and generates suggestions for optimizing energy efficiency. For example, it can suggest operating air conditioners, which consume a lot of energy at night, during peak solar power generation hours in the daytime.

[0456] Next, the generated suggestions are sent from the server to the user's terminal. The user receives these notifications through their terminal and can select the specified action. The user can also enable the system's automatic control function, allowing device settings to be automatically changed at specified times.

[0457] Furthermore, to strengthen ties with local communities, the server regularly collects information on eco-events and activities held within the region. This includes information from local governments and eco-organizations. The collected information is customized based on the user's profile and interests and provided to their device. Users can view recommended event information on their device and have the opportunity to participate in events that interest them.

[0458] For example, if the server gathers information that "an eco-fair is being held nearby this weekend," it will prioritize notifying users with young children, taking into account that the event is family-friendly. In this way, users can deepen their interaction with local residents through participation in events.

[0459] As described above, the present invention aims to optimize energy use and promote participation in local communities, and the entire system is configured to work together organically to support the user's eco-lifestyle.

[0460] The following describes the processing flow.

[0461] Step 1:

[0462] The server collects real-time energy usage data from smart meters installed in homes. This data includes usage time, consumption, and type of each device.

[0463] Step 2:

[0464] The server stores and accumulates the collected data in a database. This records usage history from the past to the present, laying the foundation for analysis.

[0465] Step 3:

[0466] The server uses machine learning algorithms to analyze information in the database and identify energy usage patterns within the home. This analysis detects peak energy consumption and wasteful consumption.

[0467] Step 4:

[0468] Based on the analysis results, the AI ​​agent generates suggestions for optimizing energy efficiency. These suggestions include recommendations for optimal device usage time and energy-saving mode settings.

[0469] Step 5:

[0470] The server notifies the user's device of the generated suggestions. These notifications include specific actions, such as, "Please adjust your air conditioner usage from nighttime to daytime solar power generation hours."

[0471] Step 6:

[0472] The user reviews the suggestions received through the terminal and either manually changes the settings or enables the automatic control function to allow the system to automatically adjust the device settings.

[0473] Step 7:

[0474] On the other hand, the server collects information on local eco-events, obtaining data from local governments and partner organizations. This information includes details such as the location, date and time, and target audience.

[0475] Step 8:

[0476] The AI ​​agent customizes the collected event information based on the user's profile and selects events of interest.

[0477] Step 9:

[0478] The server notifies the user's terminal of customized event information and sends a message such as, "Why not bring your family to the eco-fair being held nearby this weekend?"

[0479] Step 10:

[0480] Users check event information on their devices, decide to participate in events that interest them, and gain opportunities to interact with local communities through participation.

[0481] (Example 1)

[0482] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0483] This invention aims to effectively manage daily energy consumption within the home, improve energy efficiency, and promote participation in the local community. To achieve this, a system is needed that utilizes household energy consumption data and local activity information to automatically generate personalized suggestions tailored to individual user needs and appropriately notify users of these suggestions. Furthermore, a challenge remains in realizing these functions while minimizing the burden on users.

[0484] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0485] In this invention, the server includes means for collecting energy consumption data within the home, means for analyzing usage patterns based on the data, and means for generating suggestions to optimize energy efficiency based on the analysis results. This enables the generation of efficient suggestions tailored to the energy usage situation within the home and the establishment of effective relationships with the local community.

[0486] "Household energy consumption data" refers to data that measures and records the amount of energy, such as electricity and gas, consumed by various electrical appliances and equipment in a household over a certain period of time.

[0487] A "usage pattern" is a pattern extracted from energy consumption data that shows the trends and regularities in energy consumption within a specific period.

[0488] A "proposal to optimize energy efficiency" is a proposal that outlines specific action guidelines and improvement measures aimed at reducing energy consumption while maintaining necessary convenience, based on usage patterns.

[0489] A "user terminal" is an electronic device owned by an individual and used for energy management and receiving local information, and includes smartphones and tablets.

[0490] "Local activity information" refers to information about events and activities held in a specific area, and is provided with the aim of promoting community engagement.

[0491] "Customization" refers to optimizing the information and recommendations provided based on the user's interests and past participation history, and tailoring them to individual needs.

[0492] A "generative AI model for performing machine learning" is a model based on artificial intelligence technology that analyzes energy consumption and local activity patterns and is used to generate future usage predictions and optimization suggestions.

[0493] A "prompt statement" is a series of sentences that constitute the input information necessary to create suggestions and recommendations for a generative AI model, and is used to improve the accuracy and efficiency of the model.

[0494] Embodiments of the present invention will now be described. The present invention constructs a system for effectively managing household energy consumption and enhancing engagement with the local community. The details are described below.

[0495] First, the server collects energy consumption data in real time from smart meters installed in each home. This uses specific communication protocols (e.g., HTTP or MQTT) to clearly understand energy usage. The data is stored in a database management system (e.g., MySQL or PostgreSQL).

[0496] Next, an AI agent running on the server analyzes the collected data using machine learning techniques (specifically, generative AI models such as TensorFlow and PyTorch). At this stage, an algorithm is applied that learns past patterns and predicts future consumption, generating suggestions for efficiency improvements. These suggestions may include adjusting the usage time of air conditioners and lighting, or utilizing energy-saving modes.

[0497] The generated suggestions are sent from the server to the user's device (e.g., smartphone or tablet) via push notification. The user can then review the suggestions on their device and select the appropriate action. Furthermore, automated control is also achieved using IoT device control protocols (e.g., Zigbee, Z-Wave).

[0498] The server also regularly collects information on local eco-events and activities, customizes it based on the user's profile, and provides it to the device. For example, it collects information such as "an eco-fair is being held nearby this weekend" and sends notifications to families with young children encouraging them to participate.

[0499] For example, if a user is interested in learning how to save energy, the server will send a suggestion to the device saying, "You can save energy by slightly increasing the refrigerator setting." Another example of a prompt for the generating AI model is, "Analyze the user's energy usage patterns and suggest an eco-friendly event suitable for this weekend."

[0500] The system, configured in this way, aims to evolve users' lifestyles in an eco-friendly manner while simultaneously supporting increased engagement between users and their local communities.

[0501] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0502] Step 1:

[0503] The server collects energy consumption data in real time from smart meters installed in homes. It receives energy consumption information from each smart meter as input and generates structured energy consumption data for storage in a database as output. This operation involves data retrieval via communication protocols (e.g., HTTP or MQTT).

[0504] Step 2:

[0505] The server stores the collected energy consumption data in a database. It accepts the structured data formed in step 1 as input, and the data is permanently stored in a database management system (e.g., MySQL or PostgreSQL) as output. This step prepares the data for subsequent analysis processes.

[0506] Step 3:

[0507] The server's AI agent retrieves energy consumption data stored in a database and performs pattern recognition using a generative AI model. The input is historical energy consumption data, and the output is an analysis showing patterns and trends in energy use. Machine learning frameworks such as TensorFlow and PyTorch are used in this step.

[0508] Step 4:

[0509] The server generates suggestions to optimize energy efficiency based on the analysis results. Using pattern data generated by the AI ​​model as input, it outputs specific energy-saving suggestions for the user. At this stage, suggestions such as "Adjust air conditioner usage to peak hours during the day" are formed as prompts.

[0510] Step 5:

[0511] The server notifies the user's device of the generated suggestions. The input is the generated suggestion data, and the output is a notification on the user's smartphone or tablet. Using a push notification service (e.g., Firebase Cloud Messaging), the suggestions are displayed on the device in real time.

[0512] Step 6:

[0513] The user reviews the suggestions received through the device and selects an action as needed. The input is the suggestion information displayed on the device, and the output is the user's selection. Furthermore, if the user agrees, configuration changes for controlling the IoT device are automatically made.

[0514] Step 7:

[0515] The server collects local activity information and customizes it based on user interests. It takes event information from local governments and activity groups as input, and outputs event notifications optimized for each individual user. In addition to data collection from information sources, matching is performed using user profiles.

[0516] Step 8:

[0517] Users participate in local events of interest based on customized information presented on their devices. The input is the customized event information displayed on the device, and the output is the user's decision to participate in the events and their engagement. Through this process, connections with the local community are strengthened.

[0518] (Application Example 1)

[0519] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0520] In modern cities, there is a need to optimize household energy consumption and strengthen engagement with the local community. However, current systems do not adequately manage energy in real time or promote appropriate participation in local events. As a result, energy waste and lack of participation in community activities occur. A system is needed to efficiently solve these problems.

[0521] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0522] In this invention, the server includes a device for collecting household energy consumption data, a device for analyzing usage patterns based on the data, and a device for generating suggestions to optimize energy efficiency based on the analysis results. This makes it possible to reduce wasted energy consumption and notify the user of optimal energy usage suggestions in real time.

[0523] "Household energy consumption data" refers to information that shows how energy such as electricity and gas is used within a home.

[0524] "Usage patterns" refer to the results of an analysis of trends and characteristics of energy use over a specific period.

[0525] "Suggestions for optimizing energy efficiency" refer to providing specific advice and instructions for reducing energy consumption or using energy more efficiently.

[0526] A "user information terminal" is a device used to receive notifications and suggestions from the energy management system, such as a smartphone or tablet.

[0527] "Local activity information" refers to information about events and community activities that take place within a specific area.

[0528] A "device that sends push notifications based on user interests" refers to a system that transmits relevant information to the user's information terminal in real time, based on the user's interests and preferences.

[0529] The embodiments for carrying out the invention are shown below.

[0530] This invention employs a system in which a server collects energy consumption data in real time from smart meters installed in homes. The smart meters record detailed energy usage in each home and transmit this data to the server. This data is stored using an internal database system, such as MongoDB, on the server. The collected data is analyzed using machine learning software such as TensorFlow to identify patterns in energy use.

[0531] The server generates suggestions for optimizing energy efficiency based on the analysis results. These suggestions are then pushed to the user's information terminal using a notification service such as Firebase. This allows the user to receive concrete actions for energy saving in real time.

[0532] Furthermore, the server has the ability to collect information on activities taking place within the region via the internet and to filter and customize it based on the user's interests. Activity information selected according to the user's interests is then pushed to the user's smartphone or tablet.

[0533] Specific examples include suggestions to refrain from using air conditioners during peak energy consumption hours, and notifications encouraging participation in eco-fairs held on weekends. When using a generative AI model, it is possible to generate specific suggestions by using the following prompt statements.

[0534] "Generate suggestions for optimizing energy efficiency based on the user's household energy consumption data for the past month."

[0535] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0536] Step 1:

[0537] The server receives real-time energy consumption data from smart meters installed in homes. This data includes energy usage by time of day and usage status of each device. The server records the acquired data in its database system. The server structures this data and stores it in a large database to facilitate subsequent analysis processes.

[0538] Step 2:

[0539] The server uses collected energy consumption data as input and analyzes it with machine learning models utilizing TensorFlow, etc. Specifically, it identifies consumption patterns and generates suggestions for efficient energy use. This analysis process detects outliers and compares them to normal conditions to discover patterns that can improve energy efficiency.

[0540] Step 3:

[0541] The server sends energy efficiency suggestions, generated based on the analysis results, to the user's device via a notification service such as Firebase. Users can reduce energy waste by receiving these notifications and following the suggested methods. The notifications include specific advice, such as which devices to use and when, for maximum efficiency.

[0542] Step 4:

[0543] The server collects local activity information from the internet. This includes local event calendars and information provided by local governments. The collected information is then filtered based on the user's profile and past participation history, and the appropriate information is selected.

[0544] Step 5:

[0545] Information on selected local activities is customized to the user's interests and delivered via push notifications. Through these notifications, users can participate in local events that interest them and deepen their engagement with the community. This customization utilizes the results of an analysis of the user's past interests and tendencies.

[0546] Step 6:

[0547] When users participate in a local event, they can provide feedback using their device. The server collects this feedback information and uses it as data to make more accurate suggestions for future events. This feedback data is used to analyze the popularity of the event and the satisfaction level of the participants.

[0548] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0549] One possible embodiment of the present invention is a system that combines an emotion engine with a system that optimizes household energy consumption and promotes participation in local events. This system utilizes various devices installed in the home and a user terminal, and by using the emotion engine, provides more personalized suggestions.

[0550] First, the server collects energy consumption data from smart meters and smart devices in the home and stores it in a database. This data includes the usage time, energy consumption, and type of energy used for each device. Based on this data, the server identifies energy usage patterns and generates suggestions for optimization.

[0551] Next, the system's built-in emotion engine recognizes the user's emotional state in real time. Emotion recognition is based on facial expression analysis using a camera, voice tone analysis, and user interaction logs.

[0552] The server incorporates emotional data recognized by the emotion engine and adjusts energy optimization suggestions to suit the user's psychological state. For example, if the system determines that the user is stressed, it can suggest lighting settings that promote relaxation.

[0553] The user terminal will be notified of the adjusted suggestions. The user can review the notification and manually configure the suggested energy settings and action instructions, or select automatic control mode and let the system handle it.

[0554] Furthermore, the server collects local event information and uses an emotion engine to select events that match the user's interests and mood. This information is notified to the user's device and accompanied by specific messages to encourage participation. For example, it might say, "If you're feeling refreshed, why not join a yoga workshop that's sure to have a relaxing effect?"

[0555] For example, if a user is feeling irritated at home, the emotion engine will detect this, and the server will suggest lowering the lighting and playing calming music. This suggestion will be notified to the device, and if the user agrees, the settings will be automatically applied. Information about local art therapy events will also be appropriately notified.

[0556] Thus, the present invention is a system that combines efficient energy management with the provision of services based on the user's emotional state, thereby improving the user experience and promoting active engagement with the local community.

[0557] The following describes the processing flow.

[0558] Step 1:

[0559] The server collects real-time energy consumption data from smart meters and smart appliances within the home. This data includes the usage time and power consumption of each device.

[0560] Step 2:

[0561] The server stores the collected data in a database. This allows for the accumulation of past usage history, which can then be used for later analysis.

[0562] Step 3:

[0563] The server uses machine learning algorithms to analyze energy consumption information stored in the database and identify usage patterns. This analysis reveals peak consumption and wasteful consumption.

[0564] Step 4:

[0565] The system's built-in emotion engine analyzes the user's facial expressions and voice tone to recognize emotions in real time. User interaction logs are also used for emotion recognition.

[0566] Step 5:

[0567] The server adjusts suggestions to optimize energy efficiency based on the user's emotional state, as recognized by the emotion engine. For example, if the user is stressed, it might suggest changing the lighting to a softer, more relaxing light.

[0568] Step 6:

[0569] The server notifies the user's terminal of the adjusted suggestions. The notification includes the suggested energy settings and activity instructions.

[0570] Step 7:

[0571] The user checks the notification on their device and can choose to manually change device settings according to the suggestion or leave it to the system to take automatic control.

[0572] Step 8:

[0573] The server regularly collects information on events held within the region, including details on eco-related activities and local events.

[0574] Step 9:

[0575] The emotion engine selects events that are likely to be of interest to the user based on their current emotions. The selected event information is tailored to the user's profile and emotions.

[0576] Step 10:

[0577] The server notifies the user's device of event information that matches their interests and mood, along with a message encouraging them to participate. The user can check the notification and register to participate in events they like.

[0578] (Example 2)

[0579] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0580] In modern homes, a variety of devices exist, energy consumption is increasing, and efficient resource management is required. Furthermore, creating a comfortable living environment that suits individual emotions and actively participating in community activities are also important issues. However, existing systems often fail to adequately provide optimized solutions that consider these individual needs. Therefore, the present invention aims to realize a system that improves the efficiency of resource management within the home and provides personalized suggestions that respond to the user's emotional state.

[0581] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0582] In this invention, the server includes means for collecting resource consumption information within the home, means for recognizing emotional states using an emotion engine, and means for improving the quality of information and suggestions using a generative AI model. This enables efficient resource management and the generation of comfortable suggestions that respond to the user's emotions.

[0583] "Resource consumption information" refers to data on all resources used within a household, specifically including information on the amount of electricity, water, and gas used.

[0584] The "emotion engine" is a function that analyzes and recognizes the user's emotional state from factors such as voice tone and facial expressions.

[0585] A "generative AI model" refers to an algorithm that uses artificial intelligence technology to improve the accuracy of data analysis and proposal generation.

[0586] A "means for adjusting suggestions" refers to a mechanism for appropriately modifying the content and method of generated suggestions, taking into account dynamic factors such as the user's emotional state.

[0587] A "terminal" is a device used to notify users of energy consumption suggestions and local activity information, and includes smartphones and tablets.

[0588] "Automatic control" refers to a system or device operating autonomously based on pre-set rules, without requiring manual operation by the user.

[0589] "Local activity information" refers to information about events and activities held within a specific region, and is intended to encourage active participation from users.

[0590] "Personalized experiences" refer to services and information tailored to the user's emotional state and interests, meaning that different values ​​are provided for each user.

[0591] This system is designed to optimize resource consumption within the home and provide suggestions tailored to the user's emotions. Specifically, the server, terminals, and users each play their respective roles, facilitating the smooth collection, analysis, and notification of information.

[0592] The server collects resource consumption information from multiple sensors and network-connected devices installed within the home. This utilizes smart meters and other smart devices, storing the collected data in a central database. Furthermore, the server performs data analysis using software libraries such as Python's Pandas and SciPy. This identifies resource usage patterns and generates optimization suggestions to improve energy efficiency.

[0593] Regarding emotion recognition, the server uses an emotion engine to analyze the user's voice tone and facial expression data. This analysis incorporates an interface that utilizes a camera and microphone. As a result of the analysis, the server can understand the user's emotional state in real time. The server combines this emotion data with resource consumption information to provide specific and effective suggestions tailored to the user.

[0594] The terminal's role is to inform the user of suggestions and notifications sent from the server. Devices such as smartphones and tablets are used, allowing users to review suggestions and choose how to respond. For example, it's possible to suggest relaxing lighting settings or music playback to a user experiencing stress. If the user approves the suggestion, the terminal automatically continues the settings.

[0595] Furthermore, the server collects local activity information and provides personalized information based on emotions and interests. This allows users to participate in local events that match their mood and interests, deepening their connection with the community. For example, prompts such as, "If you're feeling refreshed, why not join a yoga workshop that can help you relax?" are used to encourage participation.

[0596] This system utilizes generative AI models to improve the quality of suggestions and make users' lives more comfortable and efficient.

[0597] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0598] Step 1:

[0599] The server collects resource consumption information from smart meters and smart devices within the home. It receives usage data from each device as input. The server receives this data and stores it in a database. The collected data includes time-series consumption and usage time for each device. Specifically, it accesses each device at regular intervals to retrieve information and aggregates it at the central server. The output is organized resource consumption information.

[0600] Step 2:

[0601] The server performs data analysis based on the collected resource consumption information. The resource consumption information collected in Step 1 is used as input. Trends and outliers in the data are analyzed using Python's Pandas and SciPy libraries. For example, historical data is analyzed to identify patterns of sharp increases in energy consumption during specific time periods. The output reports energy usage patterns and potential optimization opportunities.

[0602] Step 3:

[0603] The server uses an emotion engine to recognize the user's emotional state. It acquires real-time data from the camera and microphone as input. This data is analyzed by an emotion recognition model to identify the user's emotional state (e.g., stress, relaxation). Specifically, facial expression recognition and voice tone analysis are used. The output provides the user's current emotional state.

[0604] Step 4:

[0605] The server generates suggestions based on energy usage patterns and emotional states. It uses the outputs from steps 2 and 3 as input. Leveraging a generative AI model, it creates an optimal energy consumption plan that considers the user's health and comfort. For example, for a stressed user, it can suggest relaxing lighting settings. The output includes specific action suggestions and proposed setting changes.

[0606] Step 5:

[0607] The terminal notifies the user of the suggestion from the server. It receives the suggestion generated in step 4 as input. The terminal presents the notification to the user visually and audibly, prompting them to review the suggestion. Specific actions include displaying a notification on the smartphone screen and providing an audio notification. The output ensures that the suggestion is reliably communicated to the user.

[0608] Step 6:

[0609] The user responds to the suggestion displayed on the device. As input, they receive a notification in step 5. The user reviews the suggestion and decides whether to perform it manually based on their choice or allow automated control. Specifically, they can perform the action from the smartphone app or press the "Approve" button to leave it to the system. As output, an action occurs according to the user's choice.

[0610] Step 7:

[0611] The server automatically controls and adjusts the home environment based on user selections. It also collects local activity information and provides personalized event information based on the user's emotional state and interests. Inputs include the user selections from step 6 and local activity information. Based on the selections, the server changes lighting and music settings and notifies the device of appropriate local events. Outputs include the adjusted home settings and local event notifications.

[0612] (Application Example 2)

[0613] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0614] In modern home and urban energy management, achieving both efficient energy consumption and improved quality of life for residents simultaneously is a challenging task. In particular, the emotional state of individual household residents significantly influences energy consumption patterns and their willingness to participate in community activities, but current technology struggles to effectively consider this and optimize the entire system. Furthermore, while personalized information provision based on the emotions and interests of individual residents is needed in community life, there are limitations to achieving this.

[0615] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0616] In this invention, the server includes means for collecting household energy consumption data, means for analyzing energy usage patterns, and means for detecting emotional states and adjusting optimization suggestions. This enables energy-efficient lifestyle suggestions based on the emotional states of individual residents. Furthermore, it can customize local event information based on emotional states to increase motivation to participate.

[0617] "Means of collecting household energy consumption data" refers to devices and technologies that collect the amount of energy consumed and usage patterns through various sensors and smart meters installed in the home.

[0618] "Means of analyzing usage patterns" refers to technologies that analyze trends and characteristics of energy use for specific devices or time periods based on collected energy consumption data.

[0619] "Means for generating proposals to optimize energy efficiency" refers to technologies that utilize the results of usage pattern analysis to design specific advice and strategies for reducing and improving energy consumption.

[0620] "Means of notifying user terminals" refers to technologies that notify users within their home using devices such as smartphones and computers of the generated suggestions.

[0621] "Means for detecting emotional states" refers to technologies that use cameras and voice analysis techniques to identify a user's emotional state from their facial expressions and tone of voice.

[0622] "Means for adjusting optimization suggestions" refers to technologies that flexibly modify the content of energy efficiency suggestions, taking into account the user's emotional state.

[0623] "Means of collecting local event information" refers to technologies for gathering information about events and activities held in a geographically specific area.

[0624] "Means of customizing information" refers to technologies that adjust collected event information to be optimal for individual users based on their interests and emotional state.

[0625] This invention is a system that optimizes household energy consumption and improves engagement with the local community based on the emotional state of residents. The server collects energy consumption data in real time from smart meters and various sensors installed in the home. Using this data, software with a dedicated algorithm is used to analyze energy usage patterns.

[0626] Next, the server uses emotion recognition technology to detect the user's emotional state. This technology is implemented through facial expression analysis using a camera and voice tone analysis. This allows the server to determine the user's stress level and relaxation level.

[0627] Based on the emotional data and usage pattern analysis, the server generates suggestions to optimize energy efficiency. If the user is experiencing stress, for example, it might suggest relaxing settings such as lowering the lighting.

[0628] Furthermore, the server collects local event information and provides customized suggestions to encourage participation in events tailored to the user's emotional state. For example, it can send a specific message to the user such as, "There's a relaxing yoga class available. Would you like to join?"

[0629] These suggestions and notifications are delivered to the user's device via smartphone or computer. The user can review the notification and either manually follow the suggestion or let the system handle the settings in automatic control mode.

[0630] As a concrete example, if a user is feeling irritated at home, the server can detect this and provide information about nearby art therapy events, along with suggestions to gently adjust the lighting. This functionality is implemented using Python and emotion recognition modules such as EmotionEngine.

[0631] An example of a prompt might be, "In a smart city, consider ways to optimize energy consumption and encourage event participation based on user emotions."

[0632] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0633] Step 1:

[0634] The server collects energy consumption data in real time from smart meters and sensors installed in homes. This input data includes power consumption, usage time, and consumption patterns for each device. The data is initially processed and stored in a database, which provides the foundational information for subsequent analysis.

[0635] Step 2:

[0636] The server applies algorithms to the collected energy consumption data to analyze energy usage patterns. Time-series analysis is performed as a data processing step to extract consumption trends for each time period. The output includes usage characteristics by day of the week and peak consumption patterns. Based on these results, areas for improvement in energy management are identified.

[0637] Step 3:

[0638] The server uses a camera and microphone to detect the user's emotional state in real time. It uses facial expression data from the camera and tone data from the audio as input. An emotion recognition algorithm determines the user's emotional state, such as whether they are stressed or relaxed, and records the result as output.

[0639] Step 4:

[0640] The server integrates analysis results of energy consumption patterns with the user's emotional state to generate specific suggestions for optimizing energy efficiency. Using analysis results and emotional data as input, it creates adjusted energy efficiency suggestions as output. For example, if the user is stressed, it might suggest lighting settings that promote relaxation.

[0641] Step 5:

[0642] The server collects local event information and generates customized event participation suggestions based on the user's emotional state. Here, local event information is taken as input, events are filtered considering the emotional state and interests, and output as recommended events for the user.

[0643] Step 6:

[0644] The terminal notifies the user of energy optimization suggestions and event participation suggestions sent from the server. The notifications include specific action suggestions and invitations to participate in events. The user reviews the notifications and either manually implements the suggestions or selects automatic control mode to let the terminal handle it.

[0645] Step 7:

[0646] Users make decisions regarding energy settings and event participation based on suggestions through their device. The device feeds the user's decisions back to the server, which then uses this feedback to make future suggestions more personalized. This establishes a continuous optimization cycle.

[0647] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0648] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0649] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0650] [Fourth Embodiment]

[0651] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0652] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0653] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0654] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0655] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0656] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0657] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0658] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0659] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0660] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0661] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0662] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0663] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0664] Possible embodiments for implementing the present invention include an energy management system and a local community information exchange system. The specific implementation methods for each system will be described below.

[0665] First, to manage energy consumption within the home, the server collects real-time energy consumption data from smart meters installed in each home. This data includes detailed usage information, such as which devices are consuming how much energy at what time of day.

[0666] The collected data is stored in a server database, and an AI agent analyzes this data using machine learning techniques. Based on the analysis, the server identifies energy usage patterns and generates suggestions for optimizing energy efficiency. For example, it can suggest operating air conditioners, which consume a lot of energy at night, during peak solar power generation hours in the daytime.

[0667] Next, the generated suggestions are sent from the server to the user's terminal. The user receives these notifications through their terminal and can select the specified action. The user can also enable the system's automatic control function, allowing device settings to be automatically changed at specified times.

[0668] Furthermore, to strengthen ties with local communities, the server regularly collects information on eco-events and activities held within the region. This includes information from local governments and eco-organizations. The collected information is customized based on the user's profile and interests and provided to their device. Users can view recommended event information on their device and have the opportunity to participate in events that interest them.

[0669] For example, if the server gathers information that "an eco-fair is being held nearby this weekend," it will prioritize notifying users with young children, taking into account that the event is family-friendly. In this way, users can deepen their interaction with local residents through participation in events.

[0670] As described above, the present invention aims to optimize energy use and promote participation in local communities, and the entire system is configured to work together organically to support the user's eco-lifestyle.

[0671] The following describes the processing flow.

[0672] Step 1:

[0673] The server collects real-time energy usage data from smart meters installed in homes. This data includes usage time, consumption, and type of each device.

[0674] Step 2:

[0675] The server stores and accumulates the collected data in a database. This records usage history from the past to the present, laying the foundation for analysis.

[0676] Step 3:

[0677] The server uses machine learning algorithms to analyze information in the database and identify energy usage patterns within the home. This analysis detects peak energy consumption and wasteful consumption.

[0678] Step 4:

[0679] Based on the analysis results, the AI ​​agent generates suggestions for optimizing energy efficiency. These suggestions include recommendations for optimal device usage time and energy-saving mode settings.

[0680] Step 5:

[0681] The server notifies the user's device of the generated suggestions. These notifications include specific actions, such as, "Please adjust your air conditioner usage from nighttime to daytime solar power generation hours."

[0682] Step 6:

[0683] The user reviews the suggestions received through the terminal and either manually changes the settings or enables the automatic control function to allow the system to automatically adjust the device settings.

[0684] Step 7:

[0685] On the other hand, the server collects information on local eco-events, obtaining data from local governments and partner organizations. This information includes details such as the location, date and time, and target audience.

[0686] Step 8:

[0687] The AI ​​agent customizes the collected event information based on the user's profile and selects events of interest.

[0688] Step 9:

[0689] The server notifies the user's terminal of customized event information and sends a message such as, "Why not bring your family to the eco-fair being held nearby this weekend?"

[0690] Step 10:

[0691] Users check event information on their devices, decide to participate in events that interest them, and gain opportunities to interact with local communities through participation.

[0692] (Example 1)

[0693] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0694] This invention aims to effectively manage daily energy consumption within the home, improve energy efficiency, and promote participation in the local community. To achieve this, a system is needed that utilizes household energy consumption data and local activity information to automatically generate personalized suggestions tailored to individual user needs and appropriately notify users of these suggestions. Furthermore, a challenge remains in realizing these functions while minimizing the burden on users.

[0695] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0696] In this invention, the server includes means for collecting energy consumption data within the home, means for analyzing usage patterns based on the data, and means for generating suggestions to optimize energy efficiency based on the analysis results. This enables the generation of efficient suggestions tailored to the energy usage situation within the home and the establishment of effective relationships with the local community.

[0697] "Household energy consumption data" refers to data that measures and records the amount of energy, such as electricity and gas, consumed by various electrical appliances and equipment in a household over a certain period of time.

[0698] A "usage pattern" is a pattern extracted from energy consumption data that shows the trends and regularities in energy consumption within a specific period.

[0699] A "proposal to optimize energy efficiency" is a proposal that outlines specific action guidelines and improvement measures aimed at reducing energy consumption while maintaining necessary convenience, based on usage patterns.

[0700] A "user terminal" is an electronic device owned by an individual and used for energy management and receiving local information, and includes smartphones and tablets.

[0701] "Local activity information" refers to information about events and activities held in a specific area, and is provided with the aim of promoting community engagement.

[0702] "Customization" refers to optimizing the information and recommendations provided based on the user's interests and past participation history, and tailoring them to individual needs.

[0703] A "generative AI model for performing machine learning" is a model based on artificial intelligence technology that analyzes energy consumption and local activity patterns and is used to generate future usage predictions and optimization suggestions.

[0704] A "prompt statement" is a series of sentences that constitute the input information necessary to create suggestions and recommendations for a generative AI model, and is used to improve the accuracy and efficiency of the model.

[0705] Embodiments of the present invention will now be described. The present invention constructs a system for effectively managing household energy consumption and enhancing engagement with the local community. The details are described below.

[0706] First, the server collects energy consumption data in real time from smart meters installed in each home. This uses specific communication protocols (e.g., HTTP or MQTT) to clearly understand energy usage. The data is stored in a database management system (e.g., MySQL or PostgreSQL).

[0707] Next, an AI agent running on the server analyzes the collected data using machine learning techniques (specifically, generative AI models such as TensorFlow and PyTorch). At this stage, an algorithm is applied that learns past patterns and predicts future consumption, generating suggestions for efficiency improvements. These suggestions may include adjusting the usage time of air conditioners and lighting, or utilizing energy-saving modes.

[0708] The generated suggestions are sent from the server to the user's device (e.g., smartphone or tablet) via push notification. The user can then review the suggestions on their device and select the appropriate action. Furthermore, automated control is also achieved using IoT device control protocols (e.g., Zigbee, Z-Wave).

[0709] The server also regularly collects information on local eco-events and activities, customizes it based on the user's profile, and provides it to the device. For example, it collects information such as "an eco-fair is being held nearby this weekend" and sends notifications to families with young children encouraging them to participate.

[0710] For example, if a user is interested in learning how to save energy, the server will send a suggestion to the device saying, "You can save energy by slightly increasing the refrigerator setting." Another example of a prompt for the generating AI model is, "Analyze the user's energy usage patterns and suggest an eco-friendly event suitable for this weekend."

[0711] The system, configured in this way, aims to evolve users' lifestyles in an eco-friendly manner while simultaneously supporting increased engagement between users and their local communities.

[0712] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0713] Step 1:

[0714] The server collects energy consumption data in real time from smart meters installed in homes. It receives energy consumption information from each smart meter as input and generates structured energy consumption data for storage in a database as output. This operation involves data retrieval via communication protocols (e.g., HTTP or MQTT).

[0715] Step 2:

[0716] The server stores the collected energy consumption data in a database. It accepts the structured data formed in step 1 as input, and the data is permanently stored in a database management system (e.g., MySQL or PostgreSQL) as output. This step prepares the data for subsequent analysis processes.

[0717] Step 3:

[0718] The server's AI agent retrieves energy consumption data stored in a database and performs pattern recognition using a generative AI model. The input is historical energy consumption data, and the output is an analysis showing patterns and trends in energy use. Machine learning frameworks such as TensorFlow and PyTorch are used in this step.

[0719] Step 4:

[0720] The server generates suggestions to optimize energy efficiency based on the analysis results. Using pattern data generated by the AI ​​model as input, it outputs specific energy-saving suggestions for the user. At this stage, suggestions such as "Adjust air conditioner usage to peak hours during the day" are formed as prompts.

[0721] Step 5:

[0722] The server notifies the user's device of the generated suggestions. The input is the generated suggestion data, and the output is a notification on the user's smartphone or tablet. Using a push notification service (e.g., Firebase Cloud Messaging), the suggestions are displayed on the device in real time.

[0723] Step 6:

[0724] The user reviews the suggestions received through the device and selects an action as needed. The input is the suggestion information displayed on the device, and the output is the user's selection. Furthermore, if the user agrees, configuration changes for controlling the IoT device are automatically made.

[0725] Step 7:

[0726] The server collects local activity information and customizes it based on user interests. It takes event information from local governments and activity groups as input, and outputs event notifications optimized for each individual user. In addition to data collection from information sources, matching is performed using user profiles.

[0727] Step 8:

[0728] Users participate in local events of interest based on customized information presented on their devices. The input is the customized event information displayed on the device, and the output is the user's decision to participate in the events and their engagement. Through this process, connections with the local community are strengthened.

[0729] (Application Example 1)

[0730] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0731] In modern cities, there is a need to optimize household energy consumption and strengthen engagement with the local community. However, current systems do not adequately manage energy in real time or promote appropriate participation in local events. As a result, energy waste and lack of participation in community activities occur. A system is needed to efficiently solve these problems.

[0732] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0733] In this invention, the server includes a device for collecting household energy consumption data, a device for analyzing usage patterns based on the data, and a device for generating suggestions to optimize energy efficiency based on the analysis results. This makes it possible to reduce wasted energy consumption and notify the user of optimal energy usage suggestions in real time.

[0734] "Household energy consumption data" refers to information that shows how energy such as electricity and gas is used within a home.

[0735] "Usage patterns" refer to the results of an analysis of trends and characteristics of energy use over a specific period.

[0736] "Suggestions for optimizing energy efficiency" refer to providing specific advice and instructions for reducing energy consumption or using energy more efficiently.

[0737] A "user information terminal" is a device used to receive notifications and suggestions from the energy management system, such as a smartphone or tablet.

[0738] "Local activity information" refers to information about events and community activities that take place within a specific area.

[0739] A "device that sends push notifications based on user interests" refers to a system that transmits relevant information to the user's information terminal in real time, based on the user's interests and preferences.

[0740] The embodiments for carrying out the invention are shown below.

[0741] This invention employs a system in which a server collects energy consumption data in real time from smart meters installed in homes. The smart meters record detailed energy usage in each home and transmit this data to the server. This data is stored using an internal database system, such as MongoDB, on the server. The collected data is analyzed using machine learning software such as TensorFlow to identify patterns in energy use.

[0742] The server generates suggestions for optimizing energy efficiency based on the analysis results. These suggestions are then pushed to the user's information terminal using a notification service such as Firebase. This allows the user to receive concrete actions for energy saving in real time.

[0743] Furthermore, the server has the ability to collect information on activities taking place within the region via the internet and to filter and customize it based on the user's interests. Activity information selected according to the user's interests is then pushed to the user's smartphone or tablet.

[0744] Specific examples include suggestions to refrain from using air conditioners during peak energy consumption hours, and notifications encouraging participation in eco-fairs held on weekends. When using a generative AI model, it is possible to generate specific suggestions by using the following prompt statements.

[0745] "Generate suggestions for optimizing energy efficiency based on the user's household energy consumption data for the past month."

[0746] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0747] Step 1:

[0748] The server receives real-time energy consumption data from smart meters installed in homes. This data includes energy usage by time of day and usage status of each device. The server records the acquired data in its database system. The server structures this data and stores it in a large database to facilitate subsequent analysis processes.

[0749] Step 2:

[0750] The server uses collected energy consumption data as input and analyzes it with machine learning models utilizing TensorFlow, etc. Specifically, it identifies consumption patterns and generates suggestions for efficient energy use. This analysis process detects outliers and compares them to normal conditions to discover patterns that can improve energy efficiency.

[0751] Step 3:

[0752] The server sends energy efficiency suggestions, generated based on the analysis results, to the user's device via a notification service such as Firebase. Users can reduce energy waste by receiving these notifications and following the suggested methods. The notifications include specific advice, such as which devices to use and when, for maximum efficiency.

[0753] Step 4:

[0754] The server collects local activity information from the internet. This includes local event calendars and information provided by local governments. The collected information is then filtered based on the user's profile and past participation history, and the appropriate information is selected.

[0755] Step 5:

[0756] Information on selected local activities is customized to the user's interests and delivered via push notifications. Through these notifications, users can participate in local events that interest them and deepen their engagement with the community. This customization utilizes the results of an analysis of the user's past interests and tendencies.

[0757] Step 6:

[0758] When users participate in a local event, they can provide feedback using their device. The server collects this feedback information and uses it as data to make more accurate suggestions for future events. This feedback data is used to analyze the popularity of the event and the satisfaction level of the participants.

[0759] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0760] One possible embodiment of the present invention is a system that combines an emotion engine with a system that optimizes household energy consumption and promotes participation in local events. This system utilizes various devices installed in the home and a user terminal, and by using the emotion engine, provides more personalized suggestions.

[0761] First, the server collects energy consumption data from smart meters and smart devices in the home and stores it in a database. This data includes the usage time, energy consumption, and type of energy used for each device. Based on this data, the server identifies energy usage patterns and generates suggestions for optimization.

[0762] Next, the system's built-in emotion engine recognizes the user's emotional state in real time. Emotion recognition is based on facial expression analysis using a camera, voice tone analysis, and user interaction logs.

[0763] The server incorporates emotional data recognized by the emotion engine and adjusts energy optimization suggestions to suit the user's psychological state. For example, if the system determines that the user is stressed, it can suggest lighting settings that promote relaxation.

[0764] The user terminal will be notified of the adjusted suggestions. The user can review the notification and manually configure the suggested energy settings and action instructions, or select automatic control mode and let the system handle it.

[0765] Furthermore, the server collects local event information and uses an emotion engine to select events that match the user's interests and mood. This information is notified to the user's device and accompanied by specific messages to encourage participation. For example, it might say, "If you're feeling refreshed, why not join a yoga workshop that's sure to have a relaxing effect?"

[0766] For example, if a user is feeling irritated at home, the emotion engine will detect this, and the server will suggest lowering the lighting and playing calming music. This suggestion will be notified to the device, and if the user agrees, the settings will be automatically applied. Information about local art therapy events will also be appropriately notified.

[0767] Thus, the present invention is a system that combines efficient energy management with the provision of services based on the user's emotional state, thereby improving the user experience and promoting active engagement with the local community.

[0768] The following describes the processing flow.

[0769] Step 1:

[0770] The server collects real-time energy consumption data from smart meters and smart appliances within the home. This data includes the usage time and power consumption of each device.

[0771] Step 2:

[0772] The server stores the collected data in a database. This allows for the accumulation of past usage history, which can then be used for later analysis.

[0773] Step 3:

[0774] The server uses machine learning algorithms to analyze energy consumption information stored in the database and identify usage patterns. This analysis reveals peak consumption and wasteful consumption.

[0775] Step 4:

[0776] The system's built-in emotion engine analyzes the user's facial expressions and voice tone to recognize emotions in real time. User interaction logs are also used for emotion recognition.

[0777] Step 5:

[0778] The server adjusts suggestions to optimize energy efficiency based on the user's emotional state, as recognized by the emotion engine. For example, if the user is stressed, it might suggest changing the lighting to a softer, more relaxing light.

[0779] Step 6:

[0780] The server notifies the user's terminal of the adjusted suggestions. The notification includes the suggested energy settings and activity instructions.

[0781] Step 7:

[0782] The user checks the notification on their device and can choose to manually change device settings according to the suggestion or leave it to the system to take automatic control.

[0783] Step 8:

[0784] The server regularly collects information on events held within the region, including details on eco-related activities and local events.

[0785] Step 9:

[0786] The emotion engine selects events that are likely to be of interest to the user based on their current emotions. The selected event information is tailored to the user's profile and emotions.

[0787] Step 10:

[0788] The server notifies the user's device of event information that matches their interests and mood, along with a message encouraging them to participate. The user can check the notification and register to participate in events they like.

[0789] (Example 2)

[0790] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0791] In modern homes, a variety of devices exist, energy consumption is increasing, and efficient resource management is required. Furthermore, creating a comfortable living environment that suits individual emotions and actively participating in community activities are also important issues. However, existing systems often fail to adequately provide optimized solutions that consider these individual needs. Therefore, the present invention aims to realize a system that improves the efficiency of resource management within the home and provides personalized suggestions that respond to the user's emotional state.

[0792] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0793] In this invention, the server includes means for collecting resource consumption information within the home, means for recognizing emotional states using an emotion engine, and means for improving the quality of information and suggestions using a generative AI model. This enables efficient resource management and the generation of comfortable suggestions that respond to the user's emotions.

[0794] "Resource consumption information" refers to data on all resources used within a household, specifically including information on the amount of electricity, water, and gas used.

[0795] The "emotion engine" is a function that analyzes and recognizes the user's emotional state from factors such as voice tone and facial expressions.

[0796] A "generative AI model" refers to an algorithm that uses artificial intelligence technology to improve the accuracy of data analysis and proposal generation.

[0797] A "means for adjusting suggestions" refers to a mechanism for appropriately modifying the content and method of generated suggestions, taking into account dynamic factors such as the user's emotional state.

[0798] A "terminal" is a device used to notify users of energy consumption suggestions and local activity information, and includes smartphones and tablets.

[0799] "Automatic control" refers to a system or device operating autonomously based on pre-set rules, without requiring manual operation by the user.

[0800] "Local activity information" refers to information about events and activities held within a specific region, and is intended to encourage active participation from users.

[0801] "Personalized experiences" refer to services and information tailored to the user's emotional state and interests, meaning that different values ​​are provided for each user.

[0802] This system is designed to optimize resource consumption within the home and provide suggestions tailored to the user's emotions. Specifically, the server, terminals, and users each play their respective roles, facilitating the smooth collection, analysis, and notification of information.

[0803] The server collects resource consumption information from multiple sensors and network-connected devices installed within the home. This utilizes smart meters and other smart devices, storing the collected data in a central database. Furthermore, the server performs data analysis using software libraries such as Python's Pandas and SciPy. This identifies resource usage patterns and generates optimization suggestions to improve energy efficiency.

[0804] Regarding emotion recognition, the server uses an emotion engine to analyze the user's voice tone and facial expression data. This analysis incorporates an interface that utilizes a camera and microphone. As a result of the analysis, the server can understand the user's emotional state in real time. The server combines this emotion data with resource consumption information to provide specific and effective suggestions tailored to the user.

[0805] The terminal's role is to inform the user of suggestions and notifications sent from the server. Devices such as smartphones and tablets are used, allowing users to review suggestions and choose how to respond. For example, it's possible to suggest relaxing lighting settings or music playback to a user experiencing stress. If the user approves the suggestion, the terminal automatically continues the settings.

[0806] Furthermore, the server collects local activity information and provides personalized information based on emotions and interests. This allows users to participate in local events that match their mood and interests, deepening their connection with the community. For example, prompts such as, "If you're feeling refreshed, why not join a yoga workshop that can help you relax?" are used to encourage participation.

[0807] This system utilizes generative AI models to improve the quality of suggestions and make users' lives more comfortable and efficient.

[0808] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0809] Step 1:

[0810] The server collects resource consumption information from smart meters and smart devices within the home. It receives usage data from each device as input. The server receives this data and stores it in a database. The collected data includes time-series consumption and usage time for each device. Specifically, it accesses each device at regular intervals to retrieve information and aggregates it at the central server. The output is organized resource consumption information.

[0811] Step 2:

[0812] The server performs data analysis based on the collected resource consumption information. The resource consumption information collected in Step 1 is used as input. Trends and outliers in the data are analyzed using Python's Pandas and SciPy libraries. For example, historical data is analyzed to identify patterns of sharp increases in energy consumption during specific time periods. The output reports energy usage patterns and potential optimization opportunities.

[0813] Step 3:

[0814] The server uses an emotion engine to recognize the user's emotional state. It acquires real-time data from the camera and microphone as input. This data is analyzed by an emotion recognition model to identify the user's emotional state (e.g., stress, relaxation). Specifically, facial expression recognition and voice tone analysis are used. The output provides the user's current emotional state.

[0815] Step 4:

[0816] The server generates suggestions based on energy usage patterns and emotional states. It uses the outputs from steps 2 and 3 as input. Leveraging a generative AI model, it creates an optimal energy consumption plan that considers the user's health and comfort. For example, for a stressed user, it can suggest relaxing lighting settings. The output includes specific action suggestions and proposed setting changes.

[0817] Step 5:

[0818] The terminal notifies the user of the suggestion from the server. It receives the suggestion generated in step 4 as input. The terminal presents the notification to the user visually and audibly, prompting them to review the suggestion. Specific actions include displaying a notification on the smartphone screen and providing an audio notification. The output ensures that the suggestion is reliably communicated to the user.

[0819] Step 6:

[0820] The user responds to the suggestion displayed on the device. As input, they receive a notification in step 5. The user reviews the suggestion and decides whether to perform it manually based on their choice or allow automated control. Specifically, they can perform the action from the smartphone app or press the "Approve" button to leave it to the system. As output, an action occurs according to the user's choice.

[0821] Step 7:

[0822] The server automatically controls and adjusts the home environment based on user selections. It also collects local activity information and provides personalized event information based on the user's emotional state and interests. Inputs include the user selections from step 6 and local activity information. Based on the selections, the server changes lighting and music settings and notifies the device of appropriate local events. Outputs include the adjusted home settings and local event notifications.

[0823] (Application Example 2)

[0824] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0825] In modern home and urban energy management, achieving both efficient energy consumption and improved quality of life for residents simultaneously is a challenging task. In particular, the emotional state of individual household residents significantly influences energy consumption patterns and their willingness to participate in community activities, but current technology struggles to effectively consider this and optimize the entire system. Furthermore, while personalized information provision based on the emotions and interests of individual residents is needed in community life, there are limitations to achieving this.

[0826] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0827] In this invention, the server includes means for collecting household energy consumption data, means for analyzing energy usage patterns, and means for detecting emotional states and adjusting optimization suggestions. This enables energy-efficient lifestyle suggestions based on the emotional states of individual residents. Furthermore, it can customize local event information based on emotional states to increase motivation to participate.

[0828] "Means of collecting household energy consumption data" refers to devices and technologies that collect the amount of energy consumed and usage patterns through various sensors and smart meters installed in the home.

[0829] "Means of analyzing usage patterns" refers to technologies that analyze trends and characteristics of energy use for specific devices or time periods based on collected energy consumption data.

[0830] "Means for generating proposals to optimize energy efficiency" refers to technologies that utilize the results of usage pattern analysis to design specific advice and strategies for reducing and improving energy consumption.

[0831] "Means of notifying user terminals" refers to technologies that notify users within their home using devices such as smartphones and computers of the generated suggestions.

[0832] "Means for detecting emotional states" refers to technologies that use cameras and voice analysis techniques to identify a user's emotional state from their facial expressions and tone of voice.

[0833] "Means for adjusting optimization suggestions" refers to technologies that flexibly modify the content of energy efficiency suggestions, taking into account the user's emotional state.

[0834] "Means of collecting local event information" refers to technologies for gathering information about events and activities held in a geographically specific area.

[0835] "Means of customizing information" refers to technologies that adjust collected event information to be optimal for individual users based on their interests and emotional state.

[0836] This invention is a system that optimizes household energy consumption and improves engagement with the local community based on the emotional state of residents. The server collects energy consumption data in real time from smart meters and various sensors installed in the home. Using this data, software with a dedicated algorithm is used to analyze energy usage patterns.

[0837] Next, the server uses emotion recognition technology to detect the user's emotional state. This technology is implemented through facial expression analysis using a camera and voice tone analysis. This allows the server to determine the user's stress level and relaxation level.

[0838] Based on the emotional data and usage pattern analysis, the server generates suggestions to optimize energy efficiency. If the user is experiencing stress, for example, it might suggest relaxing settings such as lowering the lighting.

[0839] Furthermore, the server collects local event information and provides customized suggestions to encourage participation in events tailored to the user's emotional state. For example, it can send a specific message to the user such as, "There's a relaxing yoga class available. Would you like to join?"

[0840] These suggestions and notifications are delivered to the user's device via smartphone or computer. The user can review the notification and either manually follow the suggestion or let the system handle the settings in automatic control mode.

[0841] As a concrete example, if a user is feeling irritated at home, the server can detect this and provide information about nearby art therapy events, along with suggestions to gently adjust the lighting. This functionality is implemented using Python and emotion recognition modules such as EmotionEngine.

[0842] An example of a prompt might be, "In a smart city, consider ways to optimize energy consumption and encourage event participation based on user emotions."

[0843] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0844] Step 1:

[0845] The server collects energy consumption data in real time from smart meters and sensors installed in homes. This input data includes power consumption, usage time, and consumption patterns for each device. The data is initially processed and stored in a database, which provides the foundational information for subsequent analysis.

[0846] Step 2:

[0847] The server applies algorithms to the collected energy consumption data to analyze energy usage patterns. Time-series analysis is performed as a data processing step to extract consumption trends for each time period. The output includes usage characteristics by day of the week and peak consumption patterns. Based on these results, areas for improvement in energy management are identified.

[0848] Step 3:

[0849] The server uses a camera and microphone to detect the user's emotional state in real time. It uses facial expression data from the camera and tone data from the audio as input. An emotion recognition algorithm determines the user's emotional state, such as whether they are stressed or relaxed, and records the result as output.

[0850] Step 4:

[0851] The server integrates analysis results of energy consumption patterns with the user's emotional state to generate specific suggestions for optimizing energy efficiency. Using analysis results and emotional data as input, it creates adjusted energy efficiency suggestions as output. For example, if the user is stressed, it might suggest lighting settings that promote relaxation.

[0852] Step 5:

[0853] The server collects local event information and generates customized event participation suggestions based on the user's emotional state. Here, local event information is taken as input, events are filtered considering the emotional state and interests, and output as recommended events for the user.

[0854] Step 6:

[0855] The terminal notifies the user of energy optimization suggestions and event participation suggestions sent from the server. The notifications include specific action suggestions and invitations to participate in events. The user reviews the notifications and either manually implements the suggestions or selects automatic control mode to let the terminal handle it.

[0856] Step 7:

[0857] Users make decisions regarding energy settings and event participation based on suggestions through their device. The device feeds the user's decisions back to the server, which then uses this feedback to make future suggestions more personalized. This establishes a continuous optimization cycle.

[0858] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0859] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0860] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0861] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0862] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0863] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0864] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0865] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0866] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0867] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0868] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0869] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0870] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0871] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0872] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0873] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0874] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0875] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0876] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0877] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0878] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0879] The following is further disclosed regarding the embodiments described above.

[0880] (Claim 1)

[0881] Means for collecting household energy consumption data,

[0882] A means for analyzing usage patterns based on the aforementioned data,

[0883] A means for generating proposals to optimize energy efficiency based on the aforementioned analysis results,

[0884] A means for notifying the user terminal of the aforementioned proposal,

[0885] A system that includes this.

[0886] (Claim 2)

[0887] Means of collecting local event information,

[0888] Means for customizing the aforementioned information based on user interests,

[0889] Means for notifying the user of the customized information,

[0890] The system according to claim 1, comprising:

[0891] (Claim 3)

[0892] Means for automatically controlling the aforementioned devices within the home,

[0893] A means of collecting feedback on participation in the aforementioned local event information,

[0894] The system according to claim 1, comprising:

[0895] "Example 1"

[0896] (Claim 1)

[0897] Means for collecting household energy consumption data,

[0898] A means for analyzing usage patterns based on the aforementioned data,

[0899] A means for generating proposals to optimize energy efficiency based on the aforementioned analysis results,

[0900] Means for notifying the aforementioned proposal to the information processing device,

[0901] Means of collecting local activity information,

[0902] Means for customizing the aforementioned information based on the user's interests,

[0903] A means for presenting the customized information to the user,

[0904] A system that includes this.

[0905] (Claim 2)

[0906] Means for remotely controlling the aforementioned household equipment,

[0907] A means for collecting records of participation in the aforementioned community activity information,

[0908] The system according to claim 1, comprising:

[0909] (Claim 3)

[0910] Methods for using generative AI models to perform machine learning in data analysis,

[0911] In the aforementioned proposal generation, means for generating prompt sentences that constitute the input sentence,

[0912] The system according to claim 1, comprising:

[0913] "Application Example 1"

[0914] (Claim 1)

[0915] A device for collecting household energy consumption data,

[0916] A device for analyzing usage patterns based on the aforementioned data,

[0917] A device that generates proposals to optimize energy efficiency based on the aforementioned analysis results,

[0918] A device for notifying the user's information terminal of the aforementioned proposal,

[0919] A device for aggregating local activity information,

[0920] A device that pushes the activity information according to the user's interests,

[0921] A system that includes this.

[0922] (Claim 2)

[0923] The system according to claim 1, comprising deploying an application capable of monitoring the energy usage status within the household in real time.

[0924] (Claim 3)

[0925] The system according to claim 1, further comprising a function for aggregating participation feedback on the aforementioned activity information.

[0926] "Example 2 of combining an emotion engine"

[0927] (Claim 1)

[0928] Means for collecting information on resource consumption within households,

[0929] A means for analyzing usage patterns based on the aforementioned information,

[0930] A means for generating proposals to optimize resource efficiency based on the aforementioned analysis results,

[0931] A means for notifying the terminal of the aforementioned proposal,

[0932] A means of recognizing emotional states using an emotion engine,

[0933] A means of adjusting the proposal based on the recognized emotional state,

[0934] A system that includes this.

[0935] (Claim 2)

[0936] Means of collecting local activity information,

[0937] A means for personalizing the aforementioned activity information based on the user's interests,

[0938] A means for notifying the terminal of the aforementioned personalized activity information,

[0939] Methods for improving the quality of information and suggestions using generative AI models,

[0940] The system according to claim 1, comprising:

[0941] (Claim 3)

[0942] Means for automatically controlling the aforementioned household devices,

[0943] A means of collecting feedback on participation in the aforementioned community activity information,

[0944] A means of providing a more personalized experience by using the user's emotional state for analysis,

[0945] The system according to claim 1, comprising:

[0946] "Application example 2 when combining with an emotional engine"

[0947] (Claim 1)

[0948] Means for collecting household energy consumption data,

[0949] A means for analyzing usage patterns based on the aforementioned data,

[0950] A means for generating proposals to optimize energy efficiency based on the aforementioned analysis results,

[0951] A means for notifying the user terminal of the aforementioned proposal,

[0952] A means of detecting the user's emotional state,

[0953] Means for adjusting the optimization proposal based on the aforementioned emotional state,

[0954] A system that includes this.

[0955] (Claim 2)

[0956] Means of collecting local event information,

[0957] Means for customizing the aforementioned information based on the user's interests and emotional state,

[0958] Means for notifying the user of the customized information,

[0959] The system according to claim 1, comprising:

[0960] (Claim 3)

[0961] Means for automatically controlling the aforementioned household devices,

[0962] A means of collecting feedback on participation in the aforementioned local event information,

[0963] The system according to claim 1, comprising: [Explanation of symbols]

[0964] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for collecting household energy consumption data, A means for analyzing usage patterns based on the aforementioned data, A means for generating proposals to optimize energy efficiency based on the aforementioned analysis results, A means for notifying the user terminal of the aforementioned proposal, A system that includes this.

2. Means of collecting local event information, Means for customizing the aforementioned information based on user interests, Means for notifying the user of the customized information, The system according to claim 1, comprising:

3. Means for automatically controlling the aforementioned devices within the home, A means of collecting feedback on participation in the aforementioned local event information, The system according to claim 1, comprising:

Images