system

Abstract

We provide the system. [Solution] A data collection method for collecting behavioral data from users, Estimation means for estimating CO2 emissions based on the aforementioned behavioral data, An adjustment means that acquires real-time data from an external data source and takes into account fluctuating factors such as temperature and traffic conditions that affect the CO2 emissions calculated by the estimation means, A visualization means for recording and visualizing the CO2 emissions obtained by the estimation means and adjustment means, Furthermore, a means of proposing new actions, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There is a problem that it is difficult for individuals or groups to accurately and real-time grasp their own CO2 emissions in daily life. In this current situation, the effectiveness of reduction efforts cannot be evaluated, and it is difficult to lead to sustainable activities. Also, there is a problem that it is difficult to propose cooperation between regions or businesses or new reduction actions because there is a lack of a mechanism for visualizing the reduction results of CO2 emissions and sharing them across society.

Means for Solving the Problems

[0005] This invention provides a means for efficiently collecting user behavior data and instantly estimating CO2 emissions based on it. Furthermore, by utilizing real-time information from external data sources, it appropriately adjusts for fluctuations such as seasons, weather, and traffic conditions. This enables accurate and comprehensive calculation of CO2 emissions. In addition, visualization means are used to intuitively display the collected and estimated data, allowing users to easily understand their own contribution to the environment. Moreover, by providing functions for suggesting new actions, it builds a mechanism that promotes CO2 reduction and sustainable development for society as a whole.

[0006] A "data collection method" is a system for efficiently acquiring information about user behavior and recording it for analysis.

[0007] "Estimation means" refers to devices or algorithms used to calculate a user's CO2 emissions based on collected data.

[0008] The "adjustment mechanism" is a system that corrects the CO2 emission calculation results by taking into account real-time fluctuating factors obtained from external data.

[0009] "Visualization methods" refer to tools and interfaces for displaying data in a format that is intuitively easy for users to understand.

[0010] The "proposal method" is a system that suggests new CO2 reduction actions based on the user's past behavioral data.

[0011] "External data sources" refer to sources of information obtained from outside the system, such as weather and traffic information. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2]This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0013] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0018] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] The system of this invention aims to help individuals understand their impact on the environment and encourage new behaviors by using multiple modules to estimate CO2 emissions in real time based on user behavior and visualizing the results.

[0034] Method of data collection

[0035] Device: The user's smartphone or PC records location information and mode of transportation through a dedicated application. Using the GPS and accelerometer built into the device, the mode of transportation used, such as walking, cycling, bus, train, or car, is automatically identified. In addition, energy consumption data from smart meters and other sources is also collected.

[0036] Methods of estimating CO2 emissions

[0037] Server: The collected data is sent to a server in the cloud and processed immediately by a CO2 emissions calculation model. This calculates the CO2 emissions for each mode of transportation and energy source.

[0038] Real-time data adjustment methods

[0039] Server: Retrieves real-time data such as weather and traffic information via external APIs and incorporates it into CO2 emission calculations. The system improves prediction accuracy by considering fluctuations in energy use depending on the season and weather.

[0040] Data visualization methods

[0041] Terminal: Data calculated on the server is displayed on the terminal screen as graphs and charts. Users can check their daily CO2 emissions and reduction progress through an intuitive dashboard.

[0042] Embodiment of the new proposal

[0043] Server: Based on analysis of user behavior history, it generates action plans for further CO2 reduction. For example, it may suggest tasks to encourage the use of public transportation or to avoid energy use during specific time periods.

[0044] Specific example

[0045] User: When the user launches the dedicated app during their morning commute, they receive suggestions for the best mode of transportation from their current location. If they choose the train instead of their car, the app displays the amount of CO2 reduction resulting from that choice in real time. In addition, the app can also consider the best mode of transportation for their return home based on predicted traffic volume.

[0046] In this way, this system visualizes an individual's environmental contribution and functions as a tool to enable more sustainable actions.

[0047] The following describes the processing flow.

[0048] Step 1:

[0049] The device acquires data on the user's daily activities through the user's smartphone. Specifically, it uses the smartphone's GPS function and sensors to identify the user's current location and mode of transportation (walking, driving, cycling, public transport, etc.) and records the necessary data.

[0050] Step 2:

[0051] The device sends the acquired behavioral data to the server. The data is encrypted and uploaded to the cloud server in a secure state.

[0052] Step 3:

[0053] The server analyzes the received data and calculates CO2 emissions. Specifically, it applies CO2 emission factors corresponding to each mode of transportation to calculate the CO2 emissions associated with the user's travel. Energy consumption data is also processed during this process.

[0054] Step 4:

[0055] The server obtains real-time weather and traffic information via external APIs and incorporates it into the emissions calculation process. This allows for more accurate emissions figures that reflect conditions such as traffic congestion caused by bad weather.

[0056] Step 5:

[0057] The terminal receives CO2 emission data sent from the server and visualizes it as graphs and charts on the user interface. Based on this information, users can check their own environmental contribution.

[0058] Step 6:

[0059] The server analyzes the user's behavior patterns and generates new action plans that contribute to further reductions in CO2 emissions. These suggestions are presented to the user through their device, and the user considers specific reduction actions they can implement in their daily life.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] In modern times, there is a need to accurately understand the environmental impact of individual travel and energy consumption, and to use that understanding to improve daily life for sustainability. However, a challenge remains: there is no clear way for individual users to understand their own carbon dioxide emissions in real time and to receive concrete action suggestions based on that information.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes a device for collecting behavioral information from users, a function for calculating carbon dioxide emissions based on the behavioral information, and a function for considering fluctuations in environmental conditions and traffic information that affect the calculated carbon dioxide emissions based on real-time information obtained from external sources. This enables users to visually understand their own environmental impact, receive concrete action plans, and practice sustainable living.

[0065] A "user" is an entity that provides information and records its actions.

[0066] "Behavioral information" refers to data related to a user's movement and energy consumption.

[0067] "Carbon dioxide emissions" refers to the total amount of carbon dioxide emissions resulting from user activities.

[0068] "Device" refers to the collective term for hardware and software used to collect and process data.

[0069] "Function" refers to the operation of a program or system that performs a specific process or calculation.

[0070] "External information sources" refer to external data providers that the server connects with to obtain real-time information.

[0071] "Real-time information" refers to the latest data on current environmental conditions and traffic situations.

[0072] "Environmental conditions" refer to factors related to the natural environment, such as weather and temperature.

[0073] "Traffic information" refers to data on road congestion and the operating status of public transportation.

[0074] An "action plan" is a guideline that proposes specific actions that users should take to reduce their environmental impact.

[0075] The system of this invention helps users intuitively understand the environmental impact of their daily actions and further support them in choosing sustainable behaviors. This system acquires behavioral information through the user's smart device. Specifically, it uses a dedicated application installed on the user's smartphone or personal computer to record location information and means of travel using a built-in GPS module and accelerometer. In addition, by linking with a smart meter, it can collect energy consumption data within the home.

[0076] The collected data is sent from the terminal to a server in the cloud, where a CO2 emission calculation model is running. This model instantly calculates carbon dioxide emissions based on each mode of transportation and energy usage. Furthermore, the server acquires real-time information from external sources to take into account variable factors such as weather and traffic conditions, thereby improving the accuracy of emission calculations.

[0077] The data processed on the server is sent back to the user's terminal and visually displayed as graphs and charts on the dashboard of a dedicated application. This visualization allows users to check their current CO2 emissions in comparison to the past, enabling them to make choices that further reduce their environmental impact.

[0078] For example, if a user opens the app during their commute and enters a prompt such as, "Please tell me the best mode of transportation from my current location," the app will recommend using trains or buses. This allows the user to see in real time how much their choice contributes to reducing CO2 emissions. In this way, the system provides users with a high-resolution view of the environmental impact of their daily actions, encouraging them to make sustainable choices.

[0079] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0080] Step 1:

[0081] The device launches a dedicated application to collect user activity information. As input, the device obtains location information from GPS, acceleration data from an accelerometer, and energy usage information from a smart meter. Using this data, it determines the user's mode of transportation, specifically identifying walking, driving, cycling, and public transport. As output, it creates data summarizing the identified modes of transportation and corresponding energy consumption information, and sends it to a server in the cloud.

[0082] Step 2:

[0083] The server receives behavioral information transmitted from the terminal and runs a CO2 emission calculation model to calculate carbon dioxide emissions. As input, the server takes data on each mode of transportation and energy consumption, and processes the labeled behavioral information. For data processing, it uses CO2 emission factors for each mode of transportation to calculate total emissions. As output, it generates a dataset containing the calculated CO2 emissions.

[0084] Step 3:

[0085] The server acquires real-time data from external sources and corrects CO2 emission calculations. It imports weather information and traffic data in real time as input. The server integrates this data and performs correction calculations that take into account variable factors. Specifically, it reflects fluctuations in emissions due to increased energy consumption during bad weather and traffic congestion. The corrected emission data is output, serving as the basis for suggesting optimal actions to the user.

[0086] Step 4:

[0087] The terminal uses data received from the server to provide information to the user through a visual interface. It receives corrected CO2 emission data as input. The terminal visualizes this data in a dashboard format, showing the user daily emissions and reduction performance. As output, it provides information in interactive graphs and charts, creating visual indicators for users to review their actions.

[0088] Step 5:

[0089] The server analyzes the user's past behavioral history and generates new behavioral suggestions. As input, it evaluates the user's behavioral data history and current CO2 emission data. Using a generation AI model, it generates an action plan based on the behavioral history. Specifically, it suggests promoting the use of public transportation and energy-saving behaviors. As output, it generates a concrete action plan and sends it to the terminal, presenting the user with new behavioral options.

[0090] (Application Example 1)

[0091] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0092] In today's society, where environmental burdens continue to increase, individuals are required to accurately understand the environmental impact of their own actions and choose sustainable behaviors based on that understanding. However, conventional methods do not adequately provide real-time environmental impact estimation, suggestions for optimal modes of transportation, or visual feedback on actions based on those suggestions, making it difficult for individuals to practice reducing their environmental impact in their daily lives.

[0093] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0094] In this invention, the server includes data collection means for collecting configuration information from users, estimation means for estimating the amount of environmental load based on the configuration information, adjustment means for acquiring real-time information from an external source and considering variable items such as weather and traffic conditions that affect the amount of environmental load calculated by the estimation means, and traffic guidance means for proposing the optimal public transport route based on the selection of means of transport and visualizing the effect of reducing the environmental load. This makes it possible for individuals to understand their own environmental impact in real time and reduce their environmental load through optimal behavioral choices.

[0095] A "data collection method" is a system that obtains location information and configuration information related to means of transportation from users.

[0096] The "estimation method" is a method for calculating the amount of environmental impact resulting from user behavior based on acquired configuration information.

[0097] "Adjustment measures" refer to the process of receiving real-time information from external sources and reflecting it in the calculation results of environmental load, taking into account variable factors such as weather and traffic conditions.

[0098] "Visualization means" refers to a method of visually displaying the environmental load obtained through estimation and adjustment means and providing it in a format that is easy for users to understand.

[0099] A "suggestion mechanism" is a system that presents specific actions to reduce environmental impact based on the user's behavioral history.

[0100] "Transportation guidance methods" refer to methods that propose the optimal public transportation route based on the choice of mode of transport and visualize the environmental impact reduction effect resulting from that choice.

[0101] The system that implements this application uses the user's smartphone or PC to acquire location information and transportation data. The device has a built-in GPS and accelerometer, which are used to automatically identify the mode of transportation used, such as walking, cycling, public transport, or driving a vehicle. Furthermore, it can also acquire household energy consumption data.

[0102] The server uses estimation means to process information transmitted from data collection means on the cloud. This estimation means incorporates a statistical model that instantly calculates environmental load. The server also uses adjustment means to acquire information instantly from external sources. As a result, weather data and traffic conditions are reflected in the calculation of environmental load in real time, improving the accuracy of the calculation.

[0103] The device displays calculated environmental impact in the form of graphs and charts as a means of visualization. Through this intuitive dashboard, users can check how much environmental impact their activities are generating and how much reduction has been achieved. Furthermore, through the suggestion system, users can be offered specific environmental impact reduction actions based on their activity history.

[0104] As a concrete example, a user can use a smartphone application to compare the environmental impact of different modes of transportation in real time as they travel from their starting point to their destination. For instance, it can instantly calculate the CO2 reduction achieved by using a train and provide detailed information such as a 50% reduction compared to using a private car. An example of a prompt to input into the generating AI model would be, "Please suggest the most environmentally friendly public transport route for travel from my starting point to my destination."

[0105] In this way, through this system, users can make informed decisions to reduce their environmental impact and promote sustainable actions.

[0106] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0107] Step 1:

[0108] The device uses the user's smartphone or PC to acquire location information and mode of transportation based on GPS and accelerometer sensors. The input to this process is raw data from the sensors, and the output is identification information of the mode of transportation. Based on this data, the device performs an operation to determine the specific mode of transportation, such as walking or driving.

[0109] Step 2:

[0110] The server acquires the means of transportation and location information transmitted from the terminal and calculates the environmental impact using estimation tools. The input to this process is data from the terminal, and the output is the calculated environmental impact for each means of transportation. The server then uses a statistical model to calculate the actual CO2 emissions.

[0111] Step 3:

[0112] The server uses adjustment mechanisms to acquire weather information and traffic conditions in real time from external sources. The input to this process is immediate information from external sources, and the output is data on adjustment items that affect the environmental load. Based on the acquired data, a process is performed to dynamically correct the environmental load.

[0113] Step 4:

[0114] The server transmits the corrected environmental load data to the terminal via visualization means and displays it as graphs and charts on a dashboard accessible to the user. The input for this process is the corrected data from the server, and the output is a chart that can be visually understood. The client side performs the drawing process for visualization.

[0115] Step 5:

[0116] The user reflects on their actions based on visualized information and receives new action suggestions from the server through a suggestion mechanism. The input for this process is past action history and environmental load data, and the output is a specific action plan. The terminal queries the generation AI model using prompt messages and generates suggestions based on the conditions for the suggestions.

[0117] Step 6:

[0118] The user performs the suggested actions, and the effects are reflected in subsequent data collections. As an initial state of the implemented changes, the initial location and mode of transportation data are collected again, and a new cycle begins. This creates a process that continuously improves the system and promotes reduction of environmental impact.

[0119] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0120] The system of this invention collects and analyzes user behavioral and emotional data in real time and provides a function to support CO2 emission reduction tailored to each individual user. This makes it easier for users to intuitively and emotionally understand the impact their own actions have on the environment, and to proactively encourage new behaviors.

[0121] Data collection and emotion recognition methodology

[0122] Device: The user uses a smartphone to collect data expressing emotions (voice, facial expressions, text data, etc.) along with their daily activities. The emotion engine uses the camera and microphone built into the device to identify the user's emotional state.

[0123] Estimation of CO2 emissions and forms of consideration for emotions

[0124] Server: The collected behavioral data is analyzed on a cloud server, and CO2 emissions are calculated. The key here is not just calculating emissions, but generating more personalized feedback based on the user's emotional state.

[0125] Dynamic adjustment methods using an emotion engine

[0126] Server: The emotion engine combines collected emotion data with real-time data obtained from external sources to tailor how information is presented to the user. For example, it provides detailed data when the user is relaxed and concise advice when they are stressed.

[0127] Customizable forms of visualization and feedback

[0128] Terminal: Based on data transmitted from the server, it displays CO2 emission data in a format suitable for the user. Based on the analysis results of the emotion engine, it appropriately adjusts the content and timing of the feedback it presents.

[0129] Specific example

[0130] User: When I wake up in the morning and open the app, the emotion engine detects that I'm feeling down and presents a calming color scheme interface accordingly. If I take the bus to work instead of my usual car, the system notifies me of the CO2 reduction resulting from that choice. When I get home, it may suggest relaxing activities because my energy consumption might be higher.

[0131] By combining these emotion engines, we can create a system that supports flexible and sustainable CO2 reduction actions tailored to the user's state.

[0132] The following describes the processing flow.

[0133] Step 1:

[0134] The device uses the user's smartphone to simultaneously collect user behavioral data and emotional data. Specifically, GPS is used to obtain location information and means of transportation, while emotional data is inferred from facial expressions and voice using the camera and microphone.

[0135] Step 2:

[0136] The device securely transmits the acquired data to the server. The data is uploaded to the cloud in real time, ready for further analysis.

[0137] Step 3:

[0138] The server analyzes the received behavioral data and calculates CO2 emissions based on the mode of transport and energy consumption patterns. Furthermore, the emotion engine analyzes emotional data and evaluates the user's emotional state.

[0139] Step 4:

[0140] The server obtains real-time weather data and traffic information via external APIs. This data is used to correct for environmental changes in CO2 emission calculations.

[0141] Step 5:

[0142] The server uses the analysis results to generate feedback that suggests the most suitable CO2 emission reduction methods for the user. Taking into account the results of the emotion engine, it determines when and what kind of information the user should receive for the most effective results.

[0143] Step 6:

[0144] The device uses data received from the server to display CO2 emission data and action suggestions on the user interface. The order in which information is presented and the interface are adjusted according to the user's emotional state.

[0145] Step 7:

[0146] Users follow the instructions on their device and perform the suggested actions. For example, they may reduce CO2 emissions by trying out the public transportation suggested by the system or adopting an energy-saving lifestyle.

[0147] (Example 2)

[0148] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0149] Conventional environmental impact reduction systems can estimate CO2 emissions based on user behavior data, but they have the challenge of not being able to provide feedback or suggestions that take into account the user's emotional state, making it difficult to lead to behavioral change. Furthermore, they have difficulty flexibly incorporating the impact of external environmental information, resulting in a decrease in the accuracy of suggestions.

[0150] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0151] In this invention, the server includes means for collecting behavioral and emotional information from users, means for acquiring real-time information from external sources and considering environmental conditions, and means for analyzing emotional information and dynamically adjusting the information presentation method. This enables the provision of personalized feedback and behavioral suggestions that reflect the user's emotional state, realizing flexible and effective support for reducing CO2 emissions.

[0152] "Behavioral information" refers to data about a user's daily activities and habits, including information such as modes of transportation and power consumption patterns.

[0153] "Emotional information" refers to data about a user's emotional state, and is obtained from sources such as voice, facial expressions, and text messages.

[0154] "External information sources" are sources that provide real-time environmental data such as weather information and movement status.

[0155] "Real-time information" refers to dynamically changing data such as weather conditions and traffic conditions obtained from external sources.

[0156] "Means of analyzing emotional information" refers to the process of identifying a user's emotional state using voice analysis technology, facial recognition algorithms, and natural language processing.

[0157] "Means of dynamically adjusting the method of information presentation" refers to the process of changing the content and format of the information provided according to the user's emotional state.

[0158] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze data and personalize feedback and suggestions for users.

[0159] The system of this invention collects user behavioral and emotional information and provides support for reducing environmental impact. The configuration for implementing this system includes the following components:

[0160] 1. Data Collection

[0161] Device: Users collect information about their daily activities using smart devices. This includes data on transportation and power consumption. In addition, emotional information is collected through voice, facial expressions, and text messages. Smart devices have built-in cameras and microphones, which are used to acquire and transmit this data.

[0162] 2. Data Analysis and Feedback Generation

[0163] Server: The collected behavioral and emotional information is received by a server in the cloud. The server is equipped with a generative AI model, which is used to analyze the data. Based on the behavioral information, CO2 emissions are estimated, and emotional information is further analyzed to generate information appropriate to the user's emotional state. Real-time information obtained from external sources (weather information and traffic conditions) is also taken into consideration, which improves the accuracy of the feedback.

[0164] 3. Information Dissemination and Action Promotion

[0165] Terminal: Based on information transmitted from the server, it displays visual data on CO2 emissions and environmental impact to the user. By synchronizing with emotional information, it dynamically adjusts the content and timing of the presentation to suit the user's state. This promotes behavioral change.

[0166] For example, if a user uses public transportation for their daily commute, they will be notified of the CO2 reduction effect of that choice. Also, if a desire to relax is detected upon returning home, suggestions for activities to reduce energy consumption may be displayed.

[0167] For example, a prompt such as, "Please tell me how much CO2 emissions can be reduced by using public transport for my morning commute compared to using a private car," can elicit specific feedback.

[0168] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0169] Step 1:

[0170] The device collects information about the user's daily activities and emotions. Inputs include voice, facial expressions, and text messages captured by the camera and microphone. This data is processed and sent to a server as formalized digital data. Here, the collected data is organized to reflect the user's behavior and emotions.

[0171] Step 2:

[0172] The server receives behavioral information transmitted from the terminal as input and estimates CO2 emissions using a generative AI model. In this process, data on transportation methods and energy consumption are analyzed, and CO2 emissions are output as estimated values. At the same time, real-time information (weather information and traffic conditions) is considered and incorporated into the emissions estimation process to obtain more accurate output.

[0173] Step 3:

[0174] The server analyzes the input emotional information. A generative AI model analyzes voice, facial expressions, and text data to identify the user's emotional state. Based on this, it determines how to present information to provide optimal feedback and information. The specific output is adjusted according to the user's emotional state, ranging from detailed analysis results to concise advice.

[0175] Step 4:

[0176] The device presents the user with CO2 emissions and emotional feedback received from the server. It receives data transmitted from the server as input and displays it visually. The user interface displays information in an intuitive format using graphs and text, allowing users to receive feedback tailored to their own actions and emotions.

[0177] Step 5:

[0178] Users adjust their behavior based on the information presented and take new actions. Specifically, this includes users choosing public transportation when commuting or practicing energy-saving lifestyles. This unconsciously encourages actions that reduce environmental impact.

[0179] (Application Example 2)

[0180] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0181] The increase in carbon dioxide (CO2) emissions in modern society is a serious environmental problem. Nevertheless, it remains difficult for individual users to intuitively understand the impact their actions have on the environment and to obtain clear guidelines for sustainable behavior in their daily lives. Furthermore, conventional CO2 emission reduction systems have struggled to provide dynamic feedback that takes into account the user's emotional state. There is a need to improve this, provide optimized behavioral suggestions for each user, and raise each individual's awareness of contributing to the environment.

[0182] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0183] In this invention, the server includes means for collecting behavioral and emotional data, means for estimating CO2 emissions, and means for influencing CO2 emissions by considering environmental factors obtained from external data sources. This makes it possible to provide personalized feedback according to the user's emotional state, thereby promoting sustainable behavior.

[0184] A "user" is an individual who uses this system to receive support in reducing their own CO2 emissions.

[0185] "Behavioral data" refers to information about a user's daily activities, specifically data related to their mode of transportation and lifestyle.

[0186] "Emotional data" refers to information that represents a user's emotional state and is collected in the form of voice, facial expressions, text data, and other similar data.

[0187] "CO2 emissions" is a numerical representation of the amount of carbon dioxide emitted in connection with a specific activity or behavior.

[0188] A "correction mechanism" is a mechanism that has the function of correcting estimated CO2 emissions by taking into account environmental factors obtained from external data sources.

[0189] "Visualization means" refers to a device or software that displays CO2 emission information in a format easily understood by the user and provides feedback tailored to their emotional state.

[0190] A "proposal method" is a means that analyzes the user's behavioral history and emotional state and presents specific actions to reduce CO2 emissions.

[0191] "Real-time data" refers to data that is constantly updated, such as external environmental information and movement information, and is used to personalize user behavior and suggestions.

[0192] In the system for implementing this invention, the user's terminal primarily plays the role of data collection. The terminal is equipped with a camera and microphone, which are used to collect the user's daily behavior and emotions. Specifically, the user's emotional state is analyzed by an emotion engine through facial expressions and voice data. This emotional data, along with the user's daily behavior data, is sent to a cloud server.

[0193] The server uses a data processing engine to analyze the collected behavioral and emotional data. This engine calculates the user's CO2 emissions. This calculation takes into account real-time environmental factors obtained from external sources (e.g., weather data, traffic conditions, etc.). Adaptive algorithms are used to correct the data.

[0194] Furthermore, the visualization engine on the server generates personalized feedback based on calculated CO2 emissions and emotional data. This feedback is dynamically adjusted according to the user's emotional state and sent to the device. For example, detailed numerical data is presented when the user is relaxed, while simple advice is offered when they are stressed.

[0195] This system allows users to intuitively understand the impact their actions have on the environment and proactively encourages them to take everyday actions to reduce CO2 emissions. For example, when a user opens their smartphone each morning, the system incorporates a feature that analyzes their emotional state for the day and suggests actions to take.

[0196] As an example of how a generative AI model can be used, the prompt might look like this: "Imagine an application that allows users to collect emotional data in real time on their smartphones and receive specific action suggestions that would lead to a reduction in CO2 emissions across the entire city."

[0197] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0198] Step 1:

[0199] The device uses a camera and microphone to collect the user's facial expressions and voice. The input is real-time video and audio data, which the emotion engine analyzes to identify the user's emotional state. The output is digital data representing the user's current emotional state. Specifically, it extracts features from the video and audio and tags them to represent the emotional state.

[0200] Step 2:

[0201] The device acquires daily activity data through the app. The input is the user's chosen mode of transportation and consumption history, which is processed to create activity data. The output is detailed log data about the user's daily activities. Specifically, it constructs activity history from location information and app usage history.

[0202] Step 3:

[0203] The server receives behavioral and emotional data sent from terminals in the cloud and integrates them. The input is behavioral and emotional data, which are combined to enhance the user profile. The output is a composite dataset ready for analysis. The operation here involves storing the data in a database and evaluating the relationship between emotions and behaviors.

[0204] Step 4:

[0205] The server acquires real-time environmental data (e.g., weather data, traffic information) from external data sources. The input is a data stream from an external API, which is used as correction data for CO2 emission calculations. The output is the corrected environmental data. Specifically, data is acquired via a Web API and converted to a specified format.

[0206] Step 5:

[0207] The server estimates CO2 emissions using integrated behavioral and emotional data and external environmental data. The input is the previously integrated data, and quantitative emissions calculations are performed based on this data. The output is the user's estimated CO2 emissions. Specifically, a calculation algorithm is run to aggregate the contribution of each data point.

[0208] Step 6:

[0209] The server generates feedback for the user using estimated CO2 emissions and sentiment data. The inputs are CO2 emissions and sentiment data, and the feedback is customized based on these two factors. The output is personalized action suggestions and information. For specific actions, the data is visualized to make the suggestions easy for the user to understand, and the suggestions are constructed accordingly.

[0210] Step 7:

[0211] The terminal receives feedback from the server and presents it through the user interface. The input is customized feedback data from the server, which is presented to the user in a visualized format. The output is a screen display that allows the user to take action. Specifically, it provides an intuitive display of information on the terminal screen, offering an interaction that allows the user to easily select their next action.

[0212] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0213] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0214] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0215] [Second Embodiment]

[0216] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0217] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0218] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0219] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0220] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0221] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0222] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0223] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0224] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0225] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0226] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0227] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0228] The system of this invention aims to help individuals understand their impact on the environment and encourage new behaviors by using multiple modules to estimate CO2 emissions in real time based on user behavior and visualizing the results.

[0229] Method of data collection

[0230] Device: The user's smartphone or PC records location information and mode of transportation through a dedicated application. Using the GPS and accelerometer built into the device, the mode of transportation used, such as walking, cycling, bus, train, or car, is automatically identified. In addition, energy consumption data from smart meters and other sources is also collected.

[0231] Methods of estimating CO2 emissions

[0232] Server: The collected data is sent to a server in the cloud and processed immediately by a CO2 emissions calculation model. This calculates the CO2 emissions for each mode of transportation and energy source.

[0233] Real-time data adjustment methods

[0234] Server: Retrieves real-time data such as weather and traffic information via external APIs and incorporates it into CO2 emission calculations. The system improves prediction accuracy by considering fluctuations in energy use depending on the season and weather.

[0235] Data visualization methods

[0236] Terminal: Data calculated on the server is displayed on the terminal screen as graphs and charts. Users can check their daily CO2 emissions and reduction progress through an intuitive dashboard.

[0237] Embodiment of the new proposal

[0238] Server: Based on analysis of user behavior history, it generates action plans for further CO2 reduction. For example, it may suggest tasks to encourage the use of public transportation or to avoid energy use during specific time periods.

[0239] Specific example

[0240] User: When the user launches the dedicated app during their morning commute, they receive suggestions for the best mode of transportation from their current location. If they choose the train instead of their car, the app displays the amount of CO2 reduction resulting from that choice in real time. In addition, the app can also consider the best mode of transportation for their return home based on predicted traffic volume.

[0241] In this way, this system visualizes an individual's environmental contribution and functions as a tool to enable more sustainable actions.

[0242] The following describes the processing flow.

[0243] Step 1:

[0244] The device acquires data on the user's daily activities through the user's smartphone. Specifically, it uses the smartphone's GPS function and sensors to identify the user's current location and mode of transportation (walking, driving, cycling, public transport, etc.) and records the necessary data.

[0245] Step 2:

[0246] The device sends the acquired behavioral data to the server. The data is encrypted and uploaded to the cloud server in a secure state.

[0247] Step 3:

[0248] The server analyzes the received data and calculates CO2 emissions. Specifically, it applies CO2 emission factors corresponding to each mode of transportation to calculate the CO2 emissions associated with the user's travel. Energy consumption data is also processed during this process.

[0249] Step 4:

[0250] The server obtains real-time weather and traffic information via external APIs and incorporates it into the emissions calculation process. This allows for more accurate emissions figures that reflect conditions such as traffic congestion caused by bad weather.

[0251] Step 5:

[0252] The terminal receives CO2 emission data sent from the server and visualizes it as graphs and charts on the user interface. Based on this information, users can check their own environmental contribution.

[0253] Step 6:

[0254] The server analyzes the user's behavior patterns and generates new action plans that contribute to further reductions in CO2 emissions. These suggestions are presented to the user through their device, and the user considers specific reduction actions they can implement in their daily life.

[0255] (Example 1)

[0256] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0257] In modern times, there is a need to accurately understand the environmental impact of individual travel and energy consumption, and to use that understanding to improve daily life for sustainability. However, a challenge remains: there is no clear way for individual users to understand their own carbon dioxide emissions in real time and to receive concrete action suggestions based on that information.

[0258] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0259] In this invention, the server includes a device for collecting behavioral information from users, a function for calculating carbon dioxide emissions based on the behavioral information, and a function for considering fluctuations in environmental conditions and traffic information that affect the calculated carbon dioxide emissions based on real-time information obtained from external sources. This enables users to visually understand their own environmental impact, receive concrete action plans, and practice sustainable living.

[0260] A "user" is an entity that provides information and records its actions.

[0261] "Behavioral information" refers to data related to a user's movement and energy consumption.

[0262] "Carbon dioxide emissions" refers to the total amount of carbon dioxide emissions resulting from user activities.

[0263] "Device" refers to the collective term for hardware and software used to collect and process data.

[0264] "Function" refers to the operation of a program or system that performs a specific process or calculation.

[0265] "External information sources" refer to external data providers that the server connects with to obtain real-time information.

[0266] "Real-time information" refers to the latest data on current environmental conditions and traffic situations.

[0267] "Environmental conditions" refer to factors related to the natural environment, such as weather and temperature.

[0268] "Traffic information" refers to data on road congestion and the operating status of public transportation.

[0269] An "action plan" is a guideline that proposes specific actions that users should take to reduce their environmental impact.

[0270] The system of this invention helps users intuitively understand the environmental impact of their daily actions and further support them in choosing sustainable behaviors. This system acquires behavioral information through the user's smart device. Specifically, it uses a dedicated application installed on the user's smartphone or personal computer to record location information and means of travel using a built-in GPS module and accelerometer. In addition, by linking with a smart meter, it can collect energy consumption data within the home.

[0271] The collected data is sent from the terminal to a server in the cloud, where a CO2 emission calculation model is running. This model instantly calculates carbon dioxide emissions based on each mode of transportation and energy usage. Furthermore, the server acquires real-time information from external sources to take into account variable factors such as weather and traffic conditions, thereby improving the accuracy of emission calculations.

[0272] The data processed on the server is sent back to the user's terminal and visually displayed as graphs and charts on the dashboard of a dedicated application. This visualization allows users to check their current CO2 emissions in comparison to the past, enabling them to make choices that further reduce their environmental impact.

[0273] For example, if a user opens the app during their commute and enters a prompt such as, "Please tell me the best mode of transportation from my current location," the app will recommend using trains or buses. This allows the user to see in real time how much their choice contributes to reducing CO2 emissions. In this way, the system provides users with a high-resolution view of the environmental impact of their daily actions, encouraging them to make sustainable choices.

[0274] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0275] Step 1:

[0276] The device launches a dedicated application to collect user activity information. As input, the device obtains location information from GPS, acceleration data from an accelerometer, and energy usage information from a smart meter. Using this data, it determines the user's mode of transportation, specifically identifying walking, driving, cycling, and public transport. As output, it creates data summarizing the identified modes of transportation and corresponding energy consumption information, and sends it to a server in the cloud.

[0277] Step 2:

[0278] The server receives behavioral information transmitted from the terminal and runs a CO2 emission calculation model to calculate carbon dioxide emissions. As input, the server takes data on each mode of transportation and energy consumption, and processes the labeled behavioral information. For data processing, it uses CO2 emission factors for each mode of transportation to calculate total emissions. As output, it generates a dataset containing the calculated CO2 emissions.

[0279] Step 3:

[0280] The server acquires real-time data from external sources and corrects CO2 emission calculations. It imports weather information and traffic data in real time as input. The server integrates this data and performs correction calculations that take into account variable factors. Specifically, it reflects fluctuations in emissions due to increased energy consumption during bad weather and traffic congestion. The corrected emission data is output, serving as the basis for suggesting optimal actions to the user.

[0281] Step 4:

[0282] Using the data received by the terminal from the server, information is provided to the user through a visual interface. As input, it receives the corrected CO2 emission data. The terminal visualizes this data in a dashboard format and shows the user the daily emissions and reduction performance. As output, it provides information in an interactive graph or chart format and creates visual indicators for the user to review their actions.

[0283] Step 5:

[0284] The server analyzes the user's past behavior history and generates new action proposals. As input, it evaluates the user's behavior data history and the current CO2 emission status data. Using a generated AI model, it generates an action plan based on the behavior history. Specifically, it promotes the use of public transportation and proposes energy-saving actions. As output, it generates a specific action plan and sends it to the terminal to present a new action plan to the user.

[0285] (Application Example 1)

[0286] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0287] In modern society where environmental burdens continue to increase, it is required that individuals accurately grasp the environmental impact of their own actions and based on that, choose sustainable actions. However, with conventional methods, there has been a problem that real-time environmental burden estimation, proposal of optimal means of transportation, and visual feedback of actions based on that have not been sufficiently provided, making it difficult for individuals to practice environmental burden reduction in their daily lives.

[0288] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0289] In this invention, the server includes data collection means for collecting configuration information from users, estimation means for estimating the amount of environmental load based on the configuration information, adjustment means for acquiring real-time information from an external source and considering variable items such as weather and traffic conditions that affect the amount of environmental load calculated by the estimation means, and traffic guidance means for proposing the optimal public transport route based on the selection of means of transport and visualizing the effect of reducing the environmental load. This makes it possible for individuals to understand their own environmental impact in real time and reduce their environmental load through optimal behavioral choices.

[0290] A "data collection method" is a system that obtains location information and configuration information related to means of transportation from users.

[0291] The "estimation method" is a method for calculating the amount of environmental impact resulting from user behavior based on acquired configuration information.

[0292] "Adjustment measures" refer to the process of receiving real-time information from external sources and reflecting it in the calculation results of environmental load, taking into account variable factors such as weather and traffic conditions.

[0293] "Visualization means" refers to a method of visually displaying the environmental load obtained through estimation and adjustment means and providing it in a format that is easy for users to understand.

[0294] A "suggestion mechanism" is a system that presents specific actions to reduce environmental impact based on the user's behavioral history.

[0295] "Transportation guidance methods" refer to methods that propose the optimal public transportation route based on the choice of mode of transport and visualize the environmental impact reduction effect resulting from that choice.

[0296] The system that implements this application uses the user's smartphone or PC to acquire location information and transportation data. The device has a built-in GPS and accelerometer, which are used to automatically identify the mode of transportation used, such as walking, cycling, public transport, or driving a vehicle. Furthermore, it can also acquire household energy consumption data.

[0297] The server uses estimation means to process information transmitted from data collection means on the cloud. This estimation means incorporates a statistical model that instantly calculates environmental load. The server also uses adjustment means to acquire information instantly from external sources. As a result, weather data and traffic conditions are reflected in the calculation of environmental load in real time, improving the accuracy of the calculation.

[0298] The device displays calculated environmental impact in the form of graphs and charts as a means of visualization. Through this intuitive dashboard, users can check how much environmental impact their activities are generating and how much reduction has been achieved. Furthermore, through the suggestion system, users can be offered specific environmental impact reduction actions based on their activity history.

[0299] As a concrete example, a user can use a smartphone application to compare the environmental impact of different modes of transportation in real time as they travel from their starting point to their destination. For instance, it can instantly calculate the CO2 reduction achieved by using a train and provide detailed information such as a 50% reduction compared to using a private car. An example of a prompt to input into the generating AI model would be, "Please suggest the most environmentally friendly public transport route for travel from my starting point to my destination."

[0300] In this way, through this system, users can make informed decisions to reduce their environmental impact and promote sustainable actions.

[0301] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0302] Step 1:

[0303] The terminal uses the user's smartphone or PC to acquire location information and means of movement as data collection means based on the GPS and acceleration sensors. The input of this process is the raw data from the sensors, and the identification information of the means of movement is obtained as the output. Based on this data, an operation is performed to determine the specific means of movement such as walking or vehicle.

[0304] Step 2:

[0305] The server acquires the means of movement and location information transmitted from the terminal and calculates the environmental load amount using the estimation means. The input of this process is the data from the terminal, and the environmental load amount for each means of movement is calculated as the output. The server performs an operation to calculate the actual CO2 emission amount using a statistical model.

[0306] Step 3:

[0307] The server uses the adjustment means to acquire weather information and traffic conditions from external supply sources in real time. The input of this process is the immediate information from external supply sources, and the data of adjustment items that affect the environmental load amount is obtained as the output. Based on the acquired data, a process of dynamically correcting the environmental load amount is implemented.

[0308] Step 4:

[0309] The server transmits the corrected environmental load amount to the terminal by the visualization means and displays it on the dashboard accessible to the user in the form of graphs and charts. The input of this process is the corrected data from the server, and a chart in a visually understandable form is generated as the output. Drawing processing for visualization is performed on the client side.

[0310] Step 5:

[0311] The user reflects on their actions based on visualized information and receives new action suggestions from the server through a suggestion mechanism. The input for this process is past action history and environmental load data, and the output is a specific action plan. The terminal queries the generation AI model using prompt messages and generates suggestions based on the conditions for the suggestions.

[0312] Step 6:

[0313] The user performs the suggested actions, and the effects are reflected in subsequent data collections. As an initial state of the implemented changes, the initial location and mode of transportation data are collected again, and a new cycle begins. This creates a process that continuously improves the system and promotes reduction of environmental impact.

[0314] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0315] The system of this invention collects and analyzes user behavioral and emotional data in real time and provides a function to support CO2 emission reduction tailored to each individual user. This makes it easier for users to intuitively and emotionally understand the impact their own actions have on the environment, and to proactively encourage new behaviors.

[0316] Data collection and emotion recognition methodology

[0317] Device: The user uses a smartphone to collect data expressing emotions (voice, facial expressions, text data, etc.) along with their daily activities. The emotion engine uses the camera and microphone built into the device to identify the user's emotional state.

[0318] Estimation of CO2 emissions and forms of consideration for emotions

[0319] Server: The collected behavioral data is analyzed on a cloud server, and CO2 emissions are calculated. The key here is not just calculating emissions, but generating more personalized feedback based on the user's emotional state.

[0320] Dynamic adjustment methods using an emotion engine

[0321] Server: The emotion engine combines collected emotion data with real-time data obtained from external sources to tailor how information is presented to the user. For example, it provides detailed data when the user is relaxed and concise advice when they are stressed.

[0322] Customizable forms of visualization and feedback

[0323] Terminal: Based on data transmitted from the server, it displays CO2 emission data in a format suitable for the user. Based on the analysis results of the emotion engine, it appropriately adjusts the content and timing of the feedback it presents.

[0324] Specific example

[0325] User: When I wake up in the morning and open the app, the emotion engine detects that I'm feeling down and presents a calming color scheme interface accordingly. If I take the bus to work instead of my usual car, the system notifies me of the CO2 reduction resulting from that choice. When I get home, it may suggest relaxing activities because my energy consumption might be higher.

[0326] By combining these emotion engines, we can create a system that supports flexible and sustainable CO2 reduction actions tailored to the user's state.

[0327] The following describes the processing flow.

[0328] Step 1:

[0329] The device uses the user's smartphone to simultaneously collect user behavioral data and emotional data. Specifically, GPS is used to obtain location information and means of transportation, while emotional data is inferred from facial expressions and voice using the camera and microphone.

[0330] Step 2:

[0331] The device securely transmits the acquired data to the server. The data is uploaded to the cloud in real time, ready for further analysis.

[0332] Step 3:

[0333] The server analyzes the received behavioral data and calculates CO2 emissions based on the mode of transport and energy consumption patterns. Furthermore, the emotion engine analyzes emotional data and evaluates the user's emotional state.

[0334] Step 4:

[0335] The server obtains real-time weather data and traffic information via external APIs. This data is used to correct for environmental changes in CO2 emission calculations.

[0336] Step 5:

[0337] The server uses the analysis results to generate feedback that suggests the most suitable CO2 emission reduction methods for the user. Taking into account the results of the emotion engine, it determines when and what kind of information the user should receive for the most effective results.

[0338] Step 6:

[0339] The device uses data received from the server to display CO2 emission data and action suggestions on the user interface. The order in which information is presented and the interface are adjusted according to the user's emotional state.

[0340] Step 7:

[0341] Users follow the instructions on their device and perform the suggested actions. For example, they may reduce CO2 emissions by trying out the public transportation suggested by the system or adopting an energy-saving lifestyle.

[0342] (Example 2)

[0343] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0344] Conventional environmental impact reduction systems can estimate CO2 emissions based on user behavior data, but they have the challenge of not being able to provide feedback or suggestions that take into account the user's emotional state, making it difficult to lead to behavioral change. Furthermore, they have difficulty flexibly incorporating the impact of external environmental information, resulting in a decrease in the accuracy of suggestions.

[0345] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0346] In this invention, the server includes means for collecting behavioral and emotional information from users, means for acquiring real-time information from external sources and considering environmental conditions, and means for analyzing emotional information and dynamically adjusting the information presentation method. This enables the provision of personalized feedback and behavioral suggestions that reflect the user's emotional state, realizing flexible and effective support for reducing CO2 emissions.

[0347] "Behavioral information" refers to data about a user's daily activities and habits, including information such as modes of transportation and power consumption patterns.

[0348] "Emotional information" refers to data about a user's emotional state, and is obtained from sources such as voice, facial expressions, and text messages.

[0349] "External information sources" are sources that provide real-time environmental data such as weather information and movement status.

[0350] "Real-time information" refers to dynamically changing data such as weather conditions and traffic conditions obtained from external sources.

[0351] "Means of analyzing emotional information" refers to the process of identifying a user's emotional state using voice analysis technology, facial recognition algorithms, and natural language processing.

[0352] "Means of dynamically adjusting the method of information presentation" refers to the process of changing the content and format of the information provided according to the user's emotional state.

[0353] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze data and personalize feedback and suggestions for users.

[0354] The system of this invention collects user behavioral and emotional information and provides support for reducing environmental impact. The configuration for implementing this system includes the following components:

[0355] 1. Data Collection

[0356] Device: Users collect information about their daily activities using smart devices. This includes data on transportation and power consumption. In addition, emotional information is collected through voice, facial expressions, and text messages. Smart devices have built-in cameras and microphones, which are used to acquire and transmit this data.

[0357] 2. Data Analysis and Feedback Generation

[0358] Server: The collected behavioral and emotional information is received by a server in the cloud. The server is equipped with a generative AI model, which is used to analyze the data. Based on the behavioral information, CO2 emissions are estimated, and emotional information is further analyzed to generate information appropriate to the user's emotional state. Real-time information obtained from external sources (weather information and traffic conditions) is also taken into consideration, which improves the accuracy of the feedback.

[0359] 3. Information Dissemination and Action Promotion

[0360] Terminal: Based on information transmitted from the server, it displays visual data on CO2 emissions and environmental impact to the user. By synchronizing with emotional information, it dynamically adjusts the content and timing of the presentation to suit the user's state. This promotes behavioral change.

[0361] For example, if a user uses public transportation for their daily commute, they will be notified of the CO2 reduction effect of that choice. Also, if a desire to relax is detected upon returning home, suggestions for activities to reduce energy consumption may be displayed.

[0362] For example, a prompt such as, "Please tell me how much CO2 emissions can be reduced by using public transport for my morning commute compared to using a private car," can elicit specific feedback.

[0363] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0364] Step 1:

[0365] The device collects information about the user's daily activities and emotions. Inputs include voice, facial expressions, and text messages captured by the camera and microphone. This data is processed and sent to a server as formalized digital data. Here, the collected data is organized to reflect the user's behavior and emotions.

[0366] Step 2:

[0367] The server receives behavioral information transmitted from the terminal as input and estimates CO2 emissions using a generative AI model. In this process, data on transportation methods and energy consumption are analyzed, and CO2 emissions are output as estimated values. At the same time, real-time information (weather information and traffic conditions) is considered and incorporated into the emissions estimation process to obtain more accurate output.

[0368] Step 3:

[0369] The server analyzes the input emotional information. A generative AI model analyzes voice, facial expressions, and text data to identify the user's emotional state. Based on this, it determines how to present information to provide optimal feedback and information. The specific output is adjusted according to the user's emotional state, ranging from detailed analysis results to concise advice.

[0370] Step 4:

[0371] The device presents the user with CO2 emissions and emotional feedback received from the server. It receives data transmitted from the server as input and displays it visually. The user interface displays information in an intuitive format using graphs and text, allowing users to receive feedback tailored to their own actions and emotions.

[0372] Step 5:

[0373] Users adjust their behavior based on the information presented and take new actions. Specifically, this includes users choosing public transportation when commuting or practicing energy-saving lifestyles. This unconsciously encourages actions that reduce environmental impact.

[0374] (Application Example 2)

[0375] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0376] The increase in carbon dioxide (CO2) emissions in modern society is a serious environmental problem. Nevertheless, it remains difficult for individual users to intuitively understand the impact their actions have on the environment and to obtain clear guidelines for sustainable behavior in their daily lives. Furthermore, conventional CO2 emission reduction systems have struggled to provide dynamic feedback that takes into account the user's emotional state. There is a need to improve this, provide optimized behavioral suggestions for each user, and raise each individual's awareness of contributing to the environment.

[0377] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0378] In this invention, the server includes means for collecting behavioral and emotional data, means for estimating CO2 emissions, and means for influencing CO2 emissions by considering environmental factors obtained from external data sources. This makes it possible to provide personalized feedback according to the user's emotional state, thereby promoting sustainable behavior.

[0379] A "user" is an individual who uses this system to receive support in reducing their own CO2 emissions.

[0380] "Behavioral data" refers to information about a user's daily activities, specifically data related to their mode of transportation and lifestyle.

[0381] "Emotional data" refers to information that represents a user's emotional state and is collected in the form of voice, facial expressions, text data, and other similar data.

[0382] "CO2 emissions" is a numerical representation of the amount of carbon dioxide emitted in connection with a specific activity or behavior.

[0383] A "correction mechanism" is a mechanism that has the function of correcting estimated CO2 emissions by taking into account environmental factors obtained from external data sources.

[0384] "Visualization means" refers to a device or software that displays CO2 emission information in a format easily understood by the user and provides feedback tailored to their emotional state.

[0385] A "proposal method" is a means that analyzes the user's behavioral history and emotional state and presents specific actions to reduce CO2 emissions.

[0386] "Real-time data" refers to data that is constantly updated, such as external environmental information and movement information, and is used to personalize user behavior and suggestions.

[0387] In the system for implementing this invention, the user's terminal primarily plays the role of data collection. The terminal is equipped with a camera and microphone, which are used to collect the user's daily behavior and emotions. Specifically, the user's emotional state is analyzed by an emotion engine through facial expressions and voice data. This emotional data, along with the user's daily behavior data, is sent to a cloud server.

[0388] The server uses a data processing engine to analyze the collected behavioral and emotional data. This engine calculates the user's CO2 emissions. This calculation takes into account real-time environmental factors obtained from external sources (e.g., weather data, traffic conditions, etc.). Adaptive algorithms are used to correct the data.

[0389] Furthermore, the visualization engine on the server generates personalized feedback based on calculated CO2 emissions and emotional data. This feedback is dynamically adjusted according to the user's emotional state and sent to the device. For example, detailed numerical data is presented when the user is relaxed, while simple advice is offered when they are stressed.

[0390] This system allows users to intuitively understand the impact their actions have on the environment and proactively encourages them to take everyday actions to reduce CO2 emissions. For example, when a user opens their smartphone each morning, the system incorporates a feature that analyzes their emotional state for the day and suggests actions to take.

[0391] As an example of how a generative AI model can be used, the prompt might look like this: "Imagine an application that allows users to collect emotional data in real time on their smartphones and receive specific action suggestions that would lead to a reduction in CO2 emissions across the entire city."

[0392] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0393] Step 1:

[0394] The device uses a camera and microphone to collect the user's facial expressions and voice. The input is real-time video and audio data, which the emotion engine analyzes to identify the user's emotional state. The output is digital data representing the user's current emotional state. Specifically, it extracts features from the video and audio and tags them to represent the emotional state.

[0395] Step 2:

[0396] The device acquires daily activity data through the app. The input is the user's chosen mode of transportation and consumption history, which is processed to create activity data. The output is detailed log data about the user's daily activities. Specifically, it constructs activity history from location information and app usage history.

[0397] Step 3:

[0398] The server receives behavioral and emotional data sent from terminals in the cloud and integrates them. The input is behavioral and emotional data, which are combined to enhance the user profile. The output is a composite dataset ready for analysis. The operation here involves storing the data in a database and evaluating the relationship between emotions and behaviors.

[0399] Step 4:

[0400] The server acquires real-time environmental data (e.g., weather data, traffic information) from external data sources. The input is a data stream from an external API, which is used as correction data for CO2 emission calculations. The output is the corrected environmental data. Specifically, data is acquired via a Web API and converted to a specified format.

[0401] Step 5:

[0402] The server estimates CO2 emissions using integrated behavioral and emotional data and external environmental data. The input is the previously integrated data, and quantitative emissions calculations are performed based on this data. The output is the user's estimated CO2 emissions. Specifically, a calculation algorithm is run to aggregate the contribution of each data point.

[0403] Step 6:

[0404] The server generates feedback for the user using estimated CO2 emissions and sentiment data. The inputs are CO2 emissions and sentiment data, and the feedback is customized based on these two factors. The output is personalized action suggestions and information. For specific actions, the data is visualized to make the suggestions easy for the user to understand, and the suggestions are constructed accordingly.

[0405] Step 7:

[0406] The terminal receives feedback from the server and presents it through the user interface. The input is customized feedback data from the server, which is presented to the user in a visualized format. The output is a screen display that allows the user to take action. Specifically, it provides an intuitive display of information on the terminal screen, offering an interaction that allows the user to easily select their next action.

[0407] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0408] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0409] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0410] [Third Embodiment]

[0411] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0412] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0413] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0414] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0415] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0416] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0417] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0418] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0419] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0420] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0421] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0422] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0423] The system of this invention aims to help individuals understand their impact on the environment and encourage new behaviors by using multiple modules to estimate CO2 emissions in real time based on user behavior and visualizing the results.

[0424] Method of data collection

[0425] Device: The user's smartphone or PC records location information and mode of transportation through a dedicated application. Using the GPS and accelerometer built into the device, the mode of transportation used, such as walking, cycling, bus, train, or car, is automatically identified. In addition, energy consumption data from smart meters and other sources is also collected.

[0426] Methods of estimating CO2 emissions

[0427] Server: The collected data is sent to a server in the cloud and processed immediately by a CO2 emissions calculation model. This calculates the CO2 emissions for each mode of transportation and energy source.

[0428] Real-time data adjustment methods

[0429] Server: Retrieves real-time data such as weather and traffic information via external APIs and incorporates it into CO2 emission calculations. The system improves prediction accuracy by considering fluctuations in energy use depending on the season and weather.

[0430] Data visualization methods

[0431] Terminal: Data calculated on the server is displayed on the terminal screen as graphs and charts. Users can check their daily CO2 emissions and reduction progress through an intuitive dashboard.

[0432] Embodiment of the new proposal

[0433] Server: Based on analysis of user behavior history, it generates action plans for further CO2 reduction. For example, it may suggest tasks to encourage the use of public transportation or to avoid energy use during specific time periods.

[0434] Specific example

[0435] User: When the user launches the dedicated app during their morning commute, they receive suggestions for the best mode of transportation from their current location. If they choose the train instead of their car, the app displays the amount of CO2 reduction resulting from that choice in real time. In addition, the app can also consider the best mode of transportation for their return home based on predicted traffic volume.

[0436] In this way, this system visualizes an individual's environmental contribution and functions as a tool to enable more sustainable actions.

[0437] The following describes the processing flow.

[0438] Step 1:

[0439] The device acquires data on the user's daily activities through the user's smartphone. Specifically, it uses the smartphone's GPS function and sensors to identify the user's current location and mode of transportation (walking, driving, cycling, public transport, etc.) and records the necessary data.

[0440] Step 2:

[0441] The device sends the acquired behavioral data to the server. The data is encrypted and uploaded to the cloud server in a secure state.

[0442] Step 3:

[0443] The server analyzes the received data and calculates CO2 emissions. Specifically, it applies CO2 emission factors corresponding to each mode of transportation to calculate the CO2 emissions associated with the user's travel. Energy consumption data is also processed during this process.

[0444] Step 4:

[0445] The server obtains real-time weather and traffic information via external APIs and incorporates it into the emissions calculation process. This allows for more accurate emissions figures that reflect conditions such as traffic congestion caused by bad weather.

[0446] Step 5:

[0447] The terminal receives CO2 emission data sent from the server and visualizes it as graphs and charts on the user interface. Based on this information, users can check their own environmental contribution.

[0448] Step 6:

[0449] The server analyzes the user's behavior patterns and generates new action plans that contribute to further reductions in CO2 emissions. These suggestions are presented to the user through their device, and the user considers specific reduction actions they can implement in their daily life.

[0450] (Example 1)

[0451] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0452] In modern times, there is a need to accurately understand the environmental impact of individual travel and energy consumption, and to use that understanding to improve daily life for sustainability. However, a challenge remains: there is no clear way for individual users to understand their own carbon dioxide emissions in real time and to receive concrete action suggestions based on that information.

[0453] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0454] In this invention, the server includes a device for collecting behavioral information from users, a function for calculating carbon dioxide emissions based on the behavioral information, and a function for considering fluctuations in environmental conditions and traffic information that affect the calculated carbon dioxide emissions based on real-time information obtained from external sources. This enables users to visually understand their own environmental impact, receive concrete action plans, and practice sustainable living.

[0455] A "user" is an entity that provides information and records its actions.

[0456] "Behavioral information" refers to data related to a user's movement and energy consumption.

[0457] "Carbon dioxide emissions" refers to the total amount of carbon dioxide emissions resulting from user activities.

[0458] "Device" refers to the collective term for hardware and software used to collect and process data.

[0459] "Function" refers to the operation of a program or system that performs a specific process or calculation.

[0460] "External information sources" refer to external data providers that the server connects with to obtain real-time information.

[0461] "Real-time information" refers to the latest data on current environmental conditions and traffic situations.

[0462] "Environmental conditions" refer to factors related to the natural environment, such as weather and temperature.

[0463] "Traffic information" refers to data on road congestion and the operating status of public transportation.

[0464] An "action plan" is a guideline that proposes specific actions that users should take to reduce their environmental impact.

[0465] The system of this invention helps users intuitively understand the environmental impact of their daily actions and further support them in choosing sustainable behaviors. This system acquires behavioral information through the user's smart device. Specifically, it uses a dedicated application installed on the user's smartphone or personal computer to record location information and means of travel using a built-in GPS module and accelerometer. In addition, by linking with a smart meter, it can collect energy consumption data within the home.

[0466] The collected data is sent from the terminal to a server in the cloud, where a CO2 emission calculation model is running. This model instantly calculates carbon dioxide emissions based on each mode of transportation and energy usage. Furthermore, the server acquires real-time information from external sources to take into account variable factors such as weather and traffic conditions, thereby improving the accuracy of emission calculations.

[0467] The data processed on the server is sent back to the user's terminal and visually displayed as graphs and charts on the dashboard of a dedicated application. This visualization allows users to check their current CO2 emissions in comparison to the past, enabling them to make choices that further reduce their environmental impact.

[0468] For example, if a user opens the app during their commute and enters a prompt such as, "Please tell me the best mode of transportation from my current location," the app will recommend using trains or buses. This allows the user to see in real time how much their choice contributes to reducing CO2 emissions. In this way, the system provides users with a high-resolution view of the environmental impact of their daily actions, encouraging them to make sustainable choices.

[0469] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0470] Step 1:

[0471] The device launches a dedicated application to collect user activity information. As input, the device obtains location information from GPS, acceleration data from an accelerometer, and energy usage information from a smart meter. Using this data, it determines the user's mode of transportation, specifically identifying walking, driving, cycling, and public transport. As output, it creates data summarizing the identified modes of transportation and corresponding energy consumption information, and sends it to a server in the cloud.

[0472] Step 2:

[0473] The server receives behavioral information transmitted from the terminal and runs a CO2 emission calculation model to calculate carbon dioxide emissions. As input, the server takes data on each mode of transportation and energy consumption, and processes the labeled behavioral information. For data processing, it uses CO2 emission factors for each mode of transportation to calculate total emissions. As output, it generates a dataset containing the calculated CO2 emissions.

[0474] Step 3:

[0475] The server acquires real-time data from external sources and corrects CO2 emission calculations. It imports weather information and traffic data in real time as input. The server integrates this data and performs correction calculations that take into account variable factors. Specifically, it reflects fluctuations in emissions due to increased energy consumption during bad weather and traffic congestion. The corrected emission data is output, serving as the basis for suggesting optimal actions to the user.

[0476] Step 4:

[0477] The terminal uses data received from the server to provide information to the user through a visual interface. It receives corrected CO2 emission data as input. The terminal visualizes this data in a dashboard format, showing the user daily emissions and reduction performance. As output, it provides information in interactive graphs and charts, creating visual indicators for users to review their actions.

[0478] Step 5:

[0479] The server analyzes the user's past behavioral history and generates new behavioral suggestions. As input, it evaluates the user's behavioral data history and current CO2 emission data. Using a generation AI model, it generates an action plan based on the behavioral history. Specifically, it suggests promoting the use of public transportation and energy-saving behaviors. As output, it generates a concrete action plan and sends it to the terminal, presenting the user with new behavioral options.

[0480] (Application Example 1)

[0481] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0482] In today's society, where environmental burdens continue to increase, individuals are required to accurately understand the environmental impact of their own actions and choose sustainable behaviors based on that understanding. However, conventional methods do not adequately provide real-time environmental impact estimation, suggestions for optimal modes of transportation, or visual feedback on actions based on those suggestions, making it difficult for individuals to practice reducing their environmental impact in their daily lives.

[0483] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0484] In this invention, the server includes data collection means for collecting configuration information from users, estimation means for estimating the amount of environmental load based on the configuration information, adjustment means for acquiring real-time information from an external source and considering variable items such as weather and traffic conditions that affect the amount of environmental load calculated by the estimation means, and traffic guidance means for proposing the optimal public transport route based on the selection of means of transport and visualizing the effect of reducing the environmental load. This makes it possible for individuals to understand their own environmental impact in real time and reduce their environmental load through optimal behavioral choices.

[0485] A "data collection method" is a system that obtains location information and configuration information related to means of transportation from users.

[0486] The "estimation method" is a method for calculating the amount of environmental impact resulting from user behavior based on acquired configuration information.

[0487] "Adjustment measures" refer to the process of receiving real-time information from external sources and reflecting it in the calculation results of environmental load, taking into account variable factors such as weather and traffic conditions.

[0488] "Visualization means" refers to a method of visually displaying the environmental load obtained through estimation and adjustment means and providing it in a format that is easy for users to understand.

[0489] A "suggestion mechanism" is a system that presents specific actions to reduce environmental impact based on the user's behavioral history.

[0490] "Transportation guidance methods" refer to methods that propose the optimal public transportation route based on the choice of mode of transport and visualize the environmental impact reduction effect resulting from that choice.

[0491] The system that implements this application uses the user's smartphone or PC to acquire location information and transportation data. The device has a built-in GPS and accelerometer, which are used to automatically identify the mode of transportation used, such as walking, cycling, public transport, or driving a vehicle. Furthermore, it can also acquire household energy consumption data.

[0492] The server uses estimation means to process information transmitted from data collection means on the cloud. This estimation means incorporates a statistical model that instantly calculates environmental load. The server also uses adjustment means to acquire information instantly from external sources. As a result, weather data and traffic conditions are reflected in the calculation of environmental load in real time, improving the accuracy of the calculation.

[0493] The device displays calculated environmental impact in the form of graphs and charts as a means of visualization. Through this intuitive dashboard, users can check how much environmental impact their activities are generating and how much reduction has been achieved. Furthermore, through the suggestion system, users can be offered specific environmental impact reduction actions based on their activity history.

[0494] As a concrete example, a user can use a smartphone application to compare the environmental impact of different modes of transportation in real time as they travel from their starting point to their destination. For instance, it can instantly calculate the CO2 reduction achieved by using a train and provide detailed information such as a 50% reduction compared to using a private car. An example of a prompt to input into the generating AI model would be, "Please suggest the most environmentally friendly public transport route for travel from my starting point to my destination."

[0495] In this way, through this system, users can make informed decisions to reduce their environmental impact and promote sustainable actions.

[0496] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0497] Step 1:

[0498] The device uses the user's smartphone or PC to acquire location information and mode of transportation based on GPS and accelerometer sensors. The input to this process is raw data from the sensors, and the output is identification information of the mode of transportation. Based on this data, the device performs an operation to determine the specific mode of transportation, such as walking or driving.

[0499] Step 2:

[0500] The server acquires the means of transportation and location information transmitted from the terminal and calculates the environmental impact using estimation tools. The input to this process is data from the terminal, and the output is the calculated environmental impact for each means of transportation. The server then uses a statistical model to calculate the actual CO2 emissions.

[0501] Step 3:

[0502] The server uses adjustment mechanisms to acquire weather information and traffic conditions in real time from external sources. The input to this process is immediate information from external sources, and the output is data on adjustment items that affect the environmental load. Based on the acquired data, a process is performed to dynamically correct the environmental load.

[0503] Step 4:

[0504] The server transmits the corrected environmental load data to the terminal via visualization means and displays it as graphs and charts on a dashboard accessible to the user. The input for this process is the corrected data from the server, and the output is a chart that can be visually understood. The client side performs the drawing process for visualization.

[0505] Step 5:

[0506] The user reflects on their actions based on visualized information and receives new action suggestions from the server through a suggestion mechanism. The input for this process is past action history and environmental load data, and the output is a specific action plan. The terminal queries the generation AI model using prompt messages and generates suggestions based on the conditions for the suggestions.

[0507] Step 6:

[0508] The user performs the suggested actions, and the effects are reflected in subsequent data collections. As an initial state of the implemented changes, the initial location and mode of transportation data are collected again, and a new cycle begins. This creates a process that continuously improves the system and promotes reduction of environmental impact.

[0509] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0510] The system of this invention collects and analyzes user behavioral and emotional data in real time and provides a function to support CO2 emission reduction tailored to each individual user. This makes it easier for users to intuitively and emotionally understand the impact their own actions have on the environment, and to proactively encourage new behaviors.

[0511] Data collection and emotion recognition methodology

[0512] Device: The user uses a smartphone to collect data expressing emotions (voice, facial expressions, text data, etc.) along with their daily activities. The emotion engine uses the camera and microphone built into the device to identify the user's emotional state.

[0513] Estimation of CO2 emissions and forms of consideration for emotions

[0514] Server: The collected behavioral data is analyzed on a cloud server, and CO2 emissions are calculated. The key here is not just calculating emissions, but generating more personalized feedback based on the user's emotional state.

[0515] Dynamic adjustment methods using an emotion engine

[0516] Server: The emotion engine combines collected emotion data with real-time data obtained from external sources to tailor how information is presented to the user. For example, it provides detailed data when the user is relaxed and concise advice when they are stressed.

[0517] Customizable forms of visualization and feedback

[0518] Terminal: Based on data transmitted from the server, it displays CO2 emission data in a format suitable for the user. Based on the analysis results of the emotion engine, it appropriately adjusts the content and timing of the feedback it presents.

[0519] Specific example

[0520] User: When I wake up in the morning and open the app, the emotion engine detects that I'm feeling down and presents a calming color scheme interface accordingly. If I take the bus to work instead of my usual car, the system notifies me of the CO2 reduction resulting from that choice. When I get home, it may suggest relaxing activities because my energy consumption might be higher.

[0521] By combining these emotion engines, we can create a system that supports flexible and sustainable CO2 reduction actions tailored to the user's state.

[0522] The following describes the processing flow.

[0523] Step 1:

[0524] The device uses the user's smartphone to simultaneously collect user behavioral data and emotional data. Specifically, GPS is used to obtain location information and means of transportation, while emotional data is inferred from facial expressions and voice using the camera and microphone.

[0525] Step 2:

[0526] The device securely transmits the acquired data to the server. The data is uploaded to the cloud in real time, ready for further analysis.

[0527] Step 3:

[0528] The server analyzes the received behavioral data and calculates CO2 emissions based on the mode of transport and energy consumption patterns. Furthermore, the emotion engine analyzes emotional data and evaluates the user's emotional state.

[0529] Step 4:

[0530] The server obtains real-time weather data and traffic information via external APIs. This data is used to correct for environmental changes in CO2 emission calculations.

[0531] Step 5:

[0532] The server uses the analysis results to generate feedback that suggests the most suitable CO2 emission reduction methods for the user. Taking into account the results of the emotion engine, it determines when and what kind of information the user should receive for the most effective results.

[0533] Step 6:

[0534] The device uses data received from the server to display CO2 emission data and action suggestions on the user interface. The order in which information is presented and the interface are adjusted according to the user's emotional state.

[0535] Step 7:

[0536] Users follow the instructions on their device and perform the suggested actions. For example, they may reduce CO2 emissions by trying out the public transportation suggested by the system or adopting an energy-saving lifestyle.

[0537] (Example 2)

[0538] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0539] Conventional environmental impact reduction systems can estimate CO2 emissions based on user behavior data, but they have the challenge of not being able to provide feedback or suggestions that take into account the user's emotional state, making it difficult to lead to behavioral change. Furthermore, they have difficulty flexibly incorporating the impact of external environmental information, resulting in a decrease in the accuracy of suggestions.

[0540] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0541] In this invention, the server includes means for collecting behavioral and emotional information from users, means for acquiring real-time information from external sources and considering environmental conditions, and means for analyzing emotional information and dynamically adjusting the information presentation method. This enables the provision of personalized feedback and behavioral suggestions that reflect the user's emotional state, realizing flexible and effective support for reducing CO2 emissions.

[0542] "Behavioral information" refers to data about a user's daily activities and habits, including information such as modes of transportation and power consumption patterns.

[0543] "Emotional information" refers to data about a user's emotional state, and is obtained from sources such as voice, facial expressions, and text messages.

[0544] "External information sources" are sources that provide real-time environmental data such as weather information and movement status.

[0545] "Real-time information" refers to dynamically changing data such as weather conditions and traffic conditions obtained from external sources.

[0546] "Means of analyzing emotional information" refers to the process of identifying a user's emotional state using voice analysis technology, facial recognition algorithms, and natural language processing.

[0547] "Means of dynamically adjusting the method of information presentation" refers to the process of changing the content and format of the information provided according to the user's emotional state.

[0548] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze data and personalize feedback and suggestions for users.

[0549] The system of this invention collects user behavioral and emotional information and provides support for reducing environmental impact. The configuration for implementing this system includes the following components:

[0550] 1. Data Collection

[0551] Device: Users collect information about their daily activities using smart devices. This includes data on transportation and power consumption. In addition, emotional information is collected through voice, facial expressions, and text messages. Smart devices have built-in cameras and microphones, which are used to acquire and transmit this data.

[0552] 2. Data Analysis and Feedback Generation

[0553] Server: The collected behavioral and emotional information is received by a server in the cloud. The server is equipped with a generative AI model, which is used to analyze the data. Based on the behavioral information, CO2 emissions are estimated, and emotional information is further analyzed to generate information appropriate to the user's emotional state. Real-time information obtained from external sources (weather information and traffic conditions) is also taken into consideration, which improves the accuracy of the feedback.

[0554] 3. Information Dissemination and Action Promotion

[0555] Terminal: Based on information transmitted from the server, it displays visual data on CO2 emissions and environmental impact to the user. By synchronizing with emotional information, it dynamically adjusts the content and timing of the presentation to suit the user's state. This promotes behavioral change.

[0556] For example, if a user uses public transportation for their daily commute, they will be notified of the CO2 reduction effect of that choice. Also, if a desire to relax is detected upon returning home, suggestions for activities to reduce energy consumption may be displayed.

[0557] For example, a prompt such as, "Please tell me how much CO2 emissions can be reduced by using public transport for my morning commute compared to using a private car," can elicit specific feedback.

[0558] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0559] Step 1:

[0560] The device collects information about the user's daily activities and emotions. Inputs include voice, facial expressions, and text messages captured by the camera and microphone. This data is processed and sent to a server as formalized digital data. Here, the collected data is organized to reflect the user's behavior and emotions.

[0561] Step 2:

[0562] The server receives behavioral information transmitted from the terminal as input and estimates CO2 emissions using a generative AI model. In this process, data on transportation methods and energy consumption are analyzed, and CO2 emissions are output as estimated values. At the same time, real-time information (weather information and traffic conditions) is considered and incorporated into the emissions estimation process to obtain more accurate output.

[0563] Step 3:

[0564] The server analyzes the input emotional information. A generative AI model analyzes voice, facial expressions, and text data to identify the user's emotional state. Based on this, it determines how to present information to provide optimal feedback and information. The specific output is adjusted according to the user's emotional state, ranging from detailed analysis results to concise advice.

[0565] Step 4:

[0566] The device presents the user with CO2 emissions and emotional feedback received from the server. It receives data transmitted from the server as input and displays it visually. The user interface displays information in an intuitive format using graphs and text, allowing users to receive feedback tailored to their own actions and emotions.

[0567] Step 5:

[0568] Users adjust their behavior based on the information presented and take new actions. Specifically, this includes users choosing public transportation when commuting or practicing energy-saving lifestyles. This unconsciously encourages actions that reduce environmental impact.

[0569] (Application Example 2)

[0570] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0571] The increase in carbon dioxide (CO2) emissions in modern society is a serious environmental problem. Nevertheless, it remains difficult for individual users to intuitively understand the impact their actions have on the environment and to obtain clear guidelines for sustainable behavior in their daily lives. Furthermore, conventional CO2 emission reduction systems have struggled to provide dynamic feedback that takes into account the user's emotional state. There is a need to improve this, provide optimized behavioral suggestions for each user, and raise each individual's awareness of contributing to the environment.

[0572] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0573] In this invention, the server includes means for collecting behavioral and emotional data, means for estimating CO2 emissions, and means for influencing CO2 emissions by considering environmental factors obtained from external data sources. This makes it possible to provide personalized feedback according to the user's emotional state, thereby promoting sustainable behavior.

[0574] A "user" is an individual who uses this system to receive support in reducing their own CO2 emissions.

[0575] "Behavioral data" refers to information about a user's daily activities, specifically data related to their mode of transportation and lifestyle.

[0576] "Emotional data" refers to information that represents a user's emotional state and is collected in the form of voice, facial expressions, text data, and other similar data.

[0577] "CO2 emissions" is a numerical representation of the amount of carbon dioxide emitted in connection with a specific activity or behavior.

[0578] A "correction mechanism" is a mechanism that has the function of correcting estimated CO2 emissions by taking into account environmental factors obtained from external data sources.

[0579] "Visualization means" refers to a device or software that displays CO2 emission information in a format easily understood by the user and provides feedback tailored to their emotional state.

[0580] A "proposal method" is a means that analyzes the user's behavioral history and emotional state and presents specific actions to reduce CO2 emissions.

[0581] "Real-time data" refers to data that is constantly updated, such as external environmental information and movement information, and is used to personalize user behavior and suggestions.

[0582] In the system for implementing this invention, the user's terminal primarily plays the role of data collection. The terminal is equipped with a camera and microphone, which are used to collect the user's daily behavior and emotions. Specifically, the user's emotional state is analyzed by an emotion engine through facial expressions and voice data. This emotional data, along with the user's daily behavior data, is sent to a cloud server.

[0583] The server uses a data processing engine to analyze the collected behavioral and emotional data. This engine calculates the user's CO2 emissions. This calculation takes into account real-time environmental factors obtained from external sources (e.g., weather data, traffic conditions, etc.). Adaptive algorithms are used to correct the data.

[0584] Furthermore, the visualization engine on the server generates personalized feedback based on calculated CO2 emissions and emotional data. This feedback is dynamically adjusted according to the user's emotional state and sent to the device. For example, detailed numerical data is presented when the user is relaxed, while simple advice is offered when they are stressed.

[0585] This system allows users to intuitively understand the impact their actions have on the environment and proactively encourages them to take everyday actions to reduce CO2 emissions. For example, when a user opens their smartphone each morning, the system incorporates a feature that analyzes their emotional state for the day and suggests actions to take.

[0586] As an example of how a generative AI model can be used, the prompt might look like this: "Imagine an application that allows users to collect emotional data in real time on their smartphones and receive specific action suggestions that would lead to a reduction in CO2 emissions across the entire city."

[0587] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0588] Step 1:

[0589] The device uses a camera and microphone to collect the user's facial expressions and voice. The input is real-time video and audio data, which the emotion engine analyzes to identify the user's emotional state. The output is digital data representing the user's current emotional state. Specifically, it extracts features from the video and audio and tags them to represent the emotional state.

[0590] Step 2:

[0591] The device acquires daily activity data through the app. The input is the user's chosen mode of transportation and consumption history, which is processed to create activity data. The output is detailed log data about the user's daily activities. Specifically, it constructs activity history from location information and app usage history.

[0592] Step 3:

[0593] The server receives behavioral and emotional data sent from terminals in the cloud and integrates them. The input is behavioral and emotional data, which are combined to enhance the user profile. The output is a composite dataset ready for analysis. The operation here involves storing the data in a database and evaluating the relationship between emotions and behaviors.

[0594] Step 4:

[0595] The server acquires real-time environmental data (e.g., weather data, traffic information) from external data sources. The input is a data stream from an external API, which is used as correction data for CO2 emission calculations. The output is the corrected environmental data. Specifically, data is acquired via a Web API and converted to a specified format.

[0596] Step 5:

[0597] The server estimates CO2 emissions using integrated behavioral and emotional data and external environmental data. The input is the previously integrated data, and quantitative emissions calculations are performed based on this data. The output is the user's estimated CO2 emissions. Specifically, a calculation algorithm is run to aggregate the contribution of each data point.

[0598] Step 6:

[0599] The server generates feedback for the user using estimated CO2 emissions and sentiment data. The inputs are CO2 emissions and sentiment data, and the feedback is customized based on these two factors. The output is personalized action suggestions and information. For specific actions, the data is visualized to make the suggestions easy for the user to understand, and the suggestions are constructed accordingly.

[0600] Step 7:

[0601] The terminal receives feedback from the server and presents it through the user interface. The input is customized feedback data from the server, which is presented to the user in a visualized format. The output is a screen display that allows the user to take action. Specifically, it provides an intuitive display of information on the terminal screen, offering an interaction that allows the user to easily select their next action.

[0602] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0603] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0604] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0605] [Fourth Embodiment]

[0606] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0607] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0608] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0609] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0610] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0611] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0612] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0613] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0614] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0615] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0616] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0617] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0618] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0619] The system of this invention aims to help individuals understand their impact on the environment and encourage new behaviors by using multiple modules to estimate CO2 emissions in real time based on user behavior and visualizing the results.

[0620] Method of data collection

[0621] Device: The user's smartphone or PC records location information and mode of transportation through a dedicated application. Using the GPS and accelerometer built into the device, the mode of transportation used, such as walking, cycling, bus, train, or car, is automatically identified. In addition, energy consumption data from smart meters and other sources is also collected.

[0622] Methods of estimating CO2 emissions

[0623] Server: The collected data is sent to a server in the cloud and processed immediately by a CO2 emissions calculation model. This calculates the CO2 emissions for each mode of transportation and energy source.

[0624] Real-time data adjustment methods

[0625] Server: Retrieves real-time data such as weather and traffic information via external APIs and incorporates it into CO2 emission calculations. The system improves prediction accuracy by considering fluctuations in energy use depending on the season and weather.

[0626] Data visualization methods

[0627] Terminal: Data calculated on the server is displayed on the terminal screen as graphs and charts. Users can check their daily CO2 emissions and reduction progress through an intuitive dashboard.

[0628] Embodiment of the new proposal

[0629] Server: Based on analysis of user behavior history, it generates action plans for further CO2 reduction. For example, it may suggest tasks to encourage the use of public transportation or to avoid energy use during specific time periods.

[0630] Specific example

[0631] User: When the user launches the dedicated app during their morning commute, they receive suggestions for the best mode of transportation from their current location. If they choose the train instead of their car, the app displays the amount of CO2 reduction resulting from that choice in real time. In addition, the app can also consider the best mode of transportation for their return home based on predicted traffic volume.

[0632] In this way, this system visualizes an individual's environmental contribution and functions as a tool to enable more sustainable actions.

[0633] The following describes the processing flow.

[0634] Step 1:

[0635] The device acquires data on the user's daily activities through the user's smartphone. Specifically, it uses the smartphone's GPS function and sensors to identify the user's current location and mode of transportation (walking, driving, cycling, public transport, etc.) and records the necessary data.

[0636] Step 2:

[0637] The device sends the acquired behavioral data to the server. The data is encrypted and uploaded to the cloud server in a secure state.

[0638] Step 3:

[0639] The server analyzes the received data and calculates CO2 emissions. Specifically, it applies CO2 emission factors corresponding to each mode of transportation to calculate the CO2 emissions associated with the user's travel. Energy consumption data is also processed during this process.

[0640] Step 4:

[0641] The server obtains real-time weather and traffic information via external APIs and incorporates it into the emissions calculation process. This allows for more accurate emissions figures that reflect conditions such as traffic congestion caused by bad weather.

[0642] Step 5:

[0643] The terminal receives CO2 emission data sent from the server and visualizes it as graphs and charts on the user interface. Based on this information, users can check their own environmental contribution.

[0644] Step 6:

[0645] The server analyzes the user's behavior patterns and generates new action plans that contribute to further reductions in CO2 emissions. These suggestions are presented to the user through their device, and the user considers specific reduction actions they can implement in their daily life.

[0646] (Example 1)

[0647] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0648] In modern times, there is a need to accurately understand the environmental impact of individual travel and energy consumption, and to use that understanding to improve daily life for sustainability. However, a challenge remains: there is no clear way for individual users to understand their own carbon dioxide emissions in real time and to receive concrete action suggestions based on that information.

[0649] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0650] In this invention, the server includes a device for collecting behavioral information from users, a function for calculating carbon dioxide emissions based on the behavioral information, and a function for considering fluctuations in environmental conditions and traffic information that affect the calculated carbon dioxide emissions based on real-time information obtained from external sources. This enables users to visually understand their own environmental impact, receive concrete action plans, and practice sustainable living.

[0651] A "user" is an entity that provides information and records its actions.

[0652] "Behavioral information" refers to data related to a user's movement and energy consumption.

[0653] "Carbon dioxide emissions" refers to the total amount of carbon dioxide emissions resulting from user activities.

[0654] "Device" refers to the collective term for hardware and software used to collect and process data.

[0655] "Function" refers to the operation of a program or system that performs a specific process or calculation.

[0656] "External information sources" refer to external data providers that the server connects with to obtain real-time information.

[0657] "Real-time information" refers to the latest data on current environmental conditions and traffic situations.

[0658] "Environmental conditions" refer to factors related to the natural environment, such as weather and temperature.

[0659] "Traffic information" refers to data on road congestion and the operating status of public transportation.

[0660] An "action plan" is a guideline that proposes specific actions that users should take to reduce their environmental impact.

[0661] The system of this invention helps users intuitively understand the environmental impact of their daily actions and further support them in choosing sustainable behaviors. This system acquires behavioral information through the user's smart device. Specifically, it uses a dedicated application installed on the user's smartphone or personal computer to record location information and means of travel using a built-in GPS module and accelerometer. In addition, by linking with a smart meter, it can collect energy consumption data within the home.

[0662] The collected data is sent from the terminal to a server in the cloud, where a CO2 emission calculation model is running. This model instantly calculates carbon dioxide emissions based on each mode of transportation and energy usage. Furthermore, the server acquires real-time information from external sources to take into account variable factors such as weather and traffic conditions, thereby improving the accuracy of emission calculations.

[0663] The data processed on the server is sent back to the user's terminal and visually displayed as graphs and charts on the dashboard of a dedicated application. This visualization allows users to check their current CO2 emissions in comparison to the past, enabling them to make choices that further reduce their environmental impact.

[0664] For example, if a user opens the app during their commute and enters a prompt such as, "Please tell me the best mode of transportation from my current location," the app will recommend using trains or buses. This allows the user to see in real time how much their choice contributes to reducing CO2 emissions. In this way, the system provides users with a high-resolution view of the environmental impact of their daily actions, encouraging them to make sustainable choices.

[0665] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0666] Step 1:

[0667] The device launches a dedicated application to collect user activity information. As input, the device obtains location information from GPS, acceleration data from an accelerometer, and energy usage information from a smart meter. Using this data, it determines the user's mode of transportation, specifically identifying walking, driving, cycling, and public transport. As output, it creates data summarizing the identified modes of transportation and corresponding energy consumption information, and sends it to a server in the cloud.

[0668] Step 2:

[0669] The server receives behavioral information transmitted from the terminal and runs a CO2 emission calculation model to calculate carbon dioxide emissions. As input, the server takes data on each mode of transportation and energy consumption, and processes the labeled behavioral information. For data processing, it uses CO2 emission factors for each mode of transportation to calculate total emissions. As output, it generates a dataset containing the calculated CO2 emissions.

[0670] Step 3:

[0671] The server acquires real-time data from external sources and corrects CO2 emission calculations. It imports weather information and traffic data in real time as input. The server integrates this data and performs correction calculations that take into account variable factors. Specifically, it reflects fluctuations in emissions due to increased energy consumption during bad weather and traffic congestion. The corrected emission data is output, serving as the basis for suggesting optimal actions to the user.

[0672] Step 4:

[0673] The terminal uses data received from the server to provide information to the user through a visual interface. It receives corrected CO2 emission data as input. The terminal visualizes this data in a dashboard format, showing the user daily emissions and reduction performance. As output, it provides information in interactive graphs and charts, creating visual indicators for users to review their actions.

[0674] Step 5:

[0675] The server analyzes the user's past behavioral history and generates new behavioral suggestions. As input, it evaluates the user's behavioral data history and current CO2 emission data. Using a generation AI model, it generates an action plan based on the behavioral history. Specifically, it suggests promoting the use of public transportation and energy-saving behaviors. As output, it generates a concrete action plan and sends it to the terminal, presenting the user with new behavioral options.

[0676] (Application Example 1)

[0677] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0678] In today's society, where environmental burdens continue to increase, individuals are required to accurately understand the environmental impact of their own actions and choose sustainable behaviors based on that understanding. However, conventional methods do not adequately provide real-time environmental impact estimation, suggestions for optimal modes of transportation, or visual feedback on actions based on those suggestions, making it difficult for individuals to practice reducing their environmental impact in their daily lives.

[0679] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0680] In this invention, the server includes data collection means for collecting configuration information from users, estimation means for estimating the amount of environmental load based on the configuration information, adjustment means for acquiring real-time information from an external source and considering variable items such as weather and traffic conditions that affect the amount of environmental load calculated by the estimation means, and traffic guidance means for proposing the optimal public transport route based on the selection of means of transport and visualizing the effect of reducing the environmental load. This makes it possible for individuals to understand their own environmental impact in real time and reduce their environmental load through optimal behavioral choices.

[0681] A "data collection method" is a system that obtains location information and configuration information related to means of transportation from users.

[0682] The "estimation method" is a method for calculating the amount of environmental impact resulting from user behavior based on acquired configuration information.

[0683] "Adjustment measures" refer to the process of receiving real-time information from external sources and reflecting it in the calculation results of environmental load, taking into account variable factors such as weather and traffic conditions.

[0684] "Visualization means" refers to a method of visually displaying the environmental load obtained through estimation and adjustment means and providing it in a format that is easy for users to understand.

[0685] A "suggestion mechanism" is a system that presents specific actions to reduce environmental impact based on the user's behavioral history.

[0686] "Transportation guidance methods" refer to methods that propose the optimal public transportation route based on the choice of mode of transport and visualize the environmental impact reduction effect resulting from that choice.

[0687] The system that implements this application uses the user's smartphone or PC to acquire location information and transportation data. The device has a built-in GPS and accelerometer, which are used to automatically identify the mode of transportation used, such as walking, cycling, public transport, or driving a vehicle. Furthermore, it can also acquire household energy consumption data.

[0688] The server uses estimation means to process information transmitted from data collection means on the cloud. This estimation means incorporates a statistical model that instantly calculates environmental load. The server also uses adjustment means to acquire information instantly from external sources. As a result, weather data and traffic conditions are reflected in the calculation of environmental load in real time, improving the accuracy of the calculation.

[0689] The device displays calculated environmental impact in the form of graphs and charts as a means of visualization. Through this intuitive dashboard, users can check how much environmental impact their activities are generating and how much reduction has been achieved. Furthermore, through the suggestion system, users can be offered specific environmental impact reduction actions based on their activity history.

[0690] As a concrete example, a user can use a smartphone application to compare the environmental impact of different modes of transportation in real time as they travel from their starting point to their destination. For instance, it can instantly calculate the CO2 reduction achieved by using a train and provide detailed information such as a 50% reduction compared to using a private car. An example of a prompt to input into the generating AI model would be, "Please suggest the most environmentally friendly public transport route for travel from my starting point to my destination."

[0691] In this way, through this system, users can make informed decisions to reduce their environmental impact and promote sustainable actions.

[0692] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0693] Step 1:

[0694] The device uses the user's smartphone or PC to acquire location information and mode of transportation based on GPS and accelerometer sensors. The input to this process is raw data from the sensors, and the output is identification information of the mode of transportation. Based on this data, the device performs an operation to determine the specific mode of transportation, such as walking or driving.

[0695] Step 2:

[0696] The server acquires the means of transportation and location information transmitted from the terminal and calculates the environmental impact using estimation tools. The input to this process is data from the terminal, and the output is the calculated environmental impact for each means of transportation. The server then uses a statistical model to calculate the actual CO2 emissions.

[0697] Step 3:

[0698] The server uses adjustment mechanisms to acquire weather information and traffic conditions in real time from external sources. The input to this process is immediate information from external sources, and the output is data on adjustment items that affect the environmental load. Based on the acquired data, a process is performed to dynamically correct the environmental load.

[0699] Step 4:

[0700] The server transmits the corrected environmental load data to the terminal via visualization means and displays it as graphs and charts on a dashboard accessible to the user. The input for this process is the corrected data from the server, and the output is a chart that can be visually understood. The client side performs the drawing process for visualization.

[0701] Step 5:

[0702] The user reflects on their actions based on visualized information and receives new action suggestions from the server through a suggestion mechanism. The input for this process is past action history and environmental load data, and the output is a specific action plan. The terminal queries the generation AI model using prompt messages and generates suggestions based on the conditions for the suggestions.

[0703] Step 6:

[0704] The user performs the suggested actions, and the effects are reflected in subsequent data collections. As an initial state of the implemented changes, the initial location and mode of transportation data are collected again, and a new cycle begins. This creates a process that continuously improves the system and promotes reduction of environmental impact.

[0705] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0706] The system of this invention collects and analyzes user behavioral and emotional data in real time and provides a function to support CO2 emission reduction tailored to each individual user. This makes it easier for users to intuitively and emotionally understand the impact their own actions have on the environment, and to proactively encourage new behaviors.

[0707] Data collection and emotion recognition methodology

[0708] Device: The user uses a smartphone to collect data expressing emotions (voice, facial expressions, text data, etc.) along with their daily activities. The emotion engine uses the camera and microphone built into the device to identify the user's emotional state.

[0709] Estimation of CO2 emissions and forms of consideration for emotions

[0710] Server: The collected behavioral data is analyzed on a cloud server, and CO2 emissions are calculated. The key here is not just calculating emissions, but generating more personalized feedback based on the user's emotional state.

[0711] Dynamic adjustment methods using an emotion engine

[0712] Server: The emotion engine combines collected emotion data with real-time data obtained from external sources to tailor how information is presented to the user. For example, it provides detailed data when the user is relaxed and concise advice when they are stressed.

[0713] Customizable forms of visualization and feedback

[0714] Terminal: Based on data transmitted from the server, it displays CO2 emission data in a format suitable for the user. Based on the analysis results of the emotion engine, it appropriately adjusts the content and timing of the feedback it presents.

[0715] Specific example

[0716] User: When I wake up in the morning and open the app, the emotion engine detects that I'm feeling down and presents a calming color scheme interface accordingly. If I take the bus to work instead of my usual car, the system notifies me of the CO2 reduction resulting from that choice. When I get home, it may suggest relaxing activities because my energy consumption might be higher.

[0717] By combining these emotion engines, we can create a system that supports flexible and sustainable CO2 reduction actions tailored to the user's state.

[0718] The following describes the processing flow.

[0719] Step 1:

[0720] The device uses the user's smartphone to simultaneously collect user behavioral data and emotional data. Specifically, GPS is used to obtain location information and means of transportation, while emotional data is inferred from facial expressions and voice using the camera and microphone.

[0721] Step 2:

[0722] The device securely transmits the acquired data to the server. The data is uploaded to the cloud in real time, ready for further analysis.

[0723] Step 3:

[0724] The server analyzes the received behavioral data and calculates CO2 emissions based on the mode of transport and energy consumption patterns. Furthermore, the emotion engine analyzes emotional data and evaluates the user's emotional state.

[0725] Step 4:

[0726] The server obtains real-time weather data and traffic information via external APIs. This data is used to correct for environmental changes in CO2 emission calculations.

[0727] Step 5:

[0728] The server uses the analysis results to generate feedback that suggests the most suitable CO2 emission reduction methods for the user. Taking into account the results of the emotion engine, it determines when and what kind of information the user should receive for the most effective results.

[0729] Step 6:

[0730] The device uses data received from the server to display CO2 emission data and action suggestions on the user interface. The order in which information is presented and the interface are adjusted according to the user's emotional state.

[0731] Step 7:

[0732] Users follow the instructions on their device and perform the suggested actions. For example, they may reduce CO2 emissions by trying out the public transportation suggested by the system or adopting an energy-saving lifestyle.

[0733] (Example 2)

[0734] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0735] Conventional environmental impact reduction systems can estimate CO2 emissions based on user behavior data, but they have the challenge of not being able to provide feedback or suggestions that take into account the user's emotional state, making it difficult to lead to behavioral change. Furthermore, they have difficulty flexibly incorporating the impact of external environmental information, resulting in a decrease in the accuracy of suggestions.

[0736] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0737] In this invention, the server includes means for collecting behavioral and emotional information from users, means for acquiring real-time information from external sources and considering environmental conditions, and means for analyzing emotional information and dynamically adjusting the information presentation method. This enables the provision of personalized feedback and behavioral suggestions that reflect the user's emotional state, realizing flexible and effective support for reducing CO2 emissions.

[0738] "Behavioral information" refers to data about a user's daily activities and habits, including information such as modes of transportation and power consumption patterns.

[0739] "Emotional information" refers to data about a user's emotional state, and is obtained from sources such as voice, facial expressions, and text messages.

[0740] "External information sources" are sources that provide real-time environmental data such as weather information and movement status.

[0741] "Real-time information" refers to dynamically changing data such as weather conditions and traffic conditions obtained from external sources.

[0742] "Means of analyzing emotional information" refers to the process of identifying a user's emotional state using voice analysis technology, facial recognition algorithms, and natural language processing.

[0743] "Means of dynamically adjusting the method of information presentation" refers to the process of changing the content and format of the information provided according to the user's emotional state.

[0744] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze data and personalize feedback and suggestions for users.

[0745] The system of this invention collects user behavioral and emotional information and provides support for reducing environmental impact. The configuration for implementing this system includes the following components:

[0746] 1. Data Collection

[0747] Device: Users collect information about their daily activities using smart devices. This includes data on transportation and power consumption. In addition, emotional information is collected through voice, facial expressions, and text messages. Smart devices have built-in cameras and microphones, which are used to acquire and transmit this data.

[0748] 2. Data Analysis and Feedback Generation

[0749] Server: The collected behavioral and emotional information is received by a server in the cloud. The server is equipped with a generative AI model, which is used to analyze the data. Based on the behavioral information, CO2 emissions are estimated, and emotional information is further analyzed to generate information appropriate to the user's emotional state. Real-time information obtained from external sources (weather information and traffic conditions) is also taken into consideration, which improves the accuracy of the feedback.

[0750] 3. Information Dissemination and Action Promotion

[0751] Terminal: Based on information transmitted from the server, it displays visual data on CO2 emissions and environmental impact to the user. By synchronizing with emotional information, it dynamically adjusts the content and timing of the presentation to suit the user's state. This promotes behavioral change.

[0752] For example, if a user uses public transportation for their daily commute, they will be notified of the CO2 reduction effect of that choice. Also, if a desire to relax is detected upon returning home, suggestions for activities to reduce energy consumption may be displayed.

[0753] For example, a prompt such as, "Please tell me how much CO2 emissions can be reduced by using public transport for my morning commute compared to using a private car," can elicit specific feedback.

[0754] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0755] Step 1:

[0756] The device collects information about the user's daily activities and emotions. Inputs include voice, facial expressions, and text messages captured by the camera and microphone. This data is processed and sent to a server as formalized digital data. Here, the collected data is organized to reflect the user's behavior and emotions.

[0757] Step 2:

[0758] The server receives behavioral information transmitted from the terminal as input and estimates CO2 emissions using a generative AI model. In this process, data on transportation methods and energy consumption are analyzed, and CO2 emissions are output as estimated values. At the same time, real-time information (weather information and traffic conditions) is considered and incorporated into the emissions estimation process to obtain more accurate output.

[0759] Step 3:

[0760] The server analyzes the input emotional information. A generative AI model analyzes voice, facial expressions, and text data to identify the user's emotional state. Based on this, it determines how to present information to provide optimal feedback and information. The specific output is adjusted according to the user's emotional state, ranging from detailed analysis results to concise advice.

[0761] Step 4:

[0762] The device presents the user with CO2 emissions and emotional feedback received from the server. It receives data transmitted from the server as input and displays it visually. The user interface displays information in an intuitive format using graphs and text, allowing users to receive feedback tailored to their own actions and emotions.

[0763] Step 5:

[0764] Users adjust their behavior based on the information presented and take new actions. Specifically, this includes users choosing public transportation when commuting or practicing energy-saving lifestyles. This unconsciously encourages actions that reduce environmental impact.

[0765] (Application Example 2)

[0766] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0767] The increase in carbon dioxide (CO2) emissions in modern society is a serious environmental problem. Nevertheless, it remains difficult for individual users to intuitively understand the impact their actions have on the environment and to obtain clear guidelines for sustainable behavior in their daily lives. Furthermore, conventional CO2 emission reduction systems have struggled to provide dynamic feedback that takes into account the user's emotional state. There is a need to improve this, provide optimized behavioral suggestions for each user, and raise each individual's awareness of contributing to the environment.

[0768] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0769] In this invention, the server includes means for collecting behavioral and emotional data, means for estimating CO2 emissions, and means for influencing CO2 emissions by considering environmental factors obtained from external data sources. This makes it possible to provide personalized feedback according to the user's emotional state, thereby promoting sustainable behavior.

[0770] A "user" is an individual who uses this system to receive support in reducing their own CO2 emissions.

[0771] "Behavioral data" refers to information about a user's daily activities, specifically data related to their mode of transportation and lifestyle.

[0772] "Emotional data" refers to information that represents a user's emotional state and is collected in the form of voice, facial expressions, text data, and other similar data.

[0773] "CO2 emissions" is a numerical representation of the amount of carbon dioxide emitted in connection with a specific activity or behavior.

[0774] A "correction mechanism" is a mechanism that has the function of correcting estimated CO2 emissions by taking into account environmental factors obtained from external data sources.

[0775] "Visualization means" refers to a device or software that displays CO2 emission information in a format easily understood by the user and provides feedback tailored to their emotional state.

[0776] A "proposal method" is a means that analyzes the user's behavioral history and emotional state and presents specific actions to reduce CO2 emissions.

[0777] "Real-time data" refers to data that is constantly updated, such as external environmental information and movement information, and is used to personalize user behavior and suggestions.

[0778] In the system for implementing this invention, the user's terminal primarily plays the role of data collection. The terminal is equipped with a camera and microphone, which are used to collect the user's daily behavior and emotions. Specifically, the user's emotional state is analyzed by an emotion engine through facial expressions and voice data. This emotional data, along with the user's daily behavior data, is sent to a cloud server.

[0779] The server uses a data processing engine to analyze the collected behavioral and emotional data. This engine calculates the user's CO2 emissions. This calculation takes into account real-time environmental factors obtained from external sources (e.g., weather data, traffic conditions, etc.). Adaptive algorithms are used to correct the data.

[0780] Furthermore, the visualization engine on the server generates personalized feedback based on calculated CO2 emissions and emotional data. This feedback is dynamically adjusted according to the user's emotional state and sent to the device. For example, detailed numerical data is presented when the user is relaxed, while simple advice is offered when they are stressed.

[0781] This system allows users to intuitively understand the impact their actions have on the environment and proactively encourages them to take everyday actions to reduce CO2 emissions. For example, when a user opens their smartphone each morning, the system incorporates a feature that analyzes their emotional state for the day and suggests actions to take.

[0782] As an example of how a generative AI model can be used, the prompt might look like this: "Imagine an application that allows users to collect emotional data in real time on their smartphones and receive specific action suggestions that would lead to a reduction in CO2 emissions across the entire city."

[0783] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0784] Step 1:

[0785] The device uses a camera and microphone to collect the user's facial expressions and voice. The input is real-time video and audio data, which the emotion engine analyzes to identify the user's emotional state. The output is digital data representing the user's current emotional state. Specifically, it extracts features from the video and audio and tags them to represent the emotional state.

[0786] Step 2:

[0787] The device acquires daily activity data through the app. The input is the user's chosen mode of transportation and consumption history, which is processed to create activity data. The output is detailed log data about the user's daily activities. Specifically, it constructs activity history from location information and app usage history.

[0788] Step 3:

[0789] The server receives behavioral and emotional data sent from terminals in the cloud and integrates them. The input is behavioral and emotional data, which are combined to enhance the user profile. The output is a composite dataset ready for analysis. The operation here involves storing the data in a database and evaluating the relationship between emotions and behaviors.

[0790] Step 4:

[0791] The server acquires real-time environmental data (e.g., weather data, traffic information) from external data sources. The input is a data stream from an external API, which is used as correction data for CO2 emission calculations. The output is the corrected environmental data. Specifically, data is acquired via a Web API and converted to a specified format.

[0792] Step 5:

[0793] The server estimates CO2 emissions using integrated behavioral and emotional data and external environmental data. The input is the previously integrated data, and quantitative emissions calculations are performed based on this data. The output is the user's estimated CO2 emissions. Specifically, a calculation algorithm is run to aggregate the contribution of each data point.

[0794] Step 6:

[0795] The server generates feedback for the user using estimated CO2 emissions and sentiment data. The inputs are CO2 emissions and sentiment data, and the feedback is customized based on these two factors. The output is personalized action suggestions and information. For specific actions, the data is visualized to make the suggestions easy for the user to understand, and the suggestions are constructed accordingly.

[0796] Step 7:

[0797] The terminal receives feedback from the server and presents it through the user interface. The input is customized feedback data from the server, which is presented to the user in a visualized format. The output is a screen display that allows the user to take action. Specifically, it provides an intuitive display of information on the terminal screen, offering an interaction that allows the user to easily select their next action.

[0798] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0799] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0800] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0801] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0802] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0803] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0804] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0805] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0806] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0807] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0808] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0809] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0810] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0811] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0812] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0813] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0814] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0815] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0816] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0817] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0818] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0819] The following is further disclosed regarding the embodiments described above.

[0820] (Claim 1)

[0821] A data collection method for collecting behavioral data from users,

[0822] Estimation means for estimating CO2 emissions based on the aforementioned behavioral data,

[0823] An adjustment means that acquires real-time data from an external data source and takes into account fluctuating factors such as temperature and traffic conditions that affect the CO2 emissions calculated by the estimation means,

[0824] A visualization means for recording and visualizing the CO2 emissions obtained by the estimation means and adjustment means,

[0825] Furthermore, a means of proposing new actions,

[0826] A system that includes this.

[0827] (Claim 2)

[0828] The system according to claim 1, wherein the proposed means presents specific actions to reduce CO2 emissions based on the user's behavioral history.

[0829] (Claim 3)

[0830] The system according to claim 1, wherein the real-time data includes external weather data and traffic information, and these dynamically affect the CO2 emissions calculated by the estimation means.

[0831] "Example 1"

[0832] (Claim 1)

[0833] A device that collects behavioral information from users,

[0834] A function to calculate carbon dioxide emissions based on the aforementioned behavioral information,

[0835] A function that takes into account fluctuations in environmental conditions and traffic information that affect the calculated carbon dioxide emissions, based on real-time information obtained from external sources,

[0836] A device for recording and visualizing the calculated and considered carbon dioxide emissions,

[0837] Furthermore, it includes a suggestion function to provide new action plans,

[0838] A system that includes this.

[0839] (Claim 2)

[0840] The system according to claim 1, wherein the proposed function presents a specific action plan for reducing carbon dioxide emissions based on the user's behavioral history.

[0841] (Claim 3)

[0842] The system according to claim 1, wherein the real-time information includes external weather information and traffic information, and these dynamically affect the carbon dioxide emissions calculated by the calculation function.

[0843] "Application Example 1"

[0844] (Claim 1)

[0845] A data collection method for collecting configuration information from users,

[0846] Estimation means for estimating the amount of environmental load based on the aforementioned configuration information,

[0847] An adjustment means that acquires real-time information from an external source and takes into account fluctuating items such as weather and traffic conditions that affect the amount of environmental load calculated by the estimation means,

[0848] A visualization means for recording and visualizing the amount of environmental load obtained by the estimation means and adjustment means,

[0849] Furthermore, a means of proposing new actions,

[0850] A transportation guidance system that proposes the optimal public transport route based on the choice of mode of transport and visualizes the effect of reducing environmental impact,

[0851] A system that includes this.

[0852] (Claim 2)

[0853] The system according to claim 1, wherein the proposed means and the traffic guidance means present specific activities for reducing environmental impact based on the user's configuration history.

[0854] (Claim 3)

[0855] The system according to claim 1, wherein the real-time information includes external weather information and traffic information, and these dynamically affect the amount of environmental load calculated by the estimation means.

[0856] "Example 2 of combining an emotion engine"

[0857] (Claim 1)

[0858] Means for collecting behavioral and emotional information from users,

[0859] A means for estimating CO2 emissions based on the aforementioned behavioral information,

[0860] A means for acquiring real-time information from an external source and considering variable factors such as environmental conditions and movement status that affect the CO2 emissions calculated by the estimation means,

[0861] Means for analyzing the aforementioned emotional information and dynamically adjusting the information presentation method to provide information appropriate to the user's emotional state,

[0862] A means for recording and visually representing the CO2 emissions obtained by the estimation and adjustment means,

[0863] Furthermore, a means of proposing new actions using a generative AI model,

[0864] A system that includes this.

[0865] (Claim 2)

[0866] The system according to claim 1, wherein the proposed means presents specific activities for reducing CO2 emissions based on the user's behavioral history and emotional state.

[0867] (Claim 3)

[0868] The system according to claim 1, wherein the real-time information includes external weather information and movement conditions, which dynamically affect the CO2 emissions calculated by the estimation means, and which adjust the feedback content in conjunction with emotional information.

[0869] "Application example 2 when combining with an emotional engine"

[0870] (Claim 1)

[0871] A data collection method for collecting behavioral and emotional data from users,

[0872] Estimation means for estimating CO2 emissions based on the aforementioned behavioral data,

[0873] An adjustment means that acquires real-time data from an external data source and takes into account environmental factors that affect the CO2 emissions calculated by the estimation means,

[0874] A visualization means records and visualizes the CO2 emissions obtained by the estimation means and adjustment means, and dynamically adjusts the feedback according to the user's emotional state.

[0875] Furthermore, it proposes new actions and uses suggestion methods to appropriately adjust the content and timing of their presentation according to the user's emotions.

[0876] A system that includes this.

[0877] (Claim 2)

[0878] The system according to claim 1, wherein the proposed means presents specific actions to reduce CO2 emissions based on the user's behavioral history and emotional state.

[0879] (Claim 3)

[0880] The system according to claim 1, wherein the real-time data includes external environmental data and movement information, and these dynamically affect the CO2 emissions calculated by the estimation means. [Explanation of Symbols]

[0881] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A data collection method for collecting behavioral data from users, Estimation means for estimating CO2 emissions based on the aforementioned behavioral data, An adjustment means that acquires real-time data from an external data source and takes into account fluctuating factors such as temperature and traffic conditions that affect the CO2 emissions calculated by the estimation means, A visualization means for recording and visualizing the CO2 emissions obtained by the estimation means and adjustment means, Furthermore, a means of proposing new actions, A system that includes this.

2. The system according to claim 1, wherein the proposed means presents specific actions to reduce CO2 emissions based on the user's behavioral history.

3. The system according to claim 1, wherein the real-time data includes external weather data and traffic information, and these dynamically affect the CO2 emissions calculated by the estimation means.

Images