System

Abstract

A system is provided.SOLUTION: A system including means for collecting water flow data and environment data, means for preprocessing the collected data, means for constructing and learning a deep learning model using the preprocessed data, means for predicting a water flow using the constructed model, means for visualizing and notifying a prediction result, and means for performing infrastructure management based on the prediction result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In hydroelectric power generation, the strength of the water flow has a significant impact on the amount of power generated, so accurate water flow prediction is required. However, with conventional technology, it was difficult to effectively analyze water flow data and environmental data and predict fluctuating water flows with high accuracy. This resulted in reduced power generation efficiency, inadequate flood prevention measures, and issues with securing power during disasters. Furthermore, in managing water discharge from irrigation channels and dams, meticulous data analysis is required to adjust the appropriate timing and amount. Accordingly, more advanced prediction technology is essential to stabilize power supplies and improve the efficiency of infrastructure development. [Means for solving the problem]

[0005] This invention solves these problems by using deep learning to analyze water flow data and environmental data with high accuracy and build a predictive model. This system includes a means for collecting water flow data and environmental data, a means for preprocessing the collected data, a means for building and training a deep learning model using the preprocessed data, a means for predicting water flow using the built model, a means for visualizing and notifying the prediction results, and a means for managing infrastructure based on the prediction results. Furthermore, the model construction employs a long short-term memory network (LSTM) or a gated recurrent unit (GRU) to achieve high prediction accuracy. Furthermore, the data preprocessing process includes missing value imputation and data standardization, enabling more accurate data analysis. This maximizes power generation efficiency, stabilizes power supply, prevents flooding, and optimizes irrigation canal and dam management.

[0006] "Water flow data" is information on the flow rate and velocity of water in rivers, irrigation channels, etc.

[0007] "Environmental data" refers to information about external conditions that affect water flow, such as temperature, precipitation, topographical data, and seasonal factors.

[0008] "Preprocessing" refers to processing collected data such as filling in missing values, removing outliers, and standardizing the data to make it into an analyzable format.

[0009] A "deep learning model" is an algorithm that uses a multi-layer neural network to automatically extract features from large amounts of data and perform predictions and classifications.

[0010] A "long short-term memory network (LSTM)" is a type of recurrent neural network (RNN) that has high predictive accuracy for data with strong time dependency.

[0011] A "gated recurrent unit (GRU)" is a simplified version of an LSTM, a type of recurrent neural network (RNN) that also has high predictive accuracy for time-dependent data.

[0012] "Visualization" refers to the visual representation of data and prediction results using graphs and charts.

[0013] "Notification" refers to a means of informing users and administrators of prediction results and anomaly detection information, and includes emails and alerts.

[0014] "Infrastructure management" refers to the operation and management of infrastructure, such as dam water release plans, irrigation canal maintenance schedules, and power supply plans in the event of a disaster. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention is a system that collects water flow data and environmental data, and analyzes and predicts that data using a deep learning model. This system aims to maximize the efficiency of water flow power generation and optimize infrastructure management by collecting, analyzing, predicting, notifying, and applying information, with the server, terminal, and user each playing a specific role.

[0037] Data collection

[0038] Server: The server collects water flow data and environmental data from each observation point. The data is retrieved using API and received in JSON format. This data is stored in a database.

[0039] Data Preprocessing

[0040] Server: The collected data is first preprocessed by filling in missing values ​​and removing outliers. Data standardization is also performed, converting the data into a form suitable for analysis and learning.

[0041] Model Building

[0042] Server: Using the preprocessed data, a deep learning model is built. Recurrent neural networks such as long short-term memory networks (LSTM) and gated recurrent units (GRU) are used. This allows for the extraction of features from past data and the accurate prediction of future water flow.

[0043] Water flow prediction

[0044] Server: The constructed model is used to predict water flow. The predicted data is stored in a database or displayed in real time.

[0045] Result visualization and notification

[0046] Server: The prediction results are visualized in graphs and dashboards. If abnormal water flow is predicted, an alert is sent to the user or administrator via email or smartphone notification.

[0047] Infrastructure management applications

[0048] Terminal (user terminal): Dam water release plans and irrigation channel maintenance schedules are adjusted based on the prediction results. In the event of a disaster, emergency power supply plans are formulated based on the predicted data. If necessary, operations can be performed directly from the terminal.

[0049] Specific examples

[0050] 1. Data collection from observation points

[0051] For example, daily flow rate data, temperature, precipitation, and seasonal data are collected from a river observation point via API, and the server automatically stores this information in a database.

[0052] 2. Pretreatment

[0053] The collected data may contain missing or outliers. For such data, the missing values ​​are filled in using data from the previous or following day, and outliers are removed using statistical methods. Furthermore, the data is standardized to make it easier to analyze.

[0054] 3. Training the model

[0055] Using a long short-term memory network (LSTM), the model is trained with several years of past water flow and environmental data to capture temporal patterns and trends and accurately predict future water flow.

[0056] 4. Predicting water flow

[0057] The trained model is used to predict water flow for the following week or month. The predicted data is aggregated on the server and displayed in real time on a dashboard.

[0058] 5. Visualization and Notification

[0059] If an abnormality is predicted based on the prediction results, an alert will be sent to the administrator. For example, if a high water level is predicted, a warning will be sent to the administrator by email or smartphone app.

[0060] 6. Reflection in infrastructure management

[0061] Dam managers use the predicted water flow data to create advance water release schedules, determine the optimal timing for irrigation channel construction, and plan dredging and other work.

[0062] In this way, the present invention realizes efficient water current power generation, stabilization of power supply during disasters, and optimization of infrastructure management.

[0063] The processing flow will be explained below.

[0064] Step 1:

[0065] The server collects water flow data and environmental data from each observation point. Specifically, it sends an HTTP request through the API and receives the acquired data in JSON format. The received data is then stored in a database.

[0066] Step 2:

[0067] The server preprocesses the collected data. Specifically, it performs operations such as filling in missing values, removing outliers, and standardizing the data. For example, it uses the PANDAS library to create a data frame, fills in missing values ​​with surrounding values, and removes outliers by setting a cutoff value.

[0068] Step 3:

[0069] The server uses the preprocessed data to build a deep learning model, specifically, a long short-term memory network (LSTM) or a gated recurrent unit (GRU) using the Tensorflow (registered trademark) or Keras library, and trains the model using the training data.

[0070] Step 4:

[0071] The server uses the constructed model to predict future water flow, inputs test data into the trained model, and generates predicted water flow data, which is then stored in a database.

[0072] Step 5:

[0073] The server visualizes the forecast results, for example, using Matplotlib or Plotly libraries to display the predicted water flow data as graphs and charts, and the results are displayed in real time on a dashboard.

[0074] Step 6:

[0075] The server notifies users if an abnormal water flow is predicted. Specifically, if high or low water levels are detected based on the forecast data, an alert is sent to the user via email or smartphone notification.

[0076] Step 7:

[0077] The terminal (user terminal) manages infrastructure based on the prediction results. Based on the predicted data, it adjusts dam water release plans and irrigation channel maintenance schedules. For example, if high water flow is predicted, it plans to release water from the dam in advance to reduce the risk of flooding.

[0078] Step 8:

[0079] The terminal (user terminal) formulates a power supply plan for the time of a disaster. Based on the forecast data, it calculates the amount of power required in the event of a disaster and adjusts the power generation plan. This enables a stable power supply even in the event of a disaster.

[0080] Example 1

[0081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0082] Existing hydroelectric power generation systems and infrastructure management systems do not efficiently collect, preprocess, forecast, and notify real-time data. This makes it difficult to maximize power generation efficiency and respond quickly to disasters, and limits the accuracy and effectiveness of infrastructure management. Furthermore, in building predictive models, data preprocessing and missing value imputation are often insufficient, resulting in reduced prediction accuracy. This makes it difficult for conventional systems to achieve reliable forecasts and efficient infrastructure management.

[0083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0084] In this invention, the server includes means for collecting water flow data and environmental data, means for acquiring the collected data using an API and storing it in a database, means for preprocessing the collected data, filling in missing values, and removing outliers, means for building and training a deep learning model using the preprocessed data, means for predicting water flow using the built model, means for visualizing the prediction results in graphs or dashboards and notifying users of outliers, and means for managing infrastructure based on the prediction results. This makes it possible to efficiently preprocess collected data, perform highly accurate predictions, and visualize and notify users of the results. This enables more efficient water flow power generation, rapid disaster response, and precise infrastructure management.

[0085] "Water flow data" is information about the movement of water in bodies of water such as rivers, lakes, and dams, such as water flow rate, direction of flow, and speed.

[0086] "Environmental data" refers to information about the surrounding environment that affects water flow, such as temperature, precipitation, humidity, wind speed, and weather.

[0087] An "API (Application Programming Interface)" is a set of definitions and protocols for exchanging information between different software programs.

[0088] A "database" is a system for organizing and storing collected data and for efficiently searching, editing, and deleting it.

[0089] "Preprocessing" refers to a series of operations, such as filling in missing values, removing outliers, and standardizing data, that are performed to convert data into a form suitable for analysis and learning.

[0090] A "deep learning model" is a machine learning model based on neural networks that learns through a multi-layer network structure and has the ability to identify complex patterns.

[0091] A "long short-term memory network (LSTM)" is a type of recurrent neural network that has specialized units for efficiently learning long-term data dependencies.

[0092] A "gated recurrent unit (GRU)" is a type of recurrent neural network that has similar functionality to LSTM but has a simpler structure.

[0093] "Visualization" is the act of displaying data in a visual format, such as a graph or chart, to enable intuitive understanding of patterns and trends in the data.

[0094] "Infrastructure management" refers to the set of activities for maintaining, operating, and optimizing public facilities such as dams and irrigation channels, including efficient power supply and disaster response.

[0095] "Notification" refers to the means by which the system communicates abnormalities or important information to users and administrators, such as email or smartphone alerts.

[0096] This invention is a system that collects water flow data and environmental data, and analyzes and predicts this data using a deep learning model. This system aims to maximize the efficiency of water flow power generation and optimize infrastructure management by collecting, analyzing, predicting, notifying, and applying information, with the server, terminal, and user each playing a specific role.

[0097] The server uses an API to obtain water flow data and environmental data from observation points and receives it in JSON format. The received data is stored in a database. The collected data is then preprocessed, such as filling in missing values ​​and removing outliers. The data is also standardized and converted into a format suitable for analysis and learning.

[0098] Next, a deep learning model is constructed using the preprocessed data, using a long short-term memory network (LSTM) and a gated recurrent unit (GRU), which extracts features from past data and enables highly accurate prediction of future water flow.

[0099] The constructed model is used to predict water flow. The predicted data is then saved back into the database or displayed in real time on a dashboard. The prediction results are visualized in graphs and on a dashboard, and if an abnormal water flow is predicted, an alert is sent to the user or administrator. Notification methods include email and smartphone notifications.

[0100] Users and infrastructure managers can adjust dam water release plans and irrigation channel maintenance schedules based on the prediction results. If necessary, they can also operate the system directly from their terminals. Furthermore, in the event of a disaster, emergency power supply plans can be formulated based on the predicted data.

[0101] As a concrete example, daily flow rate data, temperature, precipitation, and seasonal data are collected from a river observation point via an API. For example, the API endpoint is "https: / / api.example.com / river_data", and this information is automatically stored in a database on the server.

[0102] Next, in the preprocessing stage, if the collected data contains missing or outliers, they are complemented or removed using previous or subsequent dates or statistical methods. All data is also standardized and organized into a form that is easy to analyze.

[0103] When using a long short-term memory network (LSTM), the network is trained with several years of water flow data and environmental data. Through this training process, a model is constructed that can accurately predict future water flow by capturing temporal patterns and trends.

[0104] Below is an example of a prompt sentence.

[0105] Example prompt sentence:

[0106] "Predict the water flow for the next month. Provide flow data, temperature data, and precipitation data for the past five years."

[0107] In this way, the present invention realizes efficient water current power generation, stabilization of power supply during disasters, and optimization of infrastructure management.

[0108] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0109] Step 1: Data collection

[0110] The server collects water flow data and environmental data from each observation point using an API. Specifically, it obtains daily flow data, temperature, precipitation, and seasonal data via the API endpoint and receives them in JSON format.

[0111] Input: API endpoint of observation point

[0112] Output: Water flow and environmental data in JSON format

[0113] Specific behavior:

[0114] The server runs the script every day at midnight and retrieves new data from the specified API endpoint "https: / / api.example.com / river_data".

[0115] The acquired data is saved in a table called "river_data" in the internal database.

[0116] Step 2: Data Preprocessing

[0117] The server performs preprocessing on the collected data, completing missing values, removing outliers, and standardizing the data.

[0118] Input: Raw water flow and environmental data stored in a database

[0119] Output: A standardized dataset with missing values ​​imputed and outliers removed

[0120] Specific behavior:

[0121] The server retrieves the raw data from the database.

[0122] For missing values, linear interpolation is performed using values ​​from the preceding and following days.

[0123] For outliers, data beyond ±3σ from the standard deviation are removed and replaced by the median.

[0124] All data is normalized to a range of 0 to 1.

[0125] Step 3: Model Building

[0126] The server uses the preprocessed data to build a deep learning model, specifically an LSTM or GRU model.

[0127] Input: Preprocessed dataset

[0128] Output: Trained deep learning model

[0129] Specific behavior:

[0130] The server defines the model using deep learning libraries such as TensorFlow and PyTorch.

[0131] The LSTM model is trained using the past five years of data as training data.

[0132] The parameters for the training process are set to 100 epochs, 32 batch size, and the Adam optimizer is used.

[0133] Step 4: Water flow prediction

[0134] Using the constructed model, the server predicts future water flow.

[0135] Input: A trained deep learning model and the latest preprocessed data

[0136] Output: Predicted future water flow data

[0137] Specific behavior:

[0138] The server inputs the latest preprocessed data into the model to predict water flow for the next month.

[0139] The prediction results are stored in the "predicted_flow" table in the database.

[0140] Step 5: Visualizing and notifying results

[0141] The server visualizes the prediction results in graphs and dashboards and notifies users of any abnormalities as needed.

[0142] Input: Predicted water flow data

[0143] Output: Visualized data display and notifications

[0144] Specific behavior:

[0145] The server uses the forecast data to create a line graph on the dashboard.

[0146] If an abnormal water flow is predicted, the administrator will be notified by email and smartphone. The subject of the email will be "High Water Level Forecast Alert," and the content will include the predicted date and time and the predicted value.

[0147] Step 6: Applying it to infrastructure management

[0148] Users and devices (user devices) use the prediction results to manage infrastructure, including adjusting dam water release plans and irrigation canal maintenance schedules.

[0149] Input: Predicted water flow data and visualized results

[0150] Output: Optimized infrastructure management plan

[0151] Specific behavior:

[0152] Users access the dashboard and check the prediction results.

[0153] Dam operators use the forecast data to adjust water release schedules and plan dredging operations.

[0154] If necessary, the dam can be operated directly from the terminal to quickly respond by releasing water in advance.

[0155] (Application example 1)

[0156] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0157] With the advancement of autonomous driving technology in recent years, there is a growing demand for autonomous vehicles to be able to respond to floods and unexpected environmental changes. However, current systems have difficulty effectively utilizing water flow and environmental data to optimize vehicle routes. This poses a challenge: there is a lack of technology to enable safe route selection during floods and efficient infrastructure management.

[0158] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0159] In this invention, the server includes means for collecting water flow data and environmental data, means for preprocessing the collected data, means for building and training a deep learning model using the preprocessed data, and means for optimizing the vehicle's driving route based on the prediction results, thereby enabling autonomous vehicles to select safe and efficient driving routes even during floods.

[0160] "Water flow data" refers to data on the flow rate, flow velocity, and flow direction of water measured in rivers, irrigation channels, etc.

[0161] "Environmental data" refers to data on weather such as temperature, precipitation, wind speed, and humidity, as well as data on the topography and land use status around rivers.

[0162] "Preprocessing" refers to data preparation processes such as filling in missing values, removing outliers, and standardizing the collected data.

[0163] A "deep learning model" is an algorithm that uses multi-layer neural networks to learn features from large amounts of data and perform tasks such as prediction and classification.

[0164] A "long short-term memory network (LSTM)" is a type of recurrent neural network (RNN) that is a neural network structure that can learn dependencies over long periods of time.

[0165] A "gated recurrent unit (GRU)" is a type of recurrent neural network (RNN), a neural network structure with a small number of parameters and high computational efficiency.

[0166] "Prediction" is the process of using a model to estimate future data.

[0167] "Visualization" is the conversion of data and forecast results into a visually understandable format such as graphs, charts, and dashboards.

[0168] "Notification" is a means of conveying warnings or information to users or administrators when certain conditions are met.

[0169] "Infrastructure management" refers to the maintenance and management of public infrastructure, including dam water release plans, irrigation canal construction schedules, and emergency power supply plans in the event of a disaster.

[0170] "Optimizing vehicle routes" means calculating the optimal route for autonomous vehicles based on water flow and environmental data, thereby achieving efficient and safe travel.

[0171] This invention is a system that optimizes the driving routes of autonomous vehicles by collecting water flow data and environmental data and analyzing and predicting them using a deep learning model. This system enables safe and efficient driving by having the server, terminal, and user each play specific roles and collect, analyze, predict, notify, and apply information.

[0172] Data collection

[0173] The server collects water flow data and environmental data from each observation point. The data collected using the API is received in JSON format and stored in a database.

[0174] Data Preprocessing

[0175] The server performs preprocessing on the collected data, such as filling in missing values ​​and removing outliers, and also standardizes the data, converting it into a form suitable for analysis and learning.

[0176] Model Building

[0177] The server uses the preprocessed data to build a deep learning model that uses long short-term memory networks (LSTM) and gated recurrent units (GRUs) to extract features from past data and predict future water flow with high accuracy.

[0178] Water flow prediction

[0179] The server uses the constructed model to predict water flow, and the predicted data is stored in a database or displayed in real time.

[0180] Result visualization and notification

[0181] The server visualizes the prediction results in graphs and dashboards. If abnormal water flow is predicted, an alert is sent to users and administrators via email, smartphone notifications, and other means.

[0182] Vehicle route optimization

[0183] The server calculates the optimal route for the autonomous vehicle based on the prediction results and transmits it to the device, allowing users to avoid the risk of flooding and travel safely and efficiently.

[0184] Specific examples

[0185] For example, daily flow rate data, temperature, precipitation, and seasonal data are collected from a river observation point via API. The server automatically stores this information in a database and performs preprocessing.

[0186] The collected data may contain missing or outliers. For such data, the missing values ​​are filled in using data from the previous or following day, and outliers are removed using statistical methods. Furthermore, the data is standardized to make it easier to analyze.

[0187] Using a long short-term memory network (LSTM), the model is trained with several years of past water flow and environmental data to capture temporal patterns and trends and accurately predict future water flow.

[0188] The trained model is used to predict water flow for the following week or month. The predicted data is aggregated on the server and displayed in real time on a dashboard.

[0189] If an abnormality is predicted based on the prediction results, an alert will be sent to the administrator. For example, if a high water level is predicted, a warning will be sent to the administrator by email or smartphone app.

[0190] Dam managers use the predicted water flow data to create advance water release schedules, determine the optimal timing for irrigation channel construction, and plan dredging and other work.

[0191] Furthermore, the system calculates the optimal route for autonomous vehicles based on the prediction results to avoid the risk of flooding. For example, if heavy rain is predicted in the Kyushu region, the system can use that information to optimize the route of autonomous vehicles in real time, making it possible to avoid areas with a high risk of flooding.

[0192] Example prompts for generative AI models

[0193] Use the following water flow and environmental data to predict the optimal path for an autonomous vehicle.

[0194] Data Format:

[0195] Water flow data (flow rate, hostel (latitude and longitude of each point), timestamp)

[0196] Environmental data (temperature, precipitation, wind speed, humidity, hostel, timestamp)

[0197] [Example of water flow data]

[0198] Flow rate: 200, Hostle: Latitude: 33.590354, Longitude: 130.401716, Time stamp: "2023-10-01T00:00:00Z"

[0199] [Example of environmental data]

[0200] Temperature: 25, Precipitation: 50, Wind Speed: 10, Humidity: 80, Hostal: Latitude: 33.590354, Longitude: 130.401716, Time Stamp: "2023-10-01T00:00:00Z"

[0201] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0202] Step 1:

[0203] The server collects water flow data and environmental data from each observation point using API. The input data includes flow rate, temperature, precipitation, wind speed, humidity, etc., and is received in JSON format. This data is stored in a database for the next processing step.

[0204] Step 2:

[0205] The server performs preprocessing on the collected data. Because the input data contains missing values ​​and outliers, the server first fills in the missing values ​​from the preceding and following data and removes the outliers using statistical methods. Next, the server standardizes the data and converts it into a format suitable for model training. The output is the preprocessed data.

[0206] Step 3:

[0207] The server uses the preprocessed data to build and train a deep learning model. This step uses long short-term memory networks (LSTM) and gated recurrent units (GRU). The input data is historical water flow and environmental data, and the model is trained based on this. The output is a trained deep learning model.

[0208] Step 4:

[0209] The server uses the trained model to predict future water flow. The input is the latest preprocessed data, on which the model makes predictions. The output is the predicted future water flow data, which is also stored in the database.

[0210] Step 5:

[0211] The server visualizes the prediction results and sends notifications as necessary. The input is the prediction results, and alerts and graphs are generated based on these. The visualized data is displayed on a dashboard, and if an abnormality is predicted, an email or smartphone notification is sent to the administrator or user. The output is the visualized data and notification information.

[0212] Step 6:

[0213] The server optimizes the autonomous vehicle's driving route based on the prediction results. The input is predicted water flow and environmental data, and the optimal driving route is calculated based on this. This calculation is performed in real time, and the calculated route information is sent to the terminal or vehicle. The output is the optimized driving route.

[0214] Keywords: generative AI model, prompt sentence

[0215] We have shown the specific processing steps of an application example.

[0216] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0217] This invention is a system that collects water flow data and environmental data, analyzes and predicts that data using a deep learning model, and combines it with an emotion engine that recognizes user emotions. This system aims to maximize the efficiency of water flow power generation and optimize infrastructure management by collecting, analyzing, predicting, notifying, and applying information, with the server, terminal, and user each playing a specific role, and further enables interaction according to the user's emotions.

[0218] Data collection

[0219] Server: The server collects water flow data and environmental data from each observation point. The data is retrieved using API and received in JSON format. This data is stored in a database.

[0220] Data Preprocessing

[0221] Server: The collected data is first preprocessed by filling in missing values ​​and removing outliers. Data standardization is also performed, converting the data into a form suitable for analysis and learning.

[0222] Model Building

[0223] Server: Using the preprocessed data, a deep learning model is built. Recurrent neural networks such as long short-term memory networks (LSTM) and gated recurrent units (GRU) are used. This allows for the extraction of features from past data and the accurate prediction of future water flow.

[0224] Water flow prediction

[0225] Server: The constructed model is used to predict water flow. The predicted data is stored in a database or displayed in real time.

[0226] Result visualization and notification

[0227] Server: The prediction results are visualized in graphs and dashboards. If abnormal water flow is predicted, an alert is sent to the user or administrator via email or smartphone notification.

[0228] Emotion Engine Operation

[0229] Server: The emotion engine recognizes the user's emotions. When the user confirms the prediction results, the emotion engine analyzes their reactions and feedback to identify the user's emotional state.

[0230] Emotion-Based Interaction

[0231] Server: The emotion engine selects appropriate visualizations and notifications based on the user's emotional state. For example, if the user is feeling anxious, it provides detailed explanations and additional support information. If optimistic emotions are recognized, it provides concise, positive feedback.

[0232] Infrastructure management applications

[0233] Terminal (user terminal): Dam water release plans and irrigation channel maintenance schedules are adjusted based on the prediction results. In the event of a disaster, emergency power supply plans are formulated based on the predicted data. If necessary, operations can be performed directly from the terminal.

[0234] Specific examples

[0235] 1. Data collection from observation points

[0236] For example, daily flow rate data, temperature, precipitation, and seasonal data are collected from a river observation point via API, and the server automatically stores this information in a database.

[0237] 2. Pretreatment

[0238] The collected data may contain missing or outliers. For such data, the missing values ​​are filled in using data from the previous or following day, and outliers are removed using statistical methods. Furthermore, the data is standardized to make it easier to analyze.

[0239] 3. Training the model

[0240] Using a long short-term memory network (LSTM), the model is trained with several years of past water flow and environmental data to capture temporal patterns and trends and accurately predict future water flow.

[0241] 4. Predicting water flow

[0242] The trained model is used to predict water flow for the following week or month. The predicted data is aggregated on the server and displayed in real time on a dashboard.

[0243] 5. Visualization and Notification

[0244] If an abnormality is predicted based on the prediction results, an alert will be sent to the administrator. For example, if a high water level is predicted, a warning will be sent to the administrator by email or smartphone app.

[0245] 6. User Emotion Recognition by Emotion Engine

[0246] The user's reaction after checking the prediction results is captured by sensors and cameras and analyzed by the emotion engine. For example, emotions such as joy, surprise, or anxiety can be identified from the user's facial expressions and tone of voice.

[0247] 7. Emotion-based interactions

[0248] The emotion engine recognizes the user's emotions and then responds accordingly: if the user appears anxious, the server can provide additional information or support links, or if the user is happy, it can simply provide a brief report.

[0249] 8. Reflection in infrastructure management

[0250] Dam managers can use the predicted water flow data to create advance water release schedules, determine the optimal timing for irrigation channel maintenance, and carry out dredging work in a planned manner. At the same time, they can use the feedback provided by the emotion engine to increase user satisfaction.

[0251] In this way, the present invention not only improves the efficiency of water current power generation, stabilizes power supply during disasters, and optimizes infrastructure management, but also improves the user experience of the entire system through interactions that are tailored to the user's emotions.

[0252] The processing flow will be explained below.

[0253] Step 1:

[0254] The server collects water flow data and environmental data from each observation point. Specifically, it sends an HTTP request through the API and retrieves the received data in JSON format. The retrieved data is then stored in a database.

[0255] Step 2:

[0256] The server preprocesses the collected data by creating a data frame, interpolating missing values ​​with previous and next values, and removing outliers using statistical methods. It also standardizes the data and converts it into a format suitable for model training.

[0257] Step 3:

[0258] The server uses the preprocessed data to build a deep learning model, specifically a neural network incorporating LSTM or GRU using TensorFlow or Keras, and trains it with the training data.

[0259] Step 4:

[0260] The server uses the constructed model to predict future water flow, inputs new data into the trained model, and generates predicted water flow data, which is then stored in a database or displayed in real time.

[0261] Step 5:

[0262] The server visualizes the forecast results, for example, using Matplotlib or Plotly to display the predicted water flow data as graphs and charts, which are then displayed in real time on a dashboard.

[0263] Step 6:

[0264] The server notifies users when abnormal water flows are predicted. Specifically, if high or low water levels are detected based on the forecast data, an alert is sent via email or smartphone notification.

[0265] Step 7:

[0266] The server recognizes the user's emotions using an emotion engine. When the user confirms the prediction results, the emotion engine analyzes their reactions and feedback to identify the user's emotional state.

[0267] Step 8:

[0268] The server provides interactions that are tailored to the user's emotional state, for example providing detailed explanations and additional support information if the user is anxious, or providing concise, positive feedback if the user is optimistic.

[0269] Step 9:

[0270] The terminal (user terminal) manages infrastructure based on the prediction results. Dam water release plans and irrigation channel maintenance schedules are adjusted based on the predicted water flow data. If necessary, a power supply plan for emergencies is also formulated.

[0271] Step 10:

[0272] The terminal (user terminal) utilizes feedback from the emotion engine. Infrastructure management is further optimized based on the user's emotions. For example, it checks whether the user feels comfortable with the proposed water discharge schedule and makes additional adjustments if necessary.

[0273] Example 2

[0274] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0275] Conventional water flow prediction systems were able to make predictions to a certain extent by analyzing collected environmental and water flow data, but no systems existed that took into account the user's emotions regarding the prediction results. As a result, they were unable to address user concerns and questions, which resulted in problems with system usability and user experience. In response to this, a new system was needed that could collect and analyze a variety of water flow and environmental data, and provide appropriate notifications and information according to the user's emotional state.

[0276] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0277] In this invention, the server includes means for collecting water flow data and environmental data, means for preprocessing the collected data, means for constructing and training a deep learning model using the preprocessed data, means for predicting water flow using the constructed model, means for visualizing and notifying the prediction results, means for managing infrastructure based on the prediction results, means for recognizing user emotions, and means for providing appropriate notifications and information based on the recognized user emotions. This enables more accurate water flow predictions and flexible responses according to the user's emotional state, resulting in improved system utilization efficiency and user experience.

[0278] "Water flow data" refers to data relating to the flow rate, flow velocity, and flow rate fluctuations of water in bodies of water such as rivers and dams.

[0279] "Environmental data" refers to meteorological data such as temperature, precipitation, wind speed, and humidity, as well as data on environmental conditions such as local land use and river water quality.

[0280] A "deep learning model" is a field of artificial intelligence that uses multi-layer neural networks to learn the characteristics of data and perform advanced analysis and predictions.

[0281] "Preprocessing" refers to the process of preparing collected raw data in a format suitable for analysis and model learning by filling in missing values, removing outliers, and standardizing the data.

[0282] A long short-term memory network (LSTM) is a type of recurrent neural network (RNN) that is an algorithm for analyzing and predicting data with temporal dependencies, and is particularly effective at learning long-term dependencies.

[0283] A "gated recurrent unit (GRU)" is a type of recurrent neural network, similar to LSTM, and is an algorithm characterized by having few parameters and high learning efficiency.

[0284] "Visualization" is the process of displaying analysis and prediction results in the form of graphs, charts, dashboards, etc., making them intuitively easy for users to understand.

[0285] "Notification" is a process in which the system communicates important information or alerts to users or administrators via email, smartphone push notifications, etc.

[0286] "Infrastructure management" is the process of determining dam discharge plans and irrigation canal maintenance schedules based on water flow forecast results, and formulating power supply plans in the event of a disaster.

[0287] The "emotion engine" is a system that analyzes the user's reactions and feedback and recognizes the user's emotional state (joy, anxiety, surprise, etc.).

[0288] "Notification and information provision based on user emotions" is the process of providing appropriate notification methods, visualization techniques, and support information according to the user's emotional state recognized by the emotion engine.

[0289] This invention is a system that collects water flow data and environmental data, and analyzes and predicts them using a deep learning model. In this system, the server, terminals, and users each play specific roles.

[0290] First, the server uses an API to collect water flow data and environmental data from each observation point. The data is received in JSON format and stored in a database. Next, the server preprocesses the collected data. Specifically, it fills in missing values, removes outliers, and standardizes the data. This converts the data into a form suitable for analysis and learning.

[0291] The server then uses the preprocessed data to build a deep learning model. It uses recurrent neural networks (RNNs) such as LSTM and GRU to extract features from past data, enabling highly accurate prediction of future water flow. The trained model is used to make predictions, and the results are stored in a database or displayed in real time.

[0292] The server visualizes the prediction results using graphs and dashboards, and if abnormal water flow is predicted, users and administrators will receive alerts via email or smartphone notifications.

[0293] Furthermore, the system is equipped with an emotion engine. When the user confirms the prediction results, the server captures their reactions and feedback using sensors and cameras, which are then analyzed by the emotion engine. The system identifies the user's emotional state from their facial expressions and tone of voice and provides appropriate notifications and information. For example, if the user is feeling anxious, the system will provide a detailed explanation and additional support information. Conversely, if the user is optimistic, it will provide a concise report.

[0294] This system aims to maximize the efficiency of water current power generation and optimize infrastructure management, and also improves the overall user experience through flexible responses based on user emotions.

[0295] Specific operation example

[0296] Data collection from observation points

[0297] The server collects daily flow rate, temperature, and precipitation data from a river observation point via API and automatically stores them in a database. This collection process is carried out using a script that automatically issues API requests at specific times.

[0298] Pre-processing

[0299] If the data obtained via the API contains missing or outliers, the server will fill in the missing parts with data from the previous or following day. Statistical methods are used to remove outliers, and the flow data is then standardized, improving the learning efficiency of the model.

[0300] Model training

[0301] The server builds a deep learning model using LSTM and GRU, and trains it on three years of water flow data and environmental data, building a model that can capture temporal patterns and trends and predict future water flow with high accuracy.

[0302] Water flow prediction and visualization

[0303] The trained model is used to predict water flow for the next week or month. The predicted data is aggregated on the server and displayed on a dashboard in real time. The prediction results are visualized in graphs and charts, allowing users to understand them intuitively.

[0304] Alert Notification

[0305] If an abnormality is predicted based on the prediction results, an alert will be sent to the administrator via email or smartphone app. For example, if a high water level is predicted, countermeasures can be taken immediately.

[0306] Emotion recognition and response using emotion engine

[0307] When the user confirms the prediction, their reaction is captured by sensors and cameras and analyzed by the emotion engine. For example, if the user appears anxious, the server will provide additional information or support links. Conversely, if the user is satisfied, it will provide a brief report.

[0308] Prompt Sentence Examples

[0309] "Please describe a system that uses a deep learning model to predict future water flow based on water flow data and environmental data. Also, explain in detail how this system is combined with an emotion engine that recognizes the user's emotions, and optimizes actions based on the prediction results depending on the user's emotional state."

[0310] When this prompt sentence is input into a generative AI model, an explanation of the system processing described above is generated.

[0311] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0312] Step 1: Collect data

[0313] server:

[0314] The server collects water flow data and environmental data from the observation points. Specifically, it obtains daily flow data, temperature, precipitation, and other data in JSON format through an API. This data is stored in a database. The input is the API response data from the observation points, and the output is the unprocessed raw data stored in the database.

[0315] Step 2: Preprocessing the data

[0316] server:

[0317] The server performs preprocessing on the collected data. Specifically, it imputes missing values ​​using data from the previous or next day and removes outliers using statistical methods. It also standardizes the data and converts it into a form suitable for analysis and model training. The input is raw data stored in a database, and the output is a preprocessed dataset.

[0318] Step 3: Building the model

[0319] server:

[0320] The server uses the preprocessed data to build a deep learning model. Specifically, it uses recurrent neural networks such as LSTM or GRU to extract features from past data. The input is the preprocessed dataset, and the output is a trained deep learning model.

[0321] Step 4: Predicting water flow

[0322] server:

[0323] The trained model is used to predict water flow. Specifically, it predicts the water flow rate for the next week or month, and the results are stored in a database or displayed in real time. The input is the trained model and newly collected preprocessed data, and the output is the predicted water flow data.

[0324] Step 5: Visualize the results

[0325] server:

[0326] The server visualizes the prediction results using graphs and dashboards. Specifically, it displays them in an easy-to-understand manner for users using line graphs, heat maps, etc. The input is the predicted water flow data, and the output is a visualized data display.

[0327] Step 6: Alert Notification

[0328] server:

[0329] If an abnormal water flow is predicted, the server sends an alert to the user or administrator via email or push notification on a smartphone. The input is the prediction result data, and the output is a notification message.

[0330] Step 7: Emotion Recognition with the Emotion Engine

[0331] server:

[0332] When the user confirms the prediction results, their reactions and feedback are captured by sensors and cameras and analyzed by the emotion engine. Specifically, the system identifies the user's emotional state from their facial expressions and tone of voice. The input is the user's reaction data, and the output is the analyzed emotional data.

[0333] Step 8: Emotion-based interaction

[0334] server:

[0335] The emotion engine recognizes the user's emotional state and provides appropriate notifications and information accordingly. Specifically, if the user appears anxious, it provides detailed explanations and additional supporting information, and if the user appears optimistic, it provides a concise report. The input is the analyzed emotion data, and the output is emotion-based notifications and information.

[0336] Step 9: Applying it to infrastructure management

[0337] Terminal (user terminal):

[0338] Based on the forecast results provided by the server, dam water release plans and irrigation channel maintenance schedules are adjusted. Specifically, dam operations and emergency power supply plans are formulated from terminals as needed. The input is predicted water flow data and emotion-based notifications and alerts, and the output is the execution of infrastructure management plans.

[0339] (Application example 2)

[0340] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0341] Conventional logistics management systems lacked sufficient collection and analysis of environmental and operational data, making efficient logistics management and forecasting difficult. Furthermore, they were unable to recognize employee emotions and interact with them based on those emotions, preventing improvements in employee satisfaction and work efficiency. This resulted in the issue of not optimizing the overall performance of the logistics center.

[0342] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting water flow data and environmental data, means for preprocessing the collected data, means for constructing and training a deep learning model using the preprocessed data, means for predicting water flow using the constructed model, means for visualizing and notifying the prediction results, means for managing infrastructure based on the prediction results, means for collecting and analyzing user emotion data, and means for adjusting interactions based on the user's emotional state. This enables efficient collection and analysis of environmental data and operation data at the logistics center, enabling optimal work instructions and interactions based on employee emotions.

[0343] "Water flow data" is information about the flow of water, including the flow rate, speed, direction, and so on.

[0344] "Environmental data" refers to information related to the physical environment of interest, including temperature, humidity, lighting levels, and the like.

[0345] "Preprocessing" refers to processes such as filling in missing values, removing outliers, and standardizing data in order to convert collected data into a form suitable for analysis and model learning.

[0346] A "deep learning model" is a machine learning model that uses neural networks to extract features from large amounts of data and perform tasks such as prediction and classification.

[0347] "Prediction results" refer to future states and values ​​calculated using deep learning models, and include predicted information about water flow, environmental conditions, etc.

[0348] "Visualization" refers to displaying data and forecast results in the form of easy-to-understand graphs and dashboards.

[0349] "Notification" refers to sending important information or alerts to users via email, smartphone notifications, etc.

[0350] "Infrastructure management" refers to the efficient operation and maintenance of equipment and systems at logistics centers and other facilities.

[0351] "Emotion data" is information about the user's emotional state, and is data obtained from facial expressions, tone of voice, behavior, and the like.

[0352] "Interaction" refers to the exchange of information and operations between a system and a user, and in particular refers to appropriate responses based on the user's emotions.

[0353] Data collection

[0354] The server collects environmental data (temperature, humidity, lighting level) and robot operation data (movement speed, location, remaining battery level) from each observation point within the logistics center. This data is obtained using an API and received in JSON format. The collected data is stored in a database.

[0355] Data Preprocessing

[0356] The server complements missing values ​​and removes outliers from the collected data, and also standardizes the data. Statistical methods are used to remove outliers, and data from the previous and following days is used to complement missing values. Libraries such as StandardScaler are used for standardization.

[0357] Model Building

[0358] The server uses the preprocessed data to build a deep learning model using a long short-term memory network (LSTM) or gated recurrent unit (GRU), which extracts features from past data and makes it possible to predict future environmental conditions and robot operating status with high accuracy.

[0359] Water flow prediction

[0360] The server uses the constructed model to make predictions, and the predicted data is stored in a database or displayed in real time on a dashboard. Based on the prediction results, optimal work plans within the logistics center can be created.

[0361] Result visualization and notification

[0362] The server visualizes the prediction results in graphs and dashboards, and if any abnormal prediction results are detected, it alerts users and administrators via email or smartphone notification functions.

[0363] Emotion Engine Operation

[0364] The server uses an emotion engine to recognize the user's emotions. It uses cameras and sensors to capture and analyze the user's reactions and feedback when checking the prediction results. It then identifies the user's emotional state from facial expressions and tone of voice.

[0365] Emotion-Based Interaction

[0366] The server selects appropriate visualization and notification methods based on the user's emotional state analyzed by the emotion engine. For example, if the user is feeling anxious, it provides detailed explanations and additional support information. On the other hand, if an optimistic emotion is recognized, it provides concise, positive feedback.

[0367] Infrastructure management applications

[0368] The server optimizes the robots' movements and work plans within the logistics center based on the prediction results and emotion data. For example, if a specific area is predicted to be hot, the robots' routes will be changed to avoid that area. Furthermore, when employees are under high stress, the robots will be used to reduce their workload.

[0369] Specific examples

[0370] Daily temperature, humidity, lighting levels, and robot operation data are collected via API from observation points installed in Area A. The server stores this data in a database, complements missing values, removes outliers, and standardizes the data. Next, an LSTM model is built using the preprocessed data to predict future environmental conditions in Area A. The prediction results are displayed in real time on a dashboard, and if high temperatures are predicted, an email notification is sent to the administrator. In addition, when the user checks the prediction results, their facial expressions are captured by a camera, and if they are feeling anxious, a detailed explanation is provided.

[0371] Example prompt sentence:

[0372] "Collect environmental data from the logistics center and robot operation data in real time, and use deep learning models to analyze and predict past data. At the same time, recognize employee emotions and provide optimal work instructions accordingly."

[0373] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0374] Step 1:

[0375] The server collects environmental data (temperature, humidity, lighting level) and robot operation data (movement speed, location, remaining battery power) from each observation point within the logistics center. This data is obtained through an API and received in JSON format. The input is various data from the observation points, and the output is a JSON file containing the raw data.

[0376] Step 2:

[0377] The server performs preprocessing on the collected data. It uses data from the previous and next days to fill in missing values ​​and statistical methods to remove outliers. It also standardizes the data using StandardScaler. The input is raw data in JSON format, and the output is preprocessed data.

[0378] Step 3:

[0379] The server uses the preprocessed data to build a deep learning model (LSTM or GRU). The model is built using the Keras or TensorFlow library. The input is the preprocessed data, and the output is a trained deep learning model.

[0380] Step 4:

[0381] The server uses the constructed model to predict future environmental conditions and the robot's operating status. The prediction results are stored in a database or displayed in real time on a dashboard. The input is the trained model and current data, and the output is predicted future data.

[0382] Step 5:

[0383] The server visualizes the prediction results and notifies users and administrators if any abnormal results are found. The server visualizes the results using graphs and dashboards, and issues alerts using email or smartphone notification functions. The input is the prediction results, and the output is the visualized data and notification messages.

[0384] Step 6:

[0385] The server uses an emotion engine to recognize the user's emotions. It uses cameras and sensors to capture the user's facial expressions and tone of voice, and then analyzes them to identify the user's emotional state. The input is the user's reaction data, and the output is analyzed emotional data.

[0386] Step 7:

[0387] The server adjusts the interaction based on the user's emotional state, for example, providing detailed explanations and additional support information if the user is anxious, or providing concise, positive feedback if the user is optimistic. The input is the analyzed emotional data, and the output is an optimized interaction.

[0388] Step 8:

[0389] The server optimizes work plans within the logistics center based on prediction results and emotional data. If high temperatures are predicted in a specific area, the robot's route is changed to avoid that area. Also, if an employee is under high stress, the robot is used to reduce their workload. The input is the prediction results and emotional data, and the output is an optimized work plan.

[0390] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0391] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0392] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0393] [Second embodiment]

[0394] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0395] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0396] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0397] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0398] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0399] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0400] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0401] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0402] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0403] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0404] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0405] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0406] This invention is a system that collects water flow data and environmental data, and analyzes and predicts that data using a deep learning model. This system aims to maximize the efficiency of water flow power generation and optimize infrastructure management by collecting, analyzing, predicting, notifying, and applying information, with the server, terminal, and user each playing a specific role.

[0407] Data collection

[0408] Server: The server collects water flow data and environmental data from each observation point. The data is retrieved using API and received in JSON format. This data is stored in a database.

[0409] Data Preprocessing

[0410] Server: The collected data is first preprocessed by filling in missing values ​​and removing outliers. Data standardization is also performed, converting the data into a form suitable for analysis and learning.

[0411] Model Building

[0412] Server: Using the preprocessed data, a deep learning model is built. Recurrent neural networks such as long short-term memory networks (LSTM) and gated recurrent units (GRU) are used. This allows for the extraction of features from past data and the accurate prediction of future water flow.

[0413] Water flow prediction

[0414] Server: The constructed model is used to predict water flow. The predicted data is stored in a database or displayed in real time.

[0415] Result visualization and notification

[0416] Server: The prediction results are visualized in graphs and dashboards. If abnormal water flow is predicted, an alert is sent to the user or administrator via email or smartphone notification.

[0417] Infrastructure management applications

[0418] Terminal (user terminal): Dam water release plans and irrigation channel maintenance schedules are adjusted based on the prediction results. In the event of a disaster, emergency power supply plans are formulated based on the predicted data. If necessary, operations can be performed directly from the terminal.

[0419] Specific examples

[0420] 1. Data collection from observation points

[0421] For example, daily flow rate data, temperature, precipitation, and seasonal data are collected from a river observation point via API, and the server automatically stores this information in a database.

[0422] 2. Pretreatment

[0423] The collected data may contain missing or outliers. For such data, the missing values ​​are filled in using data from the previous or following day, and outliers are removed using statistical methods. Furthermore, the data is standardized to make it easier to analyze.

[0424] 3. Training the model

[0425] Using a long short-term memory network (LSTM), the model is trained with several years of past water flow and environmental data to capture temporal patterns and trends and accurately predict future water flow.

[0426] 4. Predicting water flow

[0427] The trained model is used to predict water flow for the following week or month. The predicted data is aggregated on the server and displayed in real time on a dashboard.

[0428] 5. Visualization and Notification

[0429] If an abnormality is predicted based on the prediction results, an alert will be sent to the administrator. For example, if a high water level is predicted, a warning will be sent to the administrator by email or smartphone app.

[0430] 6. Reflection in infrastructure management

[0431] Dam managers use the predicted water flow data to create advance water release schedules, determine the optimal timing for irrigation channel construction, and plan dredging and other work.

[0432] In this way, the present invention realizes efficient water current power generation, stabilization of power supply during disasters, and optimization of infrastructure management.

[0433] The processing flow will be explained below.

[0434] Step 1:

[0435] The server collects water flow data and environmental data from each observation point. Specifically, it sends an HTTP request through the API and receives the acquired data in JSON format. The received data is then stored in a database.

[0436] Step 2:

[0437] The server preprocesses the collected data. Specifically, it performs operations such as filling in missing values, removing outliers, and standardizing the data. For example, it uses the PANDAS library to create a data frame, fills in missing values ​​with surrounding values, and removes outliers by setting a cutoff value.

[0438] Step 3:

[0439] The server uses the preprocessed data to build a deep learning model, specifically, a long short-term memory network (LSTM) or a gated recurrent unit (GRU) using the TensorFlow and Keras libraries, and trains the model using the training data.

[0440] Step 4:

[0441] The server uses the constructed model to predict future water flow, inputs test data into the trained model, and generates predicted water flow data, which is then stored in a database.

[0442] Step 5:

[0443] The server visualizes the forecast results, for example, using Matplotlib or Plotly libraries to display the predicted water flow data as graphs and charts, and the results are displayed in real time on a dashboard.

[0444] Step 6:

[0445] The server notifies users if an abnormal water flow is predicted. Specifically, if high or low water levels are detected based on the forecast data, an alert is sent to the user via email or smartphone notification.

[0446] Step 7:

[0447] The terminal (user terminal) manages infrastructure based on the prediction results. Based on the predicted data, it adjusts dam water release plans and irrigation channel maintenance schedules. For example, if high water flow is predicted, it plans to release water from the dam in advance to reduce the risk of flooding.

[0448] Step 8:

[0449] The terminal (user terminal) formulates a power supply plan for the time of a disaster. Based on the forecast data, it calculates the amount of power required in the event of a disaster and adjusts the power generation plan. This enables a stable power supply even in the event of a disaster.

[0450] Example 1

[0451] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0452] Existing hydroelectric power generation systems and infrastructure management systems do not efficiently collect, preprocess, forecast, and notify real-time data. This makes it difficult to maximize power generation efficiency and respond quickly to disasters, and limits the accuracy and effectiveness of infrastructure management. Furthermore, in building predictive models, data preprocessing and missing value imputation are often insufficient, resulting in reduced prediction accuracy. This makes it difficult for conventional systems to achieve reliable forecasts and efficient infrastructure management.

[0453] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0454] In this invention, the server includes means for collecting water flow data and environmental data, means for acquiring the collected data using an API and storing it in a database, means for preprocessing the collected data, filling in missing values, and removing outliers, means for building and training a deep learning model using the preprocessed data, means for predicting water flow using the built model, means for visualizing the prediction results in graphs or dashboards and notifying users of outliers, and means for managing infrastructure based on the prediction results. This makes it possible to efficiently preprocess collected data, perform highly accurate predictions, and visualize and notify users of the results. This enables more efficient water flow power generation, rapid disaster response, and precise infrastructure management.

[0455] "Water flow data" is information about the movement of water in bodies of water such as rivers, lakes, and dams, such as water flow rate, direction of flow, and speed.

[0456] "Environmental data" refers to information about the surrounding environment that affects water flow, such as temperature, precipitation, humidity, wind speed, and weather.

[0457] An "API (Application Programming Interface)" is a set of definitions and protocols for exchanging information between different software programs.

[0458] A "database" is a system for organizing and storing collected data and for efficiently searching, editing, and deleting it.

[0459] "Preprocessing" refers to a series of operations, such as filling in missing values, removing outliers, and standardizing data, that are performed to convert data into a form suitable for analysis and learning.

[0460] A "deep learning model" is a machine learning model based on neural networks that learns through a multi-layer network structure and has the ability to identify complex patterns.

[0461] A "long short-term memory network (LSTM)" is a type of recurrent neural network that has specialized units for efficiently learning long-term data dependencies.

[0462] A "gated recurrent unit (GRU)" is a type of recurrent neural network that has similar functionality to LSTM but has a simpler structure.

[0463] "Visualization" is the act of displaying data in a visual format, such as a graph or chart, to enable intuitive understanding of patterns and trends in the data.

[0464] "Infrastructure management" refers to the set of activities for maintaining, operating, and optimizing public facilities such as dams and irrigation channels, including efficient power supply and disaster response.

[0465] "Notification" refers to the means by which the system communicates abnormalities or important information to users and administrators, such as email or smartphone alerts.

[0466] This invention is a system that collects water flow data and environmental data, and analyzes and predicts this data using a deep learning model. This system aims to maximize the efficiency of water flow power generation and optimize infrastructure management by collecting, analyzing, predicting, notifying, and applying information, with the server, terminal, and user each playing a specific role.

[0467] The server uses an API to obtain water flow data and environmental data from observation points and receives it in JSON format. The received data is stored in a database. The collected data is then preprocessed, such as filling in missing values ​​and removing outliers. The data is also standardized and converted into a format suitable for analysis and learning.

[0468] Next, a deep learning model is constructed using the preprocessed data, using a long short-term memory network (LSTM) and a gated recurrent unit (GRU), which extracts features from past data and enables highly accurate prediction of future water flow.

[0469] The constructed model is used to predict water flow. The predicted data is then saved back into the database or displayed in real time on a dashboard. The prediction results are visualized in graphs and on a dashboard, and if an abnormal water flow is predicted, an alert is sent to the user or administrator. Notification methods include email and smartphone notifications.

[0470] Users and infrastructure managers can adjust dam water release plans and irrigation channel maintenance schedules based on the prediction results. If necessary, they can also operate the system directly from their terminals. Furthermore, in the event of a disaster, emergency power supply plans can be formulated based on the predicted data.

[0471] As a concrete example, daily flow rate data, temperature, precipitation, and seasonal data are collected from a river observation point via an API. For example, the API endpoint is "https: / / api.example.com / river_data", and this information is automatically stored in a database on the server.

[0472] Next, in the preprocessing stage, if the collected data contains missing or outliers, they are complemented or removed using previous or subsequent dates or statistical methods. All data is also standardized and organized into a form that is easy to analyze.

[0473] When using a long short-term memory network (LSTM), the network is trained with several years of water flow data and environmental data. Through this training process, a model is constructed that can accurately predict future water flow by capturing temporal patterns and trends.

[0474] Below is an example of a prompt sentence.

[0475] Example prompt sentence:

[0476] "Predict the water flow for the next month. Provide flow data, temperature data, and precipitation data for the past five years."

[0477] In this way, the present invention realizes efficient water current power generation, stabilization of power supply during disasters, and optimization of infrastructure management.

[0478] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0479] Step 1: Data collection

[0480] The server collects water flow data and environmental data from each observation point using an API. Specifically, it obtains daily flow data, temperature, precipitation, and seasonal data via the API endpoint and receives them in JSON format.

[0481] Input: API endpoint of observation point

[0482] Output: Water flow and environmental data in JSON format

[0483] Specific behavior:

[0484] The server runs the script every day at midnight and retrieves new data from the specified API endpoint "https: / / api.example.com / river_data".

[0485] The acquired data is saved in a table called "river_data" in the internal database.

[0486] Step 2: Data Preprocessing

[0487] The server performs preprocessing on the collected data, completing missing values, removing outliers, and standardizing the data.

[0488] Input: Raw water flow and environmental data stored in a database

[0489] Output: A standardized dataset with missing values ​​imputed and outliers removed

[0490] Specific behavior:

[0491] The server retrieves the raw data from the database.

[0492] For missing values, linear interpolation is performed using values ​​from the preceding and following days.

[0493] For outliers, data beyond ±3σ from the standard deviation are removed and replaced by the median.

[0494] All data is normalized to a range of 0 to 1.

[0495] Step 3: Model Building

[0496] The server uses the preprocessed data to build a deep learning model, specifically an LSTM or GRU model.

[0497] Input: Preprocessed dataset

[0498] Output: Trained deep learning model

[0499] Specific behavior:

[0500] The server defines the model using deep learning libraries such as TensorFlow and PyTorch.

[0501] The LSTM model is trained using the past five years of data as training data.

[0502] The parameters for the training process are set to 100 epochs, 32 batch size, and the Adam optimizer is used.

[0503] Step 4: Water flow prediction

[0504] Using the constructed model, the server predicts future water flow.

[0505] Input: A trained deep learning model and the latest preprocessed data

[0506] Output: Predicted future water flow data

[0507] Specific behavior:

[0508] The server inputs the latest preprocessed data into the model to predict water flow for the next month.

[0509] The prediction results are stored in the "predicted_flow" table in the database.

[0510] Step 5: Visualizing and notifying results

[0511] The server visualizes the prediction results in graphs and dashboards and notifies users of any abnormalities as needed.

[0512] Input: Predicted water flow data

[0513] Output: Visualized data display and notifications

[0514] Specific behavior:

[0515] The server uses the forecast data to create a line graph on the dashboard.

[0516] If an abnormal water flow is predicted, the administrator will be notified by email and smartphone. The subject of the email will be "High Water Level Forecast Alert," and the content will include the predicted date and time and the predicted value.

[0517] Step 6: Applying it to infrastructure management

[0518] Users and devices (user devices) use the prediction results to manage infrastructure, including adjusting dam water release plans and irrigation canal maintenance schedules.

[0519] Input: Predicted water flow data and visualized results

[0520] Output: Optimized infrastructure management plan

[0521] Specific behavior:

[0522] Users access the dashboard and check the prediction results.

[0523] Dam operators use the forecast data to adjust water release schedules and plan dredging operations.

[0524] If necessary, the dam can be operated directly from the terminal to quickly respond by releasing water in advance.

[0525] (Application example 1)

[0526] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0527] With the advancement of autonomous driving technology in recent years, there is a growing demand for autonomous vehicles to be able to respond to floods and unexpected environmental changes. However, current systems have difficulty effectively utilizing water flow and environmental data to optimize vehicle routes. This poses a challenge: there is a lack of technology to enable safe route selection during floods and efficient infrastructure management.

[0528] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0529] In this invention, the server includes means for collecting water flow data and environmental data, means for preprocessing the collected data, means for building and training a deep learning model using the preprocessed data, and means for optimizing the vehicle's driving route based on the prediction results, thereby enabling autonomous vehicles to select safe and efficient driving routes even during floods.

[0530] "Water flow data" refers to data on the flow rate, flow velocity, and flow direction of water measured in rivers, irrigation channels, etc.

[0531] "Environmental data" refers to data on weather such as temperature, precipitation, wind speed, and humidity, as well as data on the topography and land use status around rivers.

[0532] "Preprocessing" refers to data preparation processes such as filling in missing values, removing outliers, and standardizing the collected data.

[0533] A "deep learning model" is an algorithm that uses multi-layer neural networks to learn features from large amounts of data and perform tasks such as prediction and classification.

[0534] A "long short-term memory network (LSTM)" is a type of recurrent neural network (RNN) that is a neural network structure that can learn dependencies over long periods of time.

[0535] A "gated recurrent unit (GRU)" is a type of recurrent neural network (RNN), a neural network structure with a small number of parameters and high computational efficiency.

[0536] "Prediction" is the process of using a model to estimate future data.

[0537] "Visualization" is the conversion of data and forecast results into a visually understandable format such as graphs, charts, and dashboards.

[0538] "Notification" is a means of conveying warnings or information to users or administrators when certain conditions are met.

[0539] "Infrastructure management" refers to the maintenance and management of public infrastructure, including dam water release plans, irrigation canal construction schedules, and emergency power supply plans in the event of a disaster.

[0540] "Optimizing vehicle routes" means calculating the optimal route for autonomous vehicles based on water flow and environmental data, thereby achieving efficient and safe travel.

[0541] This invention is a system that optimizes the driving routes of autonomous vehicles by collecting water flow data and environmental data and analyzing and predicting them using a deep learning model. This system enables safe and efficient driving by having the server, terminal, and user each play specific roles and collect, analyze, predict, notify, and apply information.

[0542] Data collection

[0543] The server collects water flow data and environmental data from each observation point. The data collected using the API is received in JSON format and stored in a database.

[0544] Data Preprocessing

[0545] The server performs preprocessing on the collected data, such as filling in missing values ​​and removing outliers, and also standardizes the data, converting it into a form suitable for analysis and learning.

[0546] Model Building

[0547] The server uses the preprocessed data to build a deep learning model that uses long short-term memory networks (LSTM) and gated recurrent units (GRUs) to extract features from past data and predict future water flow with high accuracy.

[0548] Water flow prediction

[0549] The server uses the constructed model to predict water flow, and the predicted data is stored in a database or displayed in real time.

[0550] Result visualization and notification

[0551] The server visualizes the prediction results in graphs and dashboards. If abnormal water flow is predicted, an alert is sent to users and administrators via email, smartphone notifications, and other means.

[0552] Vehicle route optimization

[0553] The server calculates the optimal route for the autonomous vehicle based on the prediction results and transmits it to the device, allowing users to avoid the risk of flooding and travel safely and efficiently.

[0554] Specific examples

[0555] For example, daily flow rate data, temperature, precipitation, and seasonal data are collected from a river observation point via API. The server automatically stores this information in a database and performs preprocessing.

[0556] The collected data may contain missing or outliers. For such data, the missing values ​​are filled in using data from the previous or following day, and outliers are removed using statistical methods. Furthermore, the data is standardized to make it easier to analyze.

[0557] Using a long short-term memory network (LSTM), the model is trained with several years of past water flow and environmental data to capture temporal patterns and trends and accurately predict future water flow.

[0558] The trained model is used to predict water flow for the following week or month. The predicted data is aggregated on the server and displayed in real time on a dashboard.

[0559] If an abnormality is predicted based on the prediction results, an alert will be sent to the administrator. For example, if a high water level is predicted, a warning will be sent to the administrator by email or smartphone app.

[0560] Dam managers use the predicted water flow data to create advance water release schedules, determine the optimal timing for irrigation channel construction, and plan dredging and other work.

[0561] Furthermore, the system calculates the optimal route for autonomous vehicles based on the prediction results to avoid the risk of flooding. For example, if heavy rain is predicted in the Kyushu region, the system can use that information to optimize the route of autonomous vehicles in real time, making it possible to avoid areas with a high risk of flooding.

[0562] Example prompts for generative AI models

[0563] Use the following water flow and environmental data to predict the optimal path for an autonomous vehicle.

[0564] Data Format:

[0565] Water flow data (flow rate, hostel (latitude and longitude of each point), timestamp)

[0566] Environmental data (temperature, precipitation, wind speed, humidity, hostel, timestamp)

[0567] [Example of water flow data]

[0568] Flow rate: 200, Hostle: Latitude: 33.590354, Longitude: 130.401716, Time stamp: "2023-10-01T00:00:00Z"

[0569] [Example of environmental data]

[0570] Temperature: 25, Precipitation: 50, Wind Speed: 10, Humidity: 80, Hostal: Latitude: 33.590354, Longitude: 130.401716, Time Stamp: "2023-10-01T00:00:00Z"

[0571] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0572] Step 1:

[0573] The server collects water flow data and environmental data from each observation point using API. The input data includes flow rate, temperature, precipitation, wind speed, humidity, etc., and is received in JSON format. This data is stored in a database for the next processing step.

[0574] Step 2:

[0575] The server performs preprocessing on the collected data. Because the input data contains missing values ​​and outliers, the server first fills in the missing values ​​from the preceding and following data and removes the outliers using statistical methods. Next, the server standardizes the data and converts it into a format suitable for model training. The output is the preprocessed data.

[0576] Step 3:

[0577] The server uses the preprocessed data to build and train a deep learning model. This step uses long short-term memory networks (LSTM) and gated recurrent units (GRU). The input data is historical water flow and environmental data, and the model is trained based on this. The output is a trained deep learning model.

[0578] Step 4:

[0579] The server uses the trained model to predict future water flow. The input is the latest preprocessed data, on which the model makes predictions. The output is the predicted future water flow data, which is also stored in the database.

[0580] Step 5:

[0581] The server visualizes the prediction results and sends notifications as necessary. The input is the prediction results, and alerts and graphs are generated based on these. The visualized data is displayed on a dashboard, and if an abnormality is predicted, an email or smartphone notification is sent to the administrator or user. The output is the visualized data and notification information.

[0582] Step 6:

[0583] The server optimizes the autonomous vehicle's driving route based on the prediction results. The input is predicted water flow and environmental data, and the optimal driving route is calculated based on this. This calculation is performed in real time, and the calculated route information is sent to the terminal or vehicle. The output is the optimized driving route.

[0584] Keywords: generative AI model, prompt sentence

[0585] We have shown the specific processing steps of an application example.

[0586] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0587] This invention is a system that collects water flow data and environmental data, analyzes and predicts that data using a deep learning model, and combines it with an emotion engine that recognizes user emotions. This system aims to maximize the efficiency of water flow power generation and optimize infrastructure management by collecting, analyzing, predicting, notifying, and applying information, with the server, terminal, and user each playing a specific role, and further enables interaction according to the user's emotions.

[0588] Data collection

[0589] Server: The server collects water flow data and environmental data from each observation point. The data is retrieved using API and received in JSON format. This data is stored in a database.

[0590] Data Preprocessing

[0591] Server: The collected data is first preprocessed by filling in missing values ​​and removing outliers. Data standardization is also performed, converting the data into a form suitable for analysis and learning.

[0592] Model Building

[0593] Server: Using the preprocessed data, a deep learning model is built. Recurrent neural networks such as long short-term memory networks (LSTM) and gated recurrent units (GRU) are used. This allows for the extraction of features from past data and the accurate prediction of future water flow.

[0594] Water flow prediction

[0595] Server: The constructed model is used to predict water flow. The predicted data is stored in a database or displayed in real time.

[0596] Result visualization and notification

[0597] Server: The prediction results are visualized in graphs and dashboards. If abnormal water flow is predicted, an alert is sent to the user or administrator via email or smartphone notification.

[0598] Emotion Engine Operation

[0599] Server: The emotion engine recognizes the user's emotions. When the user confirms the prediction results, the emotion engine analyzes their reactions and feedback to identify the user's emotional state.

[0600] Emotion-Based Interaction

[0601] Server: The emotion engine selects appropriate visualizations and notifications based on the user's emotional state. For example, if the user is feeling anxious, it provides detailed explanations and additional support information. If optimistic emotions are recognized, it provides concise, positive feedback.

[0602] Infrastructure management applications

[0603] Terminal (user terminal): Dam water release plans and irrigation channel maintenance schedules are adjusted based on the prediction results. In the event of a disaster, emergency power supply plans are formulated based on the predicted data. If necessary, operations can be performed directly from the terminal.

[0604] Specific examples

[0605] 1. Data collection from observation points

[0606] For example, daily flow rate data, temperature, precipitation, and seasonal data are collected from a river observation point via API, and the server automatically stores this information in a database.

[0607] 2. Pretreatment

[0608] The collected data may contain missing or outliers. For such data, the missing values ​​are filled in using data from the previous or following day, and outliers are removed using statistical methods. Furthermore, the data is standardized to make it easier to analyze.

[0609] 3. Training the model

[0610] Using a long short-term memory network (LSTM), the model is trained with several years of past water flow and environmental data to capture temporal patterns and trends and accurately predict future water flow.

[0611] 4. Predicting water flow

[0612] The trained model is used to predict water flow for the following week or month. The predicted data is aggregated on the server and displayed in real time on a dashboard.

[0613] 5. Visualization and Notification

[0614] If an abnormality is predicted based on the prediction results, an alert will be sent to the administrator. For example, if a high water level is predicted, a warning will be sent to the administrator by email or smartphone app.

[0615] 6. User Emotion Recognition by Emotion Engine

[0616] The user's reaction after checking the prediction results is captured by sensors and cameras and analyzed by the emotion engine. For example, emotions such as joy, surprise, or anxiety can be identified from the user's facial expressions and tone of voice.

[0617] 7. Emotion-based interactions

[0618] The emotion engine recognizes the user's emotions and then responds accordingly: if the user appears anxious, the server can provide additional information or support links, or if the user is happy, it can simply provide a brief report.

[0619] 8. Reflection in infrastructure management

[0620] Dam managers can use the predicted water flow data to create advance water release schedules, determine the optimal timing for irrigation channel maintenance, and carry out dredging work in a planned manner. At the same time, they can use the feedback provided by the emotion engine to increase user satisfaction.

[0621] In this way, the present invention not only improves the efficiency of water current power generation, stabilizes power supply during disasters, and optimizes infrastructure management, but also improves the user experience of the entire system through interactions that are tailored to the user's emotions.

[0622] The processing flow will be explained below.

[0623] Step 1:

[0624] The server collects water flow data and environmental data from each observation point. Specifically, it sends an HTTP request through the API and retrieves the received data in JSON format. The retrieved data is then stored in a database.

[0625] Step 2:

[0626] The server preprocesses the collected data by creating a data frame, interpolating missing values ​​with previous and next values, and removing outliers using statistical methods. It also standardizes the data and converts it into a format suitable for model training.

[0627] Step 3:

[0628] The server uses the preprocessed data to build a deep learning model, specifically a neural network incorporating LSTM or GRU using TensorFlow or Keras, and trains it with the training data.

[0629] Step 4:

[0630] The server uses the constructed model to predict future water flow, inputs new data into the trained model, and generates predicted water flow data, which is then stored in a database or displayed in real time.

[0631] Step 5:

[0632] The server visualizes the forecast results, for example, using Matplotlib or Plotly to display the predicted water flow data as graphs and charts, which are then displayed in real time on a dashboard.

[0633] Step 6:

[0634] The server notifies users when abnormal water flows are predicted. Specifically, if high or low water levels are detected based on the forecast data, an alert is sent via email or smartphone notification.

[0635] Step 7:

[0636] The server recognizes the user's emotions using an emotion engine. When the user confirms the prediction results, the emotion engine analyzes their reactions and feedback to identify the user's emotional state.

[0637] Step 8:

[0638] The server provides interactions that are tailored to the user's emotional state, for example providing detailed explanations and additional support information if the user is anxious, or providing concise, positive feedback if the user is optimistic.

[0639] Step 9:

[0640] The terminal (user terminal) manages infrastructure based on the prediction results. Dam water release plans and irrigation channel maintenance schedules are adjusted based on the predicted water flow data. If necessary, a power supply plan for emergencies is also formulated.

[0641] Step 10:

[0642] The terminal (user terminal) utilizes feedback from the emotion engine. Infrastructure management is further optimized based on the user's emotions. For example, it checks whether the user feels comfortable with the proposed water discharge schedule and makes additional adjustments if necessary.

[0643] Example 2

[0644] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0645] Conventional water flow prediction systems were able to make predictions to a certain extent by analyzing collected environmental and water flow data, but no systems existed that took into account the user's emotions regarding the prediction results. As a result, they were unable to address user concerns and questions, which resulted in problems with system usability and user experience. In response to this, a new system was needed that could collect and analyze a variety of water flow and environmental data, and provide appropriate notifications and information according to the user's emotional state.

[0646] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0647] In this invention, the server includes means for collecting water flow data and environmental data, means for preprocessing the collected data, means for constructing and training a deep learning model using the preprocessed data, means for predicting water flow using the constructed model, means for visualizing and notifying the prediction results, means for managing infrastructure based on the prediction results, means for recognizing user emotions, and means for providing appropriate notifications and information based on the recognized user emotions. This enables more accurate water flow predictions and flexible responses according to the user's emotional state, resulting in improved system utilization efficiency and user experience.

[0648] "Water flow data" refers to data relating to the flow rate, flow velocity, and flow rate fluctuations of water in bodies of water such as rivers and dams.

[0649] "Environmental data" refers to meteorological data such as temperature, precipitation, wind speed, and humidity, as well as data on environmental conditions such as local land use and river water quality.

[0650] A "deep learning model" is a field of artificial intelligence that uses multi-layer neural networks to learn the characteristics of data and perform advanced analysis and predictions.

[0651] "Preprocessing" refers to the process of preparing collected raw data in a format suitable for analysis and model learning by filling in missing values, removing outliers, and standardizing the data.

[0652] A long short-term memory network (LSTM) is a type of recurrent neural network (RNN) that is an algorithm for analyzing and predicting data with temporal dependencies, and is particularly effective at learning long-term dependencies.

[0653] A "gated recurrent unit (GRU)" is a type of recurrent neural network, similar to LSTM, and is an algorithm characterized by having few parameters and high learning efficiency.

[0654] "Visualization" is the process of displaying analysis and prediction results in the form of graphs, charts, dashboards, etc., making them intuitively easy for users to understand.

[0655] "Notification" is a process in which the system communicates important information or alerts to users or administrators via email, smartphone push notifications, etc.

[0656] "Infrastructure management" is the process of determining dam discharge plans and irrigation canal maintenance schedules based on water flow forecast results, and formulating power supply plans in the event of a disaster.

[0657] The "emotion engine" is a system that analyzes the user's reactions and feedback and recognizes the user's emotional state (joy, anxiety, surprise, etc.).

[0658] "Notification and information provision based on user emotions" is the process of providing appropriate notification methods, visualization techniques, and support information according to the user's emotional state recognized by the emotion engine.

[0659] This invention is a system that collects water flow data and environmental data, and analyzes and predicts them using a deep learning model. In this system, the server, terminals, and users each play specific roles.

[0660] First, the server uses an API to collect water flow data and environmental data from each observation point. The data is received in JSON format and stored in a database. Next, the server preprocesses the collected data. Specifically, it fills in missing values, removes outliers, and standardizes the data. This converts the data into a form suitable for analysis and learning.

[0661] The server then uses the preprocessed data to build a deep learning model. It uses recurrent neural networks (RNNs) such as LSTM and GRU to extract features from past data, enabling highly accurate prediction of future water flow. The trained model is used to make predictions, and the results are stored in a database or displayed in real time.

[0662] The server visualizes the prediction results using graphs and dashboards, and if abnormal water flow is predicted, users and administrators will receive alerts via email or smartphone notifications.

[0663] Furthermore, the system is equipped with an emotion engine. When the user confirms the prediction results, the server captures their reactions and feedback using sensors and cameras, which are then analyzed by the emotion engine. The system identifies the user's emotional state from their facial expressions and tone of voice and provides appropriate notifications and information. For example, if the user is feeling anxious, the system will provide a detailed explanation and additional support information. Conversely, if the user is optimistic, it will provide a concise report.

[0664] This system aims to maximize the efficiency of water current power generation and optimize infrastructure management, and also improves the overall user experience through flexible responses based on user emotions.

[0665] Specific operation example

[0666] Data collection from observation points

[0667] The server collects daily flow rate, temperature, and precipitation data from a river observation point via API and automatically stores them in a database. This collection process is carried out using a script that automatically issues API requests at specific times.

[0668] Pre-processing

[0669] If the data obtained via the API contains missing or outliers, the server will fill in the missing parts with data from the previous or following day. Statistical methods are used to remove outliers, and the flow data is then standardized, improving the learning efficiency of the model.

[0670] Model training

[0671] The server builds a deep learning model using LSTM and GRU, and trains it on three years of water flow data and environmental data, building a model that can capture temporal patterns and trends and predict future water flow with high accuracy.

[0672] Water flow prediction and visualization

[0673] The trained model is used to predict water flow for the next week or month. The predicted data is aggregated on the server and displayed on a dashboard in real time. The prediction results are visualized in graphs and charts, allowing users to understand them intuitively.

[0674] Alert Notification

[0675] If an abnormality is predicted based on the prediction results, an alert will be sent to the administrator via email or smartphone app. For example, if a high water level is predicted, countermeasures can be taken immediately.

[0676] Emotion recognition and response using emotion engine

[0677] When the user confirms the prediction, their reaction is captured by sensors and cameras and analyzed by the emotion engine. For example, if the user appears anxious, the server will provide additional information or support links. Conversely, if the user is satisfied, it will provide a brief report.

[0678] Prompt Sentence Examples

[0679] "Please describe a system that uses a deep learning model to predict future water flow based on water flow data and environmental data. Also, explain in detail how this system is combined with an emotion engine that recognizes the user's emotions, and optimizes actions based on the prediction results depending on the user's emotional state."

[0680] When this prompt sentence is input into a generative AI model, an explanation of the system processing described above is generated.

[0681] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0682] Step 1: Collect data

[0683] server:

[0684] The server collects water flow data and environmental data from the observation points. Specifically, it obtains daily flow data, temperature, precipitation, and other data in JSON format through an API. This data is stored in a database. The input is the API response data from the observation points, and the output is the unprocessed raw data stored in the database.

[0685] Step 2: Preprocessing the data

[0686] server:

[0687] The server performs preprocessing on the collected data. Specifically, it imputes missing values ​​using data from the previous or next day and removes outliers using statistical methods. It also standardizes the data and converts it into a form suitable for analysis and model training. The input is raw data stored in a database, and the output is a preprocessed dataset.

[0688] Step 3: Building the model

[0689] server:

[0690] The server uses the preprocessed data to build a deep learning model. Specifically, it uses recurrent neural networks such as LSTM or GRU to extract features from past data. The input is the preprocessed dataset, and the output is a trained deep learning model.

[0691] Step 4: Predicting water flow

[0692] server:

[0693] The trained model is used to predict water flow. Specifically, it predicts the water flow rate for the next week or month, and the results are stored in a database or displayed in real time. The input is the trained model and newly collected preprocessed data, and the output is the predicted water flow data.

[0694] Step 5: Visualize the results

[0695] server:

[0696] The server visualizes the prediction results using graphs and dashboards. Specifically, it displays them in an easy-to-understand manner for users using line graphs, heat maps, etc. The input is the predicted water flow data, and the output is a visualized data display.

[0697] Step 6: Alert Notification

[0698] server:

[0699] If an abnormal water flow is predicted, the server sends an alert to the user or administrator via email or push notification on a smartphone. The input is the prediction result data, and the output is a notification message.

[0700] Step 7: Emotion Recognition with the Emotion Engine

[0701] server:

[0702] When the user confirms the prediction results, their reactions and feedback are captured by sensors and cameras and analyzed by the emotion engine. Specifically, the system identifies the user's emotional state from their facial expressions and tone of voice. The input is the user's reaction data, and the output is the analyzed emotional data.

[0703] Step 8: Emotion-based interaction

[0704] server:

[0705] The emotion engine recognizes the user's emotional state and provides appropriate notifications and information accordingly. Specifically, if the user appears anxious, it provides detailed explanations and additional supporting information, and if the user appears optimistic, it provides a concise report. The input is the analyzed emotion data, and the output is emotion-based notifications and information.

[0706] Step 9: Applying it to infrastructure management

[0707] Terminal (user terminal):

[0708] Based on the forecast results provided by the server, dam water release plans and irrigation channel maintenance schedules are adjusted. Specifically, dam operations and emergency power supply plans are formulated from terminals as needed. The input is predicted water flow data and emotion-based notifications and alerts, and the output is the execution of infrastructure management plans.

[0709] (Application example 2)

[0710] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0711] Conventional logistics management systems lacked sufficient collection and analysis of environmental and operational data, making efficient logistics management and forecasting difficult. Furthermore, they were unable to recognize employee emotions and interact with them based on those emotions, preventing improvements in employee satisfaction and work efficiency. This resulted in the issue of not optimizing the overall performance of the logistics center.

[0712] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting water flow data and environmental data, means for preprocessing the collected data, means for constructing and training a deep learning model using the preprocessed data, means for predicting water flow using the constructed model, means for visualizing and notifying the prediction results, means for managing infrastructure based on the prediction results, means for collecting and analyzing user emotion data, and means for adjusting interactions based on the user's emotional state. This enables efficient collection and analysis of environmental data and operation data at the logistics center, enabling optimal work instructions and interactions based on employee emotions.

[0713] "Water flow data" is information about the flow of water, including the flow rate, speed, direction, and so on.

[0714] "Environmental data" refers to information related to the physical environment of interest, including temperature, humidity, lighting levels, and the like.

[0715] "Preprocessing" refers to processes such as filling in missing values, removing outliers, and standardizing data in order to convert collected data into a form suitable for analysis and model learning.

[0716] A "deep learning model" is a machine learning model that uses neural networks to extract features from large amounts of data and perform tasks such as prediction and classification.

[0717] "Prediction results" refer to future states and values ​​calculated using deep learning models, and include predicted information about water flow, environmental conditions, etc.

[0718] "Visualization" refers to displaying data and forecast results in the form of easy-to-understand graphs and dashboards.

[0719] "Notification" refers to sending important information or alerts to users via email, smartphone notifications, etc.

[0720] "Infrastructure management" refers to the efficient operation and maintenance of equipment and systems at logistics centers and other facilities.

[0721] "Emotion data" is information about the user's emotional state, and is data obtained from facial expressions, tone of voice, behavior, and the like.

[0722] "Interaction" refers to the exchange of information and operations between a system and a user, and in particular refers to appropriate responses based on the user's emotions.

[0723] Data collection

[0724] The server collects environmental data (temperature, humidity, lighting level) and robot operation data (movement speed, location, remaining battery level) from each observation point within the logistics center. This data is obtained using an API and received in JSON format. The collected data is stored in a database.

[0725] Data Preprocessing

[0726] The server complements missing values ​​and removes outliers from the collected data, and also standardizes the data. Statistical methods are used to remove outliers, and data from the previous and following days is used to complement missing values. Libraries such as StandardScaler are used for standardization.

[0727] Model Building

[0728] The server uses the preprocessed data to build a deep learning model using a long short-term memory network (LSTM) or gated recurrent unit (GRU), which extracts features from past data and makes it possible to predict future environmental conditions and robot operating status with high accuracy.

[0729] Water flow prediction

[0730] The server uses the constructed model to make predictions, and the predicted data is stored in a database or displayed in real time on a dashboard. Based on the prediction results, optimal work plans within the logistics center can be created.

[0731] Result visualization and notification

[0732] The server visualizes the prediction results in graphs and dashboards, and if any abnormal prediction results are detected, it alerts users and administrators via email or smartphone notification functions.

[0733] Emotion Engine Operation

[0734] The server uses an emotion engine to recognize the user's emotions. It uses cameras and sensors to capture and analyze the user's reactions and feedback when checking the prediction results. It then identifies the user's emotional state from facial expressions and tone of voice.

[0735] Emotion-Based Interaction

[0736] The server selects appropriate visualization and notification methods based on the user's emotional state analyzed by the emotion engine. For example, if the user is feeling anxious, it provides detailed explanations and additional support information. On the other hand, if an optimistic emotion is recognized, it provides concise, positive feedback.

[0737] Infrastructure management applications

[0738] The server optimizes the robots' movements and work plans within the logistics center based on the prediction results and emotion data. For example, if a specific area is predicted to be hot, the robots' routes will be changed to avoid that area. Furthermore, when employees are under high stress, the robots will be used to reduce their workload.

[0739] Specific examples

[0740] Daily temperature, humidity, lighting levels, and robot operation data are collected via API from observation points installed in Area A. The server stores this data in a database, complements missing values, removes outliers, and standardizes the data. Next, an LSTM model is built using the preprocessed data to predict future environmental conditions in Area A. The prediction results are displayed in real time on a dashboard, and if high temperatures are predicted, an email notification is sent to the administrator. In addition, when the user checks the prediction results, their facial expressions are captured by a camera, and if they are feeling anxious, a detailed explanation is provided.

[0741] Example prompt sentence:

[0742] "Collect environmental data from the logistics center and robot operation data in real time, and use deep learning models to analyze and predict past data. At the same time, recognize employee emotions and provide optimal work instructions accordingly."

[0743] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0744] Step 1:

[0745] The server collects environmental data (temperature, humidity, lighting level) and robot operation data (movement speed, location, remaining battery power) from each observation point within the logistics center. This data is obtained through an API and received in JSON format. The input is various data from the observation points, and the output is a JSON file containing the raw data.

[0746] Step 2:

[0747] The server performs preprocessing on the collected data. It uses data from the previous and next days to fill in missing values ​​and statistical methods to remove outliers. It also standardizes the data using StandardScaler. The input is raw data in JSON format, and the output is preprocessed data.

[0748] Step 3:

[0749] The server uses the preprocessed data to build a deep learning model (LSTM or GRU). The model is built using the Keras or TensorFlow library. The input is the preprocessed data, and the output is a trained deep learning model.

[0750] Step 4:

[0751] The server uses the constructed model to predict future environmental conditions and the robot's operating status. The prediction results are stored in a database or displayed in real time on a dashboard. The input is the trained model and current data, and the output is predicted future data.

[0752] Step 5:

[0753] The server visualizes the prediction results and notifies users and administrators if any abnormal results are found. The server visualizes the results using graphs and dashboards, and issues alerts using email or smartphone notification functions. The input is the prediction results, and the output is the visualized data and notification messages.

[0754] Step 6:

[0755] The server uses an emotion engine to recognize the user's emotions. It uses cameras and sensors to capture the user's facial expressions and tone of voice, and then analyzes them to identify the user's emotional state. The input is the user's reaction data, and the output is analyzed emotional data.

[0756] Step 7:

[0757] The server adjusts the interaction based on the user's emotional state, for example, providing detailed explanations and additional support information if the user is anxious, or providing concise, positive feedback if the user is optimistic. The input is the analyzed emotional data, and the output is an optimized interaction.

[0758] Step 8:

[0759] The server optimizes work plans within the logistics center based on prediction results and emotional data. If high temperatures are predicted in a specific area, the robot's route is changed to avoid that area. Also, if an employee is under high stress, the robot is used to reduce their workload. The input is the prediction results and emotional data, and the output is an optimized work plan.

[0760] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0761] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0762] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0763] [Third embodiment]

[0764] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0765] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0766] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0767] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0768] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0769] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0770] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0771] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0772] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0773] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0774] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0775] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0776] This invention is a system that collects water flow data and environmental data, and analyzes and predicts that data using a deep learning model. This system aims to maximize the efficiency of water flow power generation and optimize infrastructure management by collecting, analyzing, predicting, notifying, and applying information, with the server, terminal, and user each playing a specific role.

[0777] Data collection

[0778] Server: The server collects water flow data and environmental data from each observation point. The data is retrieved using API and received in JSON format. This data is stored in a database.

[0779] Data Preprocessing

[0780] Server: The collected data is first preprocessed by filling in missing values ​​and removing outliers. Data standardization is also performed, converting the data into a form suitable for analysis and learning.

[0781] Model Building

[0782] Server: Using the preprocessed data, a deep learning model is built. Recurrent neural networks such as long short-term memory networks (LSTM) and gated recurrent units (GRU) are used. This allows for the extraction of features from past data and the accurate prediction of future water flow.

[0783] Water flow prediction

[0784] Server: The constructed model is used to predict water flow. The predicted data is stored in a database or displayed in real time.

[0785] Result visualization and notification

[0786] Server: The prediction results are visualized in graphs and dashboards. If abnormal water flow is predicted, an alert is sent to the user or administrator via email or smartphone notification.

[0787] Infrastructure management applications

[0788] Terminal (user terminal): Dam water release plans and irrigation channel maintenance schedules are adjusted based on the prediction results. In the event of a disaster, emergency power supply plans are formulated based on the predicted data. If necessary, operations can be performed directly from the terminal.

[0789] Specific examples

[0790] 1. Data collection from observation points

[0791] For example, daily flow rate data, temperature, precipitation, and seasonal data are collected from a river observation point via API, and the server automatically stores this information in a database.

[0792] 2. Pretreatment

[0793] The collected data may contain missing or outliers. For such data, the missing values ​​are filled in using data from the previous or following day, and outliers are removed using statistical methods. Furthermore, the data is standardized to make it easier to analyze.

[0794] 3. Training the model

[0795] Using a long short-term memory network (LSTM), the model is trained with several years of past water flow and environmental data to capture temporal patterns and trends and accurately predict future water flow.

[0796] 4. Predicting water flow

[0797] The trained model is used to predict water flow for the following week or month. The predicted data is aggregated on the server and displayed in real time on a dashboard.

[0798] 5. Visualization and Notification

[0799] If an abnormality is predicted based on the prediction results, an alert will be sent to the administrator. For example, if a high water level is predicted, a warning will be sent to the administrator by email or smartphone app.

[0800] 6. Reflection in infrastructure management

[0801] Dam managers use the predicted water flow data to create advance water release schedules, determine the optimal timing for irrigation channel construction, and plan dredging and other work.

[0802] In this way, the present invention realizes efficient water current power generation, stabilization of power supply during disasters, and optimization of infrastructure management.

[0803] The processing flow will be explained below.

[0804] Step 1:

[0805] The server collects water flow data and environmental data from each observation point. Specifically, it sends an HTTP request through the API and receives the acquired data in JSON format. The received data is then stored in a database.

[0806] Step 2:

[0807] The server preprocesses the collected data. Specifically, it performs operations such as filling in missing values, removing outliers, and standardizing the data. For example, it uses the PANDAS library to create a data frame, fills in missing values ​​with surrounding values, and removes outliers by setting a cutoff value.

[0808] Step 3:

[0809] The server uses the preprocessed data to build a deep learning model, specifically, a long short-term memory network (LSTM) or a gated recurrent unit (GRU) using the TensorFlow and Keras libraries, and trains the model using the training data.

[0810] Step 4:

[0811] The server uses the constructed model to predict future water flow, inputs test data into the trained model, and generates predicted water flow data, which is then stored in a database.

[0812] Step 5:

[0813] The server visualizes the forecast results, for example, using Matplotlib or Plotly libraries to display the predicted water flow data as graphs and charts, and the results are displayed in real time on a dashboard.

[0814] Step 6:

[0815] The server notifies users if an abnormal water flow is predicted. Specifically, if high or low water levels are detected based on the forecast data, an alert is sent to the user via email or smartphone notification.

[0816] Step 7:

[0817] The terminal (user terminal) manages infrastructure based on the prediction results. Based on the predicted data, it adjusts dam water release plans and irrigation channel maintenance schedules. For example, if high water flow is predicted, it plans to release water from the dam in advance to reduce the risk of flooding.

[0818] Step 8:

[0819] The terminal (user terminal) formulates a power supply plan for the time of a disaster. Based on the forecast data, it calculates the amount of power required in the event of a disaster and adjusts the power generation plan. This enables a stable power supply even in the event of a disaster.

[0820] Example 1

[0821] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0822] Existing hydroelectric power generation systems and infrastructure management systems do not efficiently collect, preprocess, forecast, and notify real-time data. This makes it difficult to maximize power generation efficiency and respond quickly to disasters, and limits the accuracy and effectiveness of infrastructure management. Furthermore, in building predictive models, data preprocessing and missing value imputation are often insufficient, resulting in reduced prediction accuracy. This makes it difficult for conventional systems to achieve reliable forecasts and efficient infrastructure management.

[0823] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0824] In this invention, the server includes means for collecting water flow data and environmental data, means for acquiring the collected data using an API and storing it in a database, means for preprocessing the collected data, filling in missing values, and removing outliers, means for building and training a deep learning model using the preprocessed data, means for predicting water flow using the built model, means for visualizing the prediction results in graphs or dashboards and notifying users of outliers, and means for managing infrastructure based on the prediction results. This makes it possible to efficiently preprocess collected data, perform highly accurate predictions, and visualize and notify users of the results. This enables more efficient water flow power generation, rapid disaster response, and precise infrastructure management.

[0825] "Water flow data" is information about the movement of water in bodies of water such as rivers, lakes, and dams, such as water flow rate, direction of flow, and speed.

[0826] "Environmental data" refers to information about the surrounding environment that affects water flow, such as temperature, precipitation, humidity, wind speed, and weather.

[0827] An "API (Application Programming Interface)" is a set of definitions and protocols for exchanging information between different software programs.

[0828] A "database" is a system for organizing and storing collected data and for efficiently searching, editing, and deleting it.

[0829] "Preprocessing" refers to a series of operations, such as filling in missing values, removing outliers, and standardizing data, that are performed to convert data into a form suitable for analysis and learning.

[0830] A "deep learning model" is a machine learning model based on neural networks that learns through a multi-layer network structure and has the ability to identify complex patterns.

[0831] A "long short-term memory network (LSTM)" is a type of recurrent neural network that has specialized units for efficiently learning long-term data dependencies.

[0832] A "gated recurrent unit (GRU)" is a type of recurrent neural network that has similar functionality to LSTM but has a simpler structure.

[0833] "Visualization" is the act of displaying data in a visual format, such as a graph or chart, to enable intuitive understanding of patterns and trends in the data.

[0834] "Infrastructure management" refers to the set of activities for maintaining, operating, and optimizing public facilities such as dams and irrigation channels, including efficient power supply and disaster response.

[0835] "Notification" refers to the means by which the system communicates abnormalities or important information to users and administrators, such as email or smartphone alerts.

[0836] This invention is a system that collects water flow data and environmental data, and analyzes and predicts this data using a deep learning model. This system aims to maximize the efficiency of water flow power generation and optimize infrastructure management by collecting, analyzing, predicting, notifying, and applying information, with the server, terminal, and user each playing a specific role.

[0837] The server uses an API to obtain water flow data and environmental data from observation points and receives it in JSON format. The received data is stored in a database. The collected data is then preprocessed, such as filling in missing values ​​and removing outliers. The data is also standardized and converted into a format suitable for analysis and learning.

[0838] Next, a deep learning model is constructed using the preprocessed data, using a long short-term memory network (LSTM) and a gated recurrent unit (GRU), which extracts features from past data and enables highly accurate prediction of future water flow.

[0839] The constructed model is used to predict water flow. The predicted data is then saved back into the database or displayed in real time on a dashboard. The prediction results are visualized in graphs and on a dashboard, and if an abnormal water flow is predicted, an alert is sent to the user or administrator. Notification methods include email and smartphone notifications.

[0840] Users and infrastructure managers can adjust dam water release plans and irrigation channel maintenance schedules based on the prediction results. If necessary, they can also operate the system directly from their terminals. Furthermore, in the event of a disaster, emergency power supply plans can be formulated based on the predicted data.

[0841] As a concrete example, daily flow rate data, temperature, precipitation, and seasonal data are collected from a river observation point via an API. For example, the API endpoint is "https: / / api.example.com / river_data", and this information is automatically stored in a database on the server.

[0842] Next, in the preprocessing stage, if the collected data contains missing or outliers, they are complemented or removed using previous or subsequent dates or statistical methods. All data is also standardized and organized into a form that is easy to analyze.

[0843] When using a long short-term memory network (LSTM), the network is trained with several years of water flow data and environmental data. Through this training process, a model is constructed that can accurately predict future water flow by capturing temporal patterns and trends.

[0844] Below is an example of a prompt sentence.

[0845] Example prompt sentence:

[0846] "Predict the water flow for the next month. Provide flow data, temperature data, and precipitation data for the past five years."

[0847] In this way, the present invention realizes efficient water current power generation, stabilization of power supply during disasters, and optimization of infrastructure management.

[0848] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0849] Step 1: Data collection

[0850] The server collects water flow data and environmental data from each observation point using an API. Specifically, it obtains daily flow data, temperature, precipitation, and seasonal data via the API endpoint and receives them in JSON format.

[0851] Input: API endpoint of observation point

[0852] Output: Water flow and environmental data in JSON format

[0853] Specific behavior:

[0854] The server runs the script every day at midnight and retrieves new data from the specified API endpoint "https: / / api.example.com / river_data".

[0855] The acquired data is saved in a table called "river_data" in the internal database.

[0856] Step 2: Data Preprocessing

[0857] The server performs preprocessing on the collected data, completing missing values, removing outliers, and standardizing the data.

[0858] Input: Raw water flow and environmental data stored in a database

[0859] Output: A standardized dataset with missing values ​​imputed and outliers removed

[0860] Specific behavior:

[0861] The server retrieves the raw data from the database.

[0862] For missing values, linear interpolation is performed using values ​​from the preceding and following days.

[0863] For outliers, data beyond ±3σ from the standard deviation are removed and replaced by the median.

[0864] All data is normalized to a range of 0 to 1.

[0865] Step 3: Model Building

[0866] The server uses the preprocessed data to build a deep learning model, specifically an LSTM or GRU model.

[0867] Input: Preprocessed dataset

[0868] Output: Trained deep learning model

[0869] Specific behavior:

[0870] The server defines the model using deep learning libraries such as TensorFlow and PyTorch.

[0871] The LSTM model is trained using the past five years of data as training data.

[0872] The parameters for the training process are set to 100 epochs, 32 batch size, and the Adam optimizer is used.

[0873] Step 4: Water flow prediction

[0874] Using the constructed model, the server predicts future water flow.

[0875] Input: A trained deep learning model and the latest preprocessed data

[0876] Output: Predicted future water flow data

[0877] Specific behavior:

[0878] The server inputs the latest preprocessed data into the model to predict water flow for the next month.

[0879] The prediction results are stored in the "predicted_flow" table in the database.

[0880] Step 5: Visualizing and notifying results

[0881] The server visualizes the prediction results in graphs and dashboards and notifies users of any abnormalities as needed.

[0882] Input: Predicted water flow data

[0883] Output: Visualized data display and notifications

[0884] Specific behavior:

[0885] The server uses the forecast data to create a line graph on the dashboard.

[0886] If an abnormal water flow is predicted, the administrator will be notified by email and smartphone. The subject of the email will be "High Water Level Forecast Alert," and the content will include the predicted date and time and the predicted value.

[0887] Step 6: Applying it to infrastructure management

[0888] Users and devices (user devices) use the prediction results to manage infrastructure, including adjusting dam water release plans and irrigation canal maintenance schedules.

[0889] Input: Predicted water flow data and visualized results

[0890] Output: Optimized infrastructure management plan

[0891] Specific behavior:

[0892] Users access the dashboard and check the prediction results.

[0893] Dam operators use the forecast data to adjust water release schedules and plan dredging operations.

[0894] If necessary, the dam can be operated directly from the terminal to quickly respond by releasing water in advance.

[0895] (Application example 1)

[0896] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0897] With the advancement of autonomous driving technology in recent years, there is a growing demand for autonomous vehicles to be able to respond to floods and unexpected environmental changes. However, current systems have difficulty effectively utilizing water flow and environmental data to optimize vehicle routes. This poses a challenge: there is a lack of technology to enable safe route selection during floods and efficient infrastructure management.

[0898] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0899] In this invention, the server includes means for collecting water flow data and environmental data, means for preprocessing the collected data, means for building and training a deep learning model using the preprocessed data, and means for optimizing the vehicle's driving route based on the prediction results, thereby enabling autonomous vehicles to select safe and efficient driving routes even during floods.

[0900] "Water flow data" refers to data on the flow rate, flow velocity, and flow direction of water measured in rivers, irrigation channels, etc.

[0901] "Environmental data" refers to data on weather such as temperature, precipitation, wind speed, and humidity, as well as data on the topography and land use status around rivers.

[0902] "Preprocessing" refers to data preparation processes such as filling in missing values, removing outliers, and standardizing the collected data.

[0903] A "deep learning model" is an algorithm that uses multi-layer neural networks to learn features from large amounts of data and perform tasks such as prediction and classification.

[0904] A "long short-term memory network (LSTM)" is a type of recurrent neural network (RNN) that is a neural network structure that can learn dependencies over long periods of time.

[0905] A "gated recurrent unit (GRU)" is a type of recurrent neural network (RNN), a neural network structure with a small number of parameters and high computational efficiency.

[0906] "Prediction" is the process of using a model to estimate future data.

[0907] "Visualization" is the conversion of data and forecast results into a visually understandable format such as graphs, charts, and dashboards.

[0908] "Notification" is a means of conveying warnings or information to users or administrators when certain conditions are met.

[0909] "Infrastructure management" refers to the maintenance and management of public infrastructure, including dam water release plans, irrigation canal construction schedules, and emergency power supply plans in the event of a disaster.

[0910] "Optimizing vehicle routes" means calculating the optimal route for autonomous vehicles based on water flow and environmental data, thereby achieving efficient and safe travel.

[0911] This invention is a system that optimizes the driving routes of autonomous vehicles by collecting water flow data and environmental data and analyzing and predicting them using a deep learning model. This system enables safe and efficient driving by having the server, terminal, and user each play specific roles and collect, analyze, predict, notify, and apply information.

[0912] Data collection

[0913] The server collects water flow data and environmental data from each observation point. The data collected using the API is received in JSON format and stored in a database.

[0914] Data Preprocessing

[0915] The server performs preprocessing on the collected data, such as filling in missing values ​​and removing outliers, and also standardizes the data, converting it into a form suitable for analysis and learning.

[0916] Model Building

[0917] The server uses the preprocessed data to build a deep learning model that uses long short-term memory networks (LSTM) and gated recurrent units (GRUs) to extract features from past data and predict future water flow with high accuracy.

[0918] Water flow prediction

[0919] The server uses the constructed model to predict water flow, and the predicted data is stored in a database or displayed in real time.

[0920] Result visualization and notification

[0921] The server visualizes the prediction results in graphs and dashboards. If abnormal water flow is predicted, an alert is sent to users and administrators via email, smartphone notifications, and other means.

[0922] Vehicle route optimization

[0923] The server calculates the optimal route for the autonomous vehicle based on the prediction results and transmits it to the device, allowing users to avoid the risk of flooding and travel safely and efficiently.

[0924] Specific examples

[0925] For example, daily flow rate data, temperature, precipitation, and seasonal data are collected from a river observation point via API. The server automatically stores this information in a database and performs preprocessing.

[0926] The collected data may contain missing or outliers. For such data, the missing values ​​are filled in using data from the previous or following day, and outliers are removed using statistical methods. Furthermore, the data is standardized to make it easier to analyze.

[0927] Using a long short-term memory network (LSTM), the model is trained with several years of past water flow and environmental data to capture temporal patterns and trends and accurately predict future water flow.

[0928] The trained model is used to predict water flow for the following week or month. The predicted data is aggregated on the server and displayed in real time on a dashboard.

[0929] If an abnormality is predicted based on the prediction results, an alert will be sent to the administrator. For example, if a high water level is predicted, a warning will be sent to the administrator by email or smartphone app.

[0930] Dam managers use the predicted water flow data to create advance water release schedules, determine the optimal timing for irrigation channel construction, and plan dredging and other work.

[0931] Furthermore, the system calculates the optimal route for autonomous vehicles based on the prediction results to avoid the risk of flooding. For example, if heavy rain is predicted in the Kyushu region, the system can use that information to optimize the route of autonomous vehicles in real time, making it possible to avoid areas with a high risk of flooding.

[0932] Example prompts for generative AI models

[0933] Use the following water flow and environmental data to predict the optimal path for an autonomous vehicle.

[0934] Data Format:

[0935] Water flow data (flow rate, hostel (latitude and longitude of each point), timestamp)

[0936] Environmental data (temperature, precipitation, wind speed, humidity, hostel, timestamp)

[0937] [Example of water flow data]

[0938] Flow rate: 200, Hostle: Latitude: 33.590354, Longitude: 130.401716, Time stamp: "2023-10-01T00:00:00Z"

[0939] [Example of environmental data]

[0940] Temperature: 25, Precipitation: 50, Wind Speed: 10, Humidity: 80, Hostal: Latitude: 33.590354, Longitude: 130.401716, Time Stamp: "2023-10-01T00:00:00Z"

[0941] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0942] Step 1:

[0943] The server collects water flow data and environmental data from each observation point using API. The input data includes flow rate, temperature, precipitation, wind speed, humidity, etc., and is received in JSON format. This data is stored in a database for the next processing step.

[0944] Step 2:

[0945] The server performs preprocessing on the collected data. Because the input data contains missing values ​​and outliers, the server first fills in the missing values ​​from the preceding and following data and removes the outliers using statistical methods. Next, the server standardizes the data and converts it into a format suitable for model training. The output is the preprocessed data.

[0946] Step 3:

[0947] The server uses the preprocessed data to build and train a deep learning model. This step uses long short-term memory networks (LSTM) and gated recurrent units (GRU). The input data is historical water flow and environmental data, and the model is trained based on this. The output is a trained deep learning model.

[0948] Step 4:

[0949] The server uses the trained model to predict future water flow. The input is the latest preprocessed data, on which the model makes predictions. The output is the predicted future water flow data, which is also stored in the database.

[0950] Step 5:

[0951] The server visualizes the prediction results and sends notifications as necessary. The input is the prediction results, and alerts and graphs are generated based on these. The visualized data is displayed on a dashboard, and if an abnormality is predicted, an email or smartphone notification is sent to the administrator or user. The output is the visualized data and notification information.

[0952] Step 6:

[0953] The server optimizes the autonomous vehicle's driving route based on the prediction results. The input is predicted water flow and environmental data, and the optimal driving route is calculated based on this. This calculation is performed in real time, and the calculated route information is sent to the terminal or vehicle. The output is the optimized driving route.

[0954] Keywords: generative AI model, prompt sentence

[0955] We have shown the specific processing steps of an application example.

[0956] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0957] This invention is a system that collects water flow data and environmental data, analyzes and predicts that data using a deep learning model, and combines it with an emotion engine that recognizes user emotions. This system aims to maximize the efficiency of water flow power generation and optimize infrastructure management by collecting, analyzing, predicting, notifying, and applying information, with the server, terminal, and user each playing a specific role, and further enables interaction according to the user's emotions.

[0958] Data collection

[0959] Server: The server collects water flow data and environmental data from each observation point. The data is retrieved using API and received in JSON format. This data is stored in a database.

[0960] Data Preprocessing

[0961] Server: The collected data is first preprocessed by filling in missing values ​​and removing outliers. Data standardization is also performed, converting the data into a form suitable for analysis and learning.

[0962] Model Building

[0963] Server: Using the preprocessed data, a deep learning model is built. Recurrent neural networks such as long short-term memory networks (LSTM) and gated recurrent units (GRU) are used. This allows for the extraction of features from past data and the accurate prediction of future water flow.

[0964] Water flow prediction

[0965] Server: The constructed model is used to predict water flow. The predicted data is stored in a database or displayed in real time.

[0966] Result visualization and notification

[0967] Server: The prediction results are visualized in graphs and dashboards. If abnormal water flow is predicted, an alert is sent to the user or administrator via email or smartphone notification.

[0968] Emotion Engine Operation

[0969] Server: The emotion engine recognizes the user's emotions. When the user confirms the prediction results, the emotion engine analyzes their reactions and feedback to identify the user's emotional state.

[0970] Emotion-Based Interaction

[0971] Server: The emotion engine selects appropriate visualizations and notifications based on the user's emotional state. For example, if the user is feeling anxious, it provides detailed explanations and additional support information. If optimistic emotions are recognized, it provides concise, positive feedback.

[0972] Infrastructure management applications

[0973] Terminal (user terminal): Dam water release plans and irrigation channel maintenance schedules are adjusted based on the prediction results. In the event of a disaster, emergency power supply plans are formulated based on the predicted data. If necessary, operations can be performed directly from the terminal.

[0974] Specific examples

[0975] 1. Data collection from observation points

[0976] For example, daily flow rate data, temperature, precipitation, and seasonal data are collected from a river observation point via API, and the server automatically stores this information in a database.

[0977] 2. Pretreatment

[0978] The collected data may contain missing or outliers. For such data, the missing values ​​are filled in using data from the previous or following day, and outliers are removed using statistical methods. Furthermore, the data is standardized to make it easier to analyze.

[0979] 3. Training the model

[0980] Using a long short-term memory network (LSTM), the model is trained with several years of past water flow and environmental data to capture temporal patterns and trends and accurately predict future water flow.

[0981] 4. Predicting water flow

[0982] The trained model is used to predict water flow for the following week or month. The predicted data is aggregated on the server and displayed in real time on a dashboard.

[0983] 5. Visualization and Notification

[0984] If an abnormality is predicted based on the prediction results, an alert will be sent to the administrator. For example, if a high water level is predicted, a warning will be sent to the administrator by email or smartphone app.

[0985] 6. User Emotion Recognition by Emotion Engine

[0986] The user's reaction after checking the prediction results is captured by sensors and cameras and analyzed by the emotion engine. For example, emotions such as joy, surprise, or anxiety can be identified from the user's facial expressions and tone of voice.

[0987] 7. Emotion-based interactions

[0988] The emotion engine recognizes the user's emotions and then responds accordingly: if the user appears anxious, the server can provide additional information or support links, or if the user is happy, it can simply provide a brief report.

[0989] 8. Reflection in infrastructure management

[0990] Dam managers can use the predicted water flow data to create advance water release schedules, determine the optimal timing for irrigation channel maintenance, and carry out dredging work in a planned manner. At the same time, they can use the feedback provided by the emotion engine to increase user satisfaction.

[0991] In this way, the present invention not only improves the efficiency of water current power generation, stabilizes power supply during disasters, and optimizes infrastructure management, but also improves the user experience of the entire system through interactions that are tailored to the user's emotions.

[0992] The processing flow will be explained below.

[0993] Step 1:

[0994] The server collects water flow data and environmental data from each observation point. Specifically, it sends an HTTP request through the API and retrieves the received data in JSON format. The retrieved data is then stored in a database.

[0995] Step 2:

[0996] The server preprocesses the collected data by creating a data frame, interpolating missing values ​​with previous and next values, and removing outliers using statistical methods. It also standardizes the data and converts it into a format suitable for model training.

[0997] Step 3:

[0998] The server uses the preprocessed data to build a deep learning model, specifically a neural network incorporating LSTM or GRU using TensorFlow or Keras, and trains it with the training data.

[0999] Step 4:

[1000] The server uses the constructed model to predict future water flow, inputs new data into the trained model, and generates predicted water flow data, which is then stored in a database or displayed in real time.

[1001] Step 5:

[1002] The server visualizes the forecast results, for example, using Matplotlib or Plotly to display the predicted water flow data as graphs and charts, which are then displayed in real time on a dashboard.

[1003] Step 6:

[1004] The server notifies users when abnormal water flows are predicted. Specifically, if high or low water levels are detected based on the forecast data, an alert is sent via email or smartphone notification.

[1005] Step 7:

[1006] The server recognizes the user's emotions using an emotion engine. When the user confirms the prediction results, the emotion engine analyzes their reactions and feedback to identify the user's emotional state.

[1007] Step 8:

[1008] The server provides interactions that are tailored to the user's emotional state, for example providing detailed explanations and additional support information if the user is anxious, or providing concise, positive feedback if the user is optimistic.

[1009] Step 9:

[1010] The terminal (user terminal) manages infrastructure based on the prediction results. Dam water release plans and irrigation channel maintenance schedules are adjusted based on the predicted water flow data. If necessary, a power supply plan for emergencies is also formulated.

[1011] Step 10:

[1012] The terminal (user terminal) utilizes feedback from the emotion engine. Infrastructure management is further optimized based on the user's emotions. For example, it checks whether the user feels comfortable with the proposed water discharge schedule and makes additional adjustments if necessary.

[1013] Example 2

[1014] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1015] Conventional water flow prediction systems were able to make predictions to a certain extent by analyzing collected environmental and water flow data, but no systems existed that took into account the user's emotions regarding the prediction results. As a result, they were unable to address user concerns and questions, which resulted in problems with system usability and user experience. In response to this, a new system was needed that could collect and analyze a variety of water flow and environmental data, and provide appropriate notifications and information according to the user's emotional state.

[1016] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1017] In this invention, the server includes means for collecting water flow data and environmental data, means for preprocessing the collected data, means for constructing and training a deep learning model using the preprocessed data, means for predicting water flow using the constructed model, means for visualizing and notifying the prediction results, means for managing infrastructure based on the prediction results, means for recognizing user emotions, and means for providing appropriate notifications and information based on the recognized user emotions. This enables more accurate water flow predictions and flexible responses according to the user's emotional state, resulting in improved system utilization efficiency and user experience.

[1018] "Water flow data" refers to data relating to the flow rate, flow velocity, and flow rate fluctuations of water in bodies of water such as rivers and dams.

[1019] "Environmental data" refers to meteorological data such as temperature, precipitation, wind speed, and humidity, as well as data on environmental conditions such as local land use and river water quality.

[1020] A "deep learning model" is a field of artificial intelligence that uses multi-layer neural networks to learn the characteristics of data and perform advanced analysis and predictions.

[1021] "Preprocessing" refers to the process of preparing collected raw data in a format suitable for analysis and model learning by filling in missing values, removing outliers, and standardizing the data.

[1022] A long short-term memory network (LSTM) is a type of recurrent neural network (RNN) that is an algorithm for analyzing and predicting data with temporal dependencies, and is particularly effective at learning long-term dependencies.

[1023] A "gated recurrent unit (GRU)" is a type of recurrent neural network, similar to LSTM, and is an algorithm characterized by having few parameters and high learning efficiency.

[1024] "Visualization" is the process of displaying analysis and prediction results in the form of graphs, charts, dashboards, etc., making them intuitively easy for users to understand.

[1025] "Notification" is a process in which the system communicates important information or alerts to users or administrators via email, smartphone push notifications, etc.

[1026] "Infrastructure management" is the process of determining dam discharge plans and irrigation canal maintenance schedules based on water flow forecast results, and formulating power supply plans in the event of a disaster.

[1027] The "emotion engine" is a system that analyzes the user's reactions and feedback and recognizes the user's emotional state (joy, anxiety, surprise, etc.).

[1028] "Notification and information provision based on user emotions" is the process of providing appropriate notification methods, visualization techniques, and support information according to the user's emotional state recognized by the emotion engine.

[1029] This invention is a system that collects water flow data and environmental data, and analyzes and predicts them using a deep learning model. In this system, the server, terminals, and users each play specific roles.

[1030] First, the server uses an API to collect water flow data and environmental data from each observation point. The data is received in JSON format and stored in a database. Next, the server preprocesses the collected data. Specifically, it fills in missing values, removes outliers, and standardizes the data. This converts the data into a form suitable for analysis and learning.

[1031] The server then uses the preprocessed data to build a deep learning model. It uses recurrent neural networks (RNNs) such as LSTM and GRU to extract features from past data, enabling highly accurate prediction of future water flow. The trained model is used to make predictions, and the results are stored in a database or displayed in real time.

[1032] The server visualizes the prediction results using graphs and dashboards, and if abnormal water flow is predicted, users and administrators will receive alerts via email or smartphone notifications.

[1033] Furthermore, the system is equipped with an emotion engine. When the user confirms the prediction results, the server captures their reactions and feedback using sensors and cameras, which are then analyzed by the emotion engine. The system identifies the user's emotional state from their facial expressions and tone of voice and provides appropriate notifications and information. For example, if the user is feeling anxious, the system will provide a detailed explanation and additional support information. Conversely, if the user is optimistic, it will provide a concise report.

[1034] This system aims to maximize the efficiency of water current power generation and optimize infrastructure management, and also improves the overall user experience through flexible responses based on user emotions.

[1035] Specific operation example

[1036] Data collection from observation points

[1037] The server collects daily flow rate, temperature, and precipitation data from a river observation point via API and automatically stores them in a database. This collection process is carried out using a script that automatically issues API requests at specific times.

[1038] Pre-processing

[1039] If the data obtained via the API contains missing or outliers, the server will fill in the missing parts with data from the previous or following day. Statistical methods are used to remove outliers, and the flow data is then standardized, improving the learning efficiency of the model.

[1040] Model training

[1041] The server builds a deep learning model using LSTM and GRU, and trains it on three years of water flow data and environmental data, building a model that can capture temporal patterns and trends and predict future water flow with high accuracy.

[1042] Water flow prediction and visualization

[1043] The trained model is used to predict water flow for the next week or month. The predicted data is aggregated on the server and displayed on a dashboard in real time. The prediction results are visualized in graphs and charts, allowing users to understand them intuitively.

[1044] Alert Notification

[1045] If an abnormality is predicted based on the prediction results, an alert will be sent to the administrator via email or smartphone app. For example, if a high water level is predicted, countermeasures can be taken immediately.

[1046] Emotion recognition and response using emotion engine

[1047] When the user confirms the prediction, their reaction is captured by sensors and cameras and analyzed by the emotion engine. For example, if the user appears anxious, the server will provide additional information or support links. Conversely, if the user is satisfied, it will provide a brief report.

[1048] Prompt Sentence Examples

[1049] "Please describe a system that uses a deep learning model to predict future water flow based on water flow data and environmental data. Also, explain in detail how this system is combined with an emotion engine that recognizes the user's emotions, and optimizes actions based on the prediction results depending on the user's emotional state."

[1050] When this prompt sentence is input into a generative AI model, an explanation of the system processing described above is generated.

[1051] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1052] Step 1: Collect data

[1053] server:

[1054] The server collects water flow data and environmental data from the observation points. Specifically, it obtains daily flow data, temperature, precipitation, and other data in JSON format through an API. This data is stored in a database. The input is the API response data from the observation points, and the output is the unprocessed raw data stored in the database.

[1055] Step 2: Preprocessing the data

[1056] server:

[1057] The server performs preprocessing on the collected data. Specifically, it imputes missing values ​​using data from the previous or next day and removes outliers using statistical methods. It also standardizes the data and converts it into a form suitable for analysis and model training. The input is raw data stored in a database, and the output is a preprocessed dataset.

[1058] Step 3: Building the model

[1059] server:

[1060] The server uses the preprocessed data to build a deep learning model. Specifically, it uses recurrent neural networks such as LSTM or GRU to extract features from past data. The input is the preprocessed dataset, and the output is a trained deep learning model.

[1061] Step 4: Predicting water flow

[1062] server:

[1063] The trained model is used to predict water flow. Specifically, it predicts the water flow rate for the next week or month, and the results are stored in a database or displayed in real time. The input is the trained model and newly collected preprocessed data, and the output is the predicted water flow data.

[1064] Step 5: Visualize the results

[1065] server:

[1066] The server visualizes the prediction results using graphs and dashboards. Specifically, it displays them in an easy-to-understand manner for users using line graphs, heat maps, etc. The input is the predicted water flow data, and the output is a visualized data display.

[1067] Step 6: Alert Notification

[1068] server:

[1069] If an abnormal water flow is predicted, the server sends an alert to the user or administrator via email or push notification on a smartphone. The input is the prediction result data, and the output is a notification message.

[1070] Step 7: Emotion Recognition with the Emotion Engine

[1071] server:

[1072] When the user confirms the prediction results, their reactions and feedback are captured by sensors and cameras and analyzed by the emotion engine. Specifically, the system identifies the user's emotional state from their facial expressions and tone of voice. The input is the user's reaction data, and the output is the analyzed emotional data.

[1073] Step 8: Emotion-based interaction

[1074] server:

[1075] The emotion engine recognizes the user's emotional state and provides appropriate notifications and information accordingly. Specifically, if the user appears anxious, it provides detailed explanations and additional supporting information, and if the user appears optimistic, it provides a concise report. The input is the analyzed emotion data, and the output is emotion-based notifications and information.

[1076] Step 9: Applying it to infrastructure management

[1077] Terminal (user terminal):

[1078] Based on the forecast results provided by the server, dam water release plans and irrigation channel maintenance schedules are adjusted. Specifically, dam operations and emergency power supply plans are formulated from terminals as needed. The input is predicted water flow data and emotion-based notifications and alerts, and the output is the execution of infrastructure management plans.

[1079] (Application example 2)

[1080] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1081] Conventional logistics management systems lacked sufficient collection and analysis of environmental and operational data, making efficient logistics management and forecasting difficult. Furthermore, they were unable to recognize employee emotions and interact with them based on those emotions, preventing improvements in employee satisfaction and work efficiency. This resulted in the issue of not optimizing the overall performance of the logistics center.

[1082] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting water flow data and environmental data, means for preprocessing the collected data, means for constructing and training a deep learning model using the preprocessed data, means for predicting water flow using the constructed model, means for visualizing and notifying the prediction results, means for managing infrastructure based on the prediction results, means for collecting and analyzing user emotion data, and means for adjusting interactions based on the user's emotional state. This enables efficient collection and analysis of environmental data and operation data at the logistics center, enabling optimal work instructions and interactions based on employee emotions.

[1083] "Water flow data" is information about the flow of water, including the flow rate, speed, direction, and so on.

[1084] "Environmental data" refers to information related to the physical environment of interest, including temperature, humidity, lighting levels, and the like.

[1085] "Preprocessing" refers to processes such as filling in missing values, removing outliers, and standardizing data in order to convert collected data into a form suitable for analysis and model learning.

[1086] A "deep learning model" is a machine learning model that uses neural networks to extract features from large amounts of data and perform tasks such as prediction and classification.

[1087] "Prediction results" refer to future states and values ​​calculated using deep learning models, and include predicted information about water flow, environmental conditions, etc.

[1088] "Visualization" refers to displaying data and forecast results in the form of easy-to-understand graphs and dashboards.

[1089] "Notification" refers to sending important information or alerts to users via email, smartphone notifications, etc.

[1090] "Infrastructure management" refers to the efficient operation and maintenance of equipment and systems at logistics centers and other facilities.

[1091] "Emotion data" is information about the user's emotional state, and is data obtained from facial expressions, tone of voice, behavior, and the like.

[1092] "Interaction" refers to the exchange of information and operations between a system and a user, and in particular refers to appropriate responses based on the user's emotions.

[1093] Data collection

[1094] The server collects environmental data (temperature, humidity, lighting level) and robot operation data (movement speed, location, remaining battery level) from each observation point within the logistics center. This data is obtained using an API and received in JSON format. The collected data is stored in a database.

[1095] Data Preprocessing

[1096] The server complements missing values ​​and removes outliers from the collected data, and also standardizes the data. Statistical methods are used to remove outliers, and data from the previous and following days is used to complement missing values. Libraries such as StandardScaler are used for standardization.

[1097] Model Building

[1098] The server uses the preprocessed data to build a deep learning model using a long short-term memory network (LSTM) or gated recurrent unit (GRU), which extracts features from past data and makes it possible to predict future environmental conditions and robot operating status with high accuracy.

[1099] Water flow prediction

[1100] The server uses the constructed model to make predictions, and the predicted data is stored in a database or displayed in real time on a dashboard. Based on the prediction results, optimal work plans within the logistics center can be created.

[1101] Result visualization and notification

[1102] The server visualizes the prediction results in graphs and dashboards, and if any abnormal prediction results are detected, it alerts users and administrators via email or smartphone notification functions.

[1103] Emotion Engine Operation

[1104] The server uses an emotion engine to recognize the user's emotions. It uses cameras and sensors to capture and analyze the user's reactions and feedback when checking the prediction results. It then identifies the user's emotional state from facial expressions and tone of voice.

[1105] Emotion-Based Interaction

[1106] The server selects appropriate visualization and notification methods based on the user's emotional state analyzed by the emotion engine. For example, if the user is feeling anxious, it provides detailed explanations and additional support information. On the other hand, if an optimistic emotion is recognized, it provides concise, positive feedback.

[1107] Infrastructure management applications

[1108] The server optimizes the robots' movements and work plans within the logistics center based on the prediction results and emotion data. For example, if a specific area is predicted to be hot, the robots' routes will be changed to avoid that area. Furthermore, when employees are under high stress, the robots will be used to reduce their workload.

[1109] Specific examples

[1110] Daily temperature, humidity, lighting levels, and robot operation data are collected via API from observation points installed in Area A. The server stores this data in a database, complements missing values, removes outliers, and standardizes the data. Next, an LSTM model is built using the preprocessed data to predict future environmental conditions in Area A. The prediction results are displayed in real time on a dashboard, and if high temperatures are predicted, an email notification is sent to the administrator. In addition, when the user checks the prediction results, their facial expressions are captured by a camera, and if they are feeling anxious, a detailed explanation is provided.

[1111] Example prompt sentence:

[1112] "Collect environmental data from the logistics center and robot operation data in real time, and use deep learning models to analyze and predict past data. At the same time, recognize employee emotions and provide optimal work instructions accordingly."

[1113] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1114] Step 1:

[1115] The server collects environmental data (temperature, humidity, lighting level) and robot operation data (movement speed, location, remaining battery power) from each observation point within the logistics center. This data is obtained through an API and received in JSON format. The input is various data from the observation points, and the output is a JSON file containing the raw data.

[1116] Step 2:

[1117] The server performs preprocessing on the collected data. It uses data from the previous and next days to fill in missing values ​​and statistical methods to remove outliers. It also standardizes the data using StandardScaler. The input is raw data in JSON format, and the output is preprocessed data.

[1118] Step 3:

[1119] The server uses the preprocessed data to build a deep learning model (LSTM or GRU). The model is built using the Keras or TensorFlow library. The input is the preprocessed data, and the output is a trained deep learning model.

[1120] Step 4:

[1121] The server uses the constructed model to predict future environmental conditions and the robot's operating status. The prediction results are stored in a database or displayed in real time on a dashboard. The input is the trained model and current data, and the output is predicted future data.

[1122] Step 5:

[1123] The server visualizes the prediction results and notifies users and administrators if any abnormal results are found. The server visualizes the results using graphs and dashboards, and issues alerts using email or smartphone notification functions. The input is the prediction results, and the output is the visualized data and notification messages.

[1124] Step 6:

[1125] The server uses an emotion engine to recognize the user's emotions. It uses cameras and sensors to capture the user's facial expressions and tone of voice, and then analyzes them to identify the user's emotional state. The input is the user's reaction data, and the output is analyzed emotional data.

[1126] Step 7:

[1127] The server adjusts the interaction based on the user's emotional state, for example, providing detailed explanations and additional support information if the user is anxious, or providing concise, positive feedback if the user is optimistic. The input is the analyzed emotional data, and the output is an optimized interaction.

[1128] Step 8:

[1129] The server optimizes work plans within the logistics center based on prediction results and emotional data. If high temperatures are predicted in a specific area, the robot's route is changed to avoid that area. Also, if an employee is under high stress, the robot is used to reduce their workload. The input is the prediction results and emotional data, and the output is an optimized work plan.

[1130] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1131] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1132] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1133] [Fourth embodiment]

[1134] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1135] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1136] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1137] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1138] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1139] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1140] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1141] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1142] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1143] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1144] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1145] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1146] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1147] This invention is a system that collects water flow data and environmental data, and analyzes and predicts that data using a deep learning model. This system aims to maximize the efficiency of water flow power generation and optimize infrastructure management by collecting, analyzing, predicting, notifying, and applying information, with the server, terminal, and user each playing a specific role.

[1148] Data collection

[1149] Server: The server collects water flow data and environmental data from each observation point. The data is retrieved using API and received in JSON format. This data is stored in a database.

[1150] Data Preprocessing

[1151] Server: The collected data is first preprocessed by filling in missing values ​​and removing outliers. Data standardization is also performed, converting the data into a form suitable for analysis and learning.

[1152] Model Building

[1153] Server: Using the preprocessed data, a deep learning model is built. Recurrent neural networks such as long short-term memory networks (LSTM) and gated recurrent units (GRU) are used. This allows for the extraction of features from past data and the accurate prediction of future water flow.

[1154] Water flow prediction

[1155] Server: The constructed model is used to predict water flow. The predicted data is stored in a database or displayed in real time.

[1156] Result visualization and notification

[1157] Server: The prediction results are visualized in graphs and dashboards. If abnormal water flow is predicted, an alert is sent to the user or administrator via email or smartphone notification.

[1158] Infrastructure management applications

[1159] Terminal (user terminal): Dam water release plans and irrigation channel maintenance schedules are adjusted based on the prediction results. In the event of a disaster, emergency power supply plans are formulated based on the predicted data. If necessary, operations can be performed directly from the terminal.

[1160] Specific examples

[1161] 1. Data collection from observation points

[1162] For example, daily flow rate data, temperature, precipitation, and seasonal data are collected from a river observation point via API, and the server automatically stores this information in a database.

[1163] 2. Pretreatment

[1164] The collected data may contain missing or outliers. For such data, the missing values ​​are filled in using data from the previous or following day, and outliers are removed using statistical methods. Furthermore, the data is standardized to make it easier to analyze.

[1165] 3. Training the model

[1166] Using a long short-term memory network (LSTM), the model is trained with several years of past water flow and environmental data to capture temporal patterns and trends and accurately predict future water flow.

[1167] 4. Predicting water flow

[1168] The trained model is used to predict water flow for the following week or month. The predicted data is aggregated on the server and displayed in real time on a dashboard.

[1169] 5. Visualization and Notification

[1170] If an abnormality is predicted based on the prediction results, an alert will be sent to the administrator. For example, if a high water level is predicted, a warning will be sent to the administrator by email or smartphone app.

[1171] 6. Reflection in infrastructure management

[1172] Dam managers use the predicted water flow data to create advance water release schedules, determine the optimal timing for irrigation channel construction, and plan dredging and other work.

[1173] In this way, the present invention realizes efficient water current power generation, stabilization of power supply during disasters, and optimization of infrastructure management.

[1174] The processing flow will be explained below.

[1175] Step 1:

[1176] The server collects water flow data and environmental data from each observation point. Specifically, it sends an HTTP request through the API and receives the acquired data in JSON format. The received data is then stored in a database.

[1177] Step 2:

[1178] The server preprocesses the collected data. Specifically, it performs operations such as filling in missing values, removing outliers, and standardizing the data. For example, it uses the PANDAS library to create a data frame, fills in missing values ​​with surrounding values, and removes outliers by setting a cutoff value.

[1179] Step 3:

[1180] The server uses the preprocessed data to build a deep learning model, specifically, a long short-term memory network (LSTM) or a gated recurrent unit (GRU) using the TensorFlow and Keras libraries, and trains the model using the training data.

[1181] Step 4:

[1182] The server uses the constructed model to predict future water flow, inputs test data into the trained model, and generates predicted water flow data, which is then stored in a database.

[1183] Step 5:

[1184] The server visualizes the forecast results, for example, using Matplotlib or Plotly libraries to display the predicted water flow data as graphs and charts, and the results are displayed in real time on a dashboard.

[1185] Step 6:

[1186] The server notifies users if an abnormal water flow is predicted. Specifically, if high or low water levels are detected based on the forecast data, an alert is sent to the user via email or smartphone notification.

[1187] Step 7:

[1188] The terminal (user terminal) manages infrastructure based on the prediction results. Based on the predicted data, it adjusts dam water release plans and irrigation channel maintenance schedules. For example, if high water flow is predicted, it plans to release water from the dam in advance to reduce the risk of flooding.

[1189] Step 8:

[1190] The terminal (user terminal) formulates a power supply plan for the time of a disaster. Based on the forecast data, it calculates the amount of power required in the event of a disaster and adjusts the power generation plan. This enables a stable power supply even in the event of a disaster.

[1191] Example 1

[1192] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1193] Existing hydroelectric power generation systems and infrastructure management systems do not efficiently collect, preprocess, forecast, and notify real-time data. This makes it difficult to maximize power generation efficiency and respond quickly to disasters, and limits the accuracy and effectiveness of infrastructure management. Furthermore, in building predictive models, data preprocessing and missing value imputation are often insufficient, resulting in reduced prediction accuracy. This makes it difficult for conventional systems to achieve reliable forecasts and efficient infrastructure management.

[1194] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1195] In this invention, the server includes means for collecting water flow data and environmental data, means for acquiring the collected data using an API and storing it in a database, means for preprocessing the collected data, filling in missing values, and removing outliers, means for building and training a deep learning model using the preprocessed data, means for predicting water flow using the built model, means for visualizing the prediction results in graphs or dashboards and notifying users of outliers, and means for managing infrastructure based on the prediction results. This makes it possible to efficiently preprocess collected data, perform highly accurate predictions, and visualize and notify users of the results. This enables more efficient water flow power generation, rapid disaster response, and precise infrastructure management.

[1196] "Water flow data" is information about the movement of water in bodies of water such as rivers, lakes, and dams, such as water flow rate, direction of flow, and speed.

[1197] "Environmental data" refers to information about the surrounding environment that affects water flow, such as temperature, precipitation, humidity, wind speed, and weather.

[1198] An "API (Application Programming Interface)" is a set of definitions and protocols for exchanging information between different software programs.

[1199] A "database" is a system for organizing and storing collected data and for efficiently searching, editing, and deleting it.

[1200] "Preprocessing" refers to a series of operations, such as filling in missing values, removing outliers, and standardizing data, that are performed to convert data into a form suitable for analysis and learning.

[1201] A "deep learning model" is a machine learning model based on neural networks that learns through a multi-layer network structure and has the ability to identify complex patterns.

[1202] A "long short-term memory network (LSTM)" is a type of recurrent neural network that has specialized units for efficiently learning long-term data dependencies.

[1203] A "gated recurrent unit (GRU)" is a type of recurrent neural network that has similar functionality to LSTM but has a simpler structure.

[1204] "Visualization" is the act of displaying data in a visual format, such as a graph or chart, to enable intuitive understanding of patterns and trends in the data.

[1205] "Infrastructure management" refers to the set of activities for maintaining, operating, and optimizing public facilities such as dams and irrigation channels, including efficient power supply and disaster response.

[1206] "Notification" refers to the means by which the system communicates abnormalities or important information to users and administrators, such as email or smartphone alerts.

[1207] This invention is a system that collects water flow data and environmental data, and analyzes and predicts this data using a deep learning model. This system aims to maximize the efficiency of water flow power generation and optimize infrastructure management by collecting, analyzing, predicting, notifying, and applying information, with the server, terminal, and user each playing a specific role.

[1208] The server uses an API to obtain water flow data and environmental data from observation points and receives it in JSON format. The received data is stored in a database. The collected data is then preprocessed, such as filling in missing values ​​and removing outliers. The data is also standardized and converted into a format suitable for analysis and learning.

[1209] Next, a deep learning model is constructed using the preprocessed data, using a long short-term memory network (LSTM) and a gated recurrent unit (GRU), which extracts features from past data and enables highly accurate prediction of future water flow.

[1210] The constructed model is used to predict water flow. The predicted data is then saved back into the database or displayed in real time on a dashboard. The prediction results are visualized in graphs and on a dashboard, and if an abnormal water flow is predicted, an alert is sent to the user or administrator. Notification methods include email and smartphone notifications.

[1211] Users and infrastructure managers can adjust dam water release plans and irrigation channel maintenance schedules based on the prediction results. If necessary, they can also operate the system directly from their terminals. Furthermore, in the event of a disaster, emergency power supply plans can be formulated based on the predicted data.

[1212] As a concrete example, daily flow rate data, temperature, precipitation, and seasonal data are collected from a river observation point via an API. For example, the API endpoint is "https: / / api.example.com / river_data", and this information is automatically stored in a database on the server.

[1213] Next, in the preprocessing stage, if the collected data contains missing or outliers, they are complemented or removed using previous or subsequent dates or statistical methods. All data is also standardized and organized into a form that is easy to analyze.

[1214] When using a long short-term memory network (LSTM), the network is trained with several years of water flow data and environmental data. Through this training process, a model is constructed that can accurately predict future water flow by capturing temporal patterns and trends.

[1215] Below is an example of a prompt sentence.

[1216] Example prompt sentence:

[1217] "Predict the water flow for the next month. Provide flow data, temperature data, and precipitation data for the past five years."

[1218] In this way, the present invention realizes efficient water current power generation, stabilization of power supply during disasters, and optimization of infrastructure management.

[1219] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1220] Step 1: Data collection

[1221] The server collects water flow data and environmental data from each observation point using an API. Specifically, it obtains daily flow data, temperature, precipitation, and seasonal data via the API endpoint and receives them in JSON format.

[1222] Input: API endpoint of observation point

[1223] Output: Water flow and environmental data in JSON format

[1224] Specific behavior:

[1225] The server runs the script every day at midnight and retrieves new data from the specified API endpoint "https: / / api.example.com / river_data".

[1226] The acquired data is saved in a table called "river_data" in the internal database.

[1227] Step 2: Data Preprocessing

[1228] The server performs preprocessing on the collected data, completing missing values, removing outliers, and standardizing the data.

[1229] Input: Raw water flow and environmental data stored in a database

[1230] Output: A standardized dataset with missing values ​​imputed and outliers removed

[1231] Specific behavior:

[1232] The server retrieves the raw data from the database.

[1233] For missing values, linear interpolation is performed using values ​​from the preceding and following days.

[1234] For outliers, data beyond ±3σ from the standard deviation are removed and replaced by the median.

[1235] All data is normalized to a range of 0 to 1.

[1236] Step 3: Model Building

[1237] The server uses the preprocessed data to build a deep learning model, specifically an LSTM or GRU model.

[1238] Input: Preprocessed dataset

[1239] Output: Trained deep learning model

[1240] Specific behavior:

[1241] The server defines the model using deep learning libraries such as TensorFlow and PyTorch.

[1242] The LSTM model is trained using the past five years of data as training data.

[1243] The parameters for the training process are set to 100 epochs, 32 batch size, and the Adam optimizer is used.

[1244] Step 4: Water flow prediction

[1245] Using the constructed model, the server predicts future water flow.

[1246] Input: A trained deep learning model and the latest preprocessed data

[1247] Output: Predicted future water flow data

[1248] Specific behavior:

[1249] The server inputs the latest preprocessed data into the model to predict water flow for the next month.

[1250] The prediction results are stored in the "predicted_flow" table in the database.

[1251] Step 5: Visualizing and notifying results

[1252] The server visualizes the prediction results in graphs and dashboards and notifies users of any abnormalities as needed.

[1253] Input: Predicted water flow data

[1254] Output: Visualized data display and notifications

[1255] Specific behavior:

[1256] The server uses the forecast data to create a line graph on the dashboard.

[1257] If an abnormal water flow is predicted, the administrator will be notified by email and smartphone. The subject of the email will be "High Water Level Forecast Alert," and the content will include the predicted date and time and the predicted value.

[1258] Step 6: Applying it to infrastructure management

[1259] Users and devices (user devices) use the prediction results to manage infrastructure, including adjusting dam water release plans and irrigation canal maintenance schedules.

[1260] Input: Predicted water flow data and visualized results

[1261] Output: Optimized infrastructure management plan

[1262] Specific behavior:

[1263] Users access the dashboard and check the prediction results.

[1264] Dam operators use the forecast data to adjust water release schedules and plan dredging operations.

[1265] If necessary, the dam can be operated directly from the terminal to quickly respond by releasing water in advance.

[1266] (Application example 1)

[1267] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1268] With the advancement of autonomous driving technology in recent years, there is a growing demand for autonomous vehicles to be able to respond to floods and unexpected environmental changes. However, current systems have difficulty effectively utilizing water flow and environmental data to optimize vehicle routes. This poses a challenge: there is a lack of technology to enable safe route selection during floods and efficient infrastructure management.

[1269] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1270] In this invention, the server includes means for collecting water flow data and environmental data, means for preprocessing the collected data, means for building and training a deep learning model using the preprocessed data, and means for optimizing the vehicle's driving route based on the prediction results, thereby enabling autonomous vehicles to select safe and efficient driving routes even during floods.

[1271] "Water flow data" refers to data on the flow rate, flow velocity, and flow direction of water measured in rivers, irrigation channels, etc.

[1272] "Environmental data" refers to data on weather such as temperature, precipitation, wind speed, and humidity, as well as data on the topography and land use status around rivers.

[1273] "Preprocessing" refers to data preparation processes such as filling in missing values, removing outliers, and standardizing the collected data.

[1274] A "deep learning model" is an algorithm that uses multi-layer neural networks to learn features from large amounts of data and perform tasks such as prediction and classification.

[1275] A "long short-term memory network (LSTM)" is a type of recurrent neural network (RNN) that is a neural network structure that can learn dependencies over long periods of time.

[1276] A "gated recurrent unit (GRU)" is a type of recurrent neural network (RNN), a neural network structure with a small number of parameters and high computational efficiency.

[1277] "Prediction" is the process of using a model to estimate future data.

[1278] "Visualization" is the conversion of data and forecast results into a visually understandable format such as graphs, charts, and dashboards.

[1279] "Notification" is a means of conveying warnings or information to users or administrators when certain conditions are met.

[1280] "Infrastructure management" refers to the maintenance and management of public infrastructure, including dam water release plans, irrigation canal construction schedules, and emergency power supply plans in the event of a disaster.

[1281] "Optimizing vehicle routes" means calculating the optimal route for autonomous vehicles based on water flow and environmental data, thereby achieving efficient and safe travel.

[1282] This invention is a system that optimizes the driving routes of autonomous vehicles by collecting water flow data and environmental data and analyzing and predicting them using a deep learning model. This system enables safe and efficient driving by having the server, terminal, and user each play specific roles and collect, analyze, predict, notify, and apply information.

[1283] Data collection

[1284] The server collects water flow data and environmental data from each observation point. The data collected using the API is received in JSON format and stored in a database.

[1285] Data Preprocessing

[1286] The server performs preprocessing on the collected data, such as filling in missing values ​​and removing outliers, and also standardizes the data, converting it into a form suitable for analysis and learning.

[1287] Model Building

[1288] The server uses the preprocessed data to build a deep learning model that uses long short-term memory networks (LSTM) and gated recurrent units (GRUs) to extract features from past data and predict future water flow with high accuracy.

[1289] Water flow prediction

[1290] The server uses the constructed model to predict water flow, and the predicted data is stored in a database or displayed in real time.

[1291] Result visualization and notification

[1292] The server visualizes the prediction results in graphs and dashboards. If abnormal water flow is predicted, an alert is sent to users and administrators via email, smartphone notifications, and other means.

[1293] Vehicle route optimization

[1294] The server calculates the optimal route for the autonomous vehicle based on the prediction results and transmits it to the device, allowing users to avoid the risk of flooding and travel safely and efficiently.

[1295] Specific examples

[1296] For example, daily flow rate data, temperature, precipitation, and seasonal data are collected from a river observation point via API. The server automatically stores this information in a database and performs preprocessing.

[1297] The collected data may contain missing or outliers. For such data, the missing values ​​are filled in using data from the previous or following day, and outliers are removed using statistical methods. Furthermore, the data is standardized to make it easier to analyze.

[1298] Using a long short-term memory network (LSTM), the model is trained with several years of past water flow and environmental data to capture temporal patterns and trends and accurately predict future water flow.

[1299] The trained model is used to predict water flow for the following week or month. The predicted data is aggregated on the server and displayed in real time on a dashboard.

[1300] If an abnormality is predicted based on the prediction results, an alert will be sent to the administrator. For example, if a high water level is predicted, a warning will be sent to the administrator by email or smartphone app.

[1301] Dam managers use the predicted water flow data to create advance water release schedules, determine the optimal timing for irrigation channel construction, and plan dredging and other work.

[1302] Furthermore, the system calculates the optimal route for autonomous vehicles based on the prediction results to avoid the risk of flooding. For example, if heavy rain is predicted in the Kyushu region, the system can use that information to optimize the route of autonomous vehicles in real time, making it possible to avoid areas with a high risk of flooding.

[1303] Example prompts for generative AI models

[1304] Use the following water flow and environmental data to predict the optimal path for an autonomous vehicle.

[1305] Data Format:

[1306] Water flow data (flow rate, hostel (latitude and longitude of each point), timestamp)

[1307] Environmental data (temperature, precipitation, wind speed, humidity, hostel, timestamp)

[1308] [Example of water flow data]

[1309] Flow rate: 200, Hostle: Latitude: 33.590354, Longitude: 130.401716, Time stamp: "2023-10-01T00:00:00Z"

[1310] [Example of environmental data]

[1311] Temperature: 25, Precipitation: 50, Wind Speed: 10, Humidity: 80, Hostal: Latitude: 33.590354, Longitude: 130.401716, Time Stamp: "2023-10-01T00:00:00Z"

[1312] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1313] Step 1:

[1314] The server collects water flow data and environmental data from each observation point using API. The input data includes flow rate, temperature, precipitation, wind speed, humidity, etc., and is received in JSON format. This data is stored in a database for the next processing step.

[1315] Step 2:

[1316] The server performs preprocessing on the collected data. Because the input data contains missing values ​​and outliers, the server first fills in the missing values ​​from the preceding and following data and removes the outliers using statistical methods. Next, the server standardizes the data and converts it into a format suitable for model training. The output is the preprocessed data.

[1317] Step 3:

[1318] The server uses the preprocessed data to build and train a deep learning model. This step uses long short-term memory networks (LSTM) and gated recurrent units (GRU). The input data is historical water flow and environmental data, and the model is trained based on this. The output is a trained deep learning model.

[1319] Step 4:

[1320] The server uses the trained model to predict future water flow. The input is the latest preprocessed data, on which the model makes predictions. The output is the predicted future water flow data, which is also stored in the database.

[1321] Step 5:

[1322] The server visualizes the prediction results and sends notifications as necessary. The input is the prediction results, and alerts and graphs are generated based on these. The visualized data is displayed on a dashboard, and if an abnormality is predicted, an email or smartphone notification is sent to the administrator or user. The output is the visualized data and notification information.

[1323] Step 6:

[1324] The server optimizes the autonomous vehicle's driving route based on the prediction results. The input is predicted water flow and environmental data, and the optimal driving route is calculated based on this. This calculation is performed in real time, and the calculated route information is sent to the terminal or vehicle. The output is the optimized driving route.

[1325] Keywords: generative AI model, prompt sentence

[1326] We have shown the specific processing steps of an application example.

[1327] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1328] This invention is a system that collects water flow data and environmental data, analyzes and predicts that data using a deep learning model, and combines it with an emotion engine that recognizes user emotions. This system aims to maximize the efficiency of water flow power generation and optimize infrastructure management by collecting, analyzing, predicting, notifying, and applying information, with the server, terminal, and user each playing a specific role, and further enables interaction according to the user's emotions.

[1329] Data collection

[1330] Server: The server collects water flow data and environmental data from each observation point. The data is retrieved using API and received in JSON format. This data is stored in a database.

[1331] Data Preprocessing

[1332] Server: The collected data is first preprocessed by filling in missing values ​​and removing outliers. Data standardization is also performed, converting the data into a form suitable for analysis and learning.

[1333] Model Building

[1334] Server: Using the preprocessed data, a deep learning model is built. Recurrent neural networks such as long short-term memory networks (LSTM) and gated recurrent units (GRU) are used. This allows for the extraction of features from past data and the accurate prediction of future water flow.

[1335] Water flow prediction

[1336] Server: The constructed model is used to predict water flow. The predicted data is stored in a database or displayed in real time.

[1337] Result visualization and notification

[1338] Server: The prediction results are visualized in graphs and dashboards. If abnormal water flow is predicted, an alert is sent to the user or administrator via email or smartphone notification.

[1339] Emotion Engine Operation

[1340] Server: The emotion engine recognizes the user's emotions. When the user confirms the prediction results, the emotion engine analyzes their reactions and feedback to identify the user's emotional state.

[1341] Emotion-Based Interaction

[1342] Server: The emotion engine selects appropriate visualizations and notifications based on the user's emotional state. For example, if the user is feeling anxious, it provides detailed explanations and additional support information. If optimistic emotions are recognized, it provides concise, positive feedback.

[1343] Infrastructure management applications

[1344] Terminal (user terminal): Dam water release plans and irrigation channel maintenance schedules are adjusted based on the prediction results. In the event of a disaster, emergency power supply plans are formulated based on the predicted data. If necessary, operations can be performed directly from the terminal.

[1345] Specific examples

[1346] 1. Data collection from observation points

[1347] For example, daily flow rate data, temperature, precipitation, and seasonal data are collected from a river observation point via API, and the server automatically stores this information in a database.

[1348] 2. Pretreatment

[1349] The collected data may contain missing or outliers. For such data, the missing values ​​are filled in using data from the previous or following day, and outliers are removed using statistical methods. Furthermore, the data is standardized to make it easier to analyze.

[1350] 3. Training the model

[1351] Using a long short-term memory network (LSTM), the model is trained with several years of past water flow and environmental data to capture temporal patterns and trends and accurately predict future water flow.

[1352] 4. Predicting water flow

[1353] The trained model is used to predict water flow for the following week or month. The predicted data is aggregated on the server and displayed in real time on a dashboard.

[1354] 5. Visualization and Notification

[1355] If an abnormality is predicted based on the prediction results, an alert will be sent to the administrator. For example, if a high water level is predicted, a warning will be sent to the administrator by email or smartphone app.

[1356] 6. User Emotion Recognition by Emotion Engine

[1357] The user's reaction after checking the prediction results is captured by sensors and cameras and analyzed by the emotion engine. For example, emotions such as joy, surprise, or anxiety can be identified from the user's facial expressions and tone of voice.

[1358] 7. Emotion-based interactions

[1359] The emotion engine recognizes the user's emotions and then responds accordingly: if the user appears anxious, the server can provide additional information or support links, or if the user is happy, it can simply provide a brief report.

[1360] 8. Reflection in infrastructure management

[1361] Dam managers can use the predicted water flow data to create advance water release schedules, determine the optimal timing for irrigation channel maintenance, and carry out dredging work in a planned manner. At the same time, they can use the feedback provided by the emotion engine to increase user satisfaction.

[1362] In this way, the present invention not only improves the efficiency of water current power generation, stabilizes power supply during disasters, and optimizes infrastructure management, but also improves the user experience of the entire system through interactions that are tailored to the user's emotions.

[1363] The processing flow will be explained below.

[1364] Step 1:

[1365] The server collects water flow data and environmental data from each observation point. Specifically, it sends an HTTP request through the API and retrieves the received data in JSON format. The retrieved data is then stored in a database.

[1366] Step 2:

[1367] The server preprocesses the collected data by creating a data frame, interpolating missing values ​​with previous and next values, and removing outliers using statistical methods. It also standardizes the data and converts it into a format suitable for model training.

[1368] Step 3:

[1369] The server uses the preprocessed data to build a deep learning model, specifically a neural network incorporating LSTM or GRU using TensorFlow or Keras, and trains it with the training data.

[1370] Step 4:

[1371] The server uses the constructed model to predict future water flow, inputs new data into the trained model, and generates predicted water flow data, which is then stored in a database or displayed in real time.

[1372] Step 5:

[1373] The server visualizes the forecast results, for example, using Matplotlib or Plotly to display the predicted water flow data as graphs and charts, which are then displayed in real time on a dashboard.

[1374] Step 6:

[1375] The server notifies users when abnormal water flows are predicted. Specifically, if high or low water levels are detected based on the forecast data, an alert is sent via email or smartphone notification.

[1376] Step 7:

[1377] The server recognizes the user's emotions using an emotion engine. When the user confirms the prediction results, the emotion engine analyzes their reactions and feedback to identify the user's emotional state.

[1378] Step 8:

[1379] The server provides interactions that are tailored to the user's emotional state, for example providing detailed explanations and additional support information if the user is anxious, or providing concise, positive feedback if the user is optimistic.

[1380] Step 9:

[1381] The terminal (user terminal) manages infrastructure based on the prediction results. Dam water release plans and irrigation channel maintenance schedules are adjusted based on the predicted water flow data. If necessary, a power supply plan for emergencies is also formulated.

[1382] Step 10:

[1383] The terminal (user terminal) utilizes feedback from the emotion engine. Infrastructure management is further optimized based on the user's emotions. For example, it checks whether the user feels comfortable with the proposed water discharge schedule and makes additional adjustments if necessary.

[1384] Example 2

[1385] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1386] Conventional water flow prediction systems were able to make predictions to a certain extent by analyzing collected environmental and water flow data, but no systems existed that took into account the user's emotions regarding the prediction results. As a result, they were unable to address user concerns and questions, which resulted in problems with system usability and user experience. In response to this, a new system was needed that could collect and analyze a variety of water flow and environmental data, and provide appropriate notifications and information according to the user's emotional state.

[1387] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1388] In this invention, the server includes means for collecting water flow data and environmental data, means for preprocessing the collected data, means for constructing and training a deep learning model using the preprocessed data, means for predicting water flow using the constructed model, means for visualizing and notifying the prediction results, means for managing infrastructure based on the prediction results, means for recognizing user emotions, and means for providing appropriate notifications and information based on the recognized user emotions. This enables more accurate water flow predictions and flexible responses according to the user's emotional state, resulting in improved system utilization efficiency and user experience.

[1389] "Water flow data" refers to data relating to the flow rate, flow velocity, and flow rate fluctuations of water in bodies of water such as rivers and dams.

[1390] "Environmental data" refers to meteorological data such as temperature, precipitation, wind speed, and humidity, as well as data on environmental conditions such as local land use and river water quality.

[1391] A "deep learning model" is a field of artificial intelligence that uses multi-layer neural networks to learn the characteristics of data and perform advanced analysis and predictions.

[1392] "Preprocessing" refers to the process of preparing collected raw data in a format suitable for analysis and model learning by filling in missing values, removing outliers, and standardizing the data.

[1393] A long short-term memory network (LSTM) is a type of recurrent neural network (RNN) that is an algorithm for analyzing and predicting data with temporal dependencies, and is particularly effective at learning long-term dependencies.

[1394] A "gated recurrent unit (GRU)" is a type of recurrent neural network, similar to LSTM, and is an algorithm characterized by having few parameters and high learning efficiency.

[1395] "Visualization" is the process of displaying analysis and prediction results in the form of graphs, charts, dashboards, etc., making them intuitively easy for users to understand.

[1396] "Notification" is a process in which the system communicates important information or alerts to users or administrators via email, smartphone push notifications, etc.

[1397] "Infrastructure management" is the process of determining dam discharge plans and irrigation canal maintenance schedules based on water flow forecast results, and formulating power supply plans in the event of a disaster.

[1398] The "emotion engine" is a system that analyzes the user's reactions and feedback and recognizes the user's emotional state (joy, anxiety, surprise, etc.).

[1399] "Notification and information provision based on user emotions" is the process of providing appropriate notification methods, visualization techniques, and support information according to the user's emotional state recognized by the emotion engine.

[1400] This invention is a system that collects water flow data and environmental data, and analyzes and predicts them using a deep learning model. In this system, the server, terminals, and users each play specific roles.

[1401] First, the server uses an API to collect water flow data and environmental data from each observation point. The data is received in JSON format and stored in a database. Next, the server preprocesses the collected data. Specifically, it fills in missing values, removes outliers, and standardizes the data. This converts the data into a form suitable for analysis and learning.

[1402] The server then uses the preprocessed data to build a deep learning model. It uses recurrent neural networks (RNNs) such as LSTM and GRU to extract features from past data, enabling highly accurate prediction of future water flow. The trained model is used to make predictions, and the results are stored in a database or displayed in real time.

[1403] The server visualizes the prediction results using graphs and dashboards, and if abnormal water flow is predicted, users and administrators will receive alerts via email or smartphone notifications.

[1404] Furthermore, the system is equipped with an emotion engine. When the user confirms the prediction results, the server captures their reactions and feedback using sensors and cameras, which are then analyzed by the emotion engine. The system identifies the user's emotional state from their facial expressions and tone of voice and provides appropriate notifications and information. For example, if the user is feeling anxious, the system will provide a detailed explanation and additional support information. Conversely, if the user is optimistic, it will provide a concise report.

[1405] This system aims to maximize the efficiency of water current power generation and optimize infrastructure management, and also improves the overall user experience through flexible responses based on user emotions.

[1406] Specific operation example

[1407] Data collection from observation points

[1408] The server collects daily flow rate, temperature, and precipitation data from a river observation point via API and automatically stores them in a database. This collection process is carried out using a script that automatically issues API requests at specific times.

[1409] Pre-processing

[1410] If the data obtained via the API contains missing or outliers, the server will fill in the missing parts with data from the previous or following day. Statistical methods are used to remove outliers, and the flow data is then standardized, improving the learning efficiency of the model.

[1411] Model training

[1412] The server builds a deep learning model using LSTM and GRU, and trains it on three years of water flow data and environmental data, building a model that can capture temporal patterns and trends and predict future water flow with high accuracy.

[1413] Water flow prediction and visualization

[1414] The trained model is used to predict water flow for the next week or month. The predicted data is aggregated on the server and displayed on a dashboard in real time. The prediction results are visualized in graphs and charts, allowing users to understand them intuitively.

[1415] Alert Notification

[1416] If an abnormality is predicted based on the prediction results, an alert will be sent to the administrator via email or smartphone app. For example, if a high water level is predicted, countermeasures can be taken immediately.

[1417] Emotion recognition and response using emotion engine

[1418] When the user confirms the prediction, their reaction is captured by sensors and cameras and analyzed by the emotion engine. For example, if the user appears anxious, the server will provide additional information or support links. Conversely, if the user is satisfied, it will provide a brief report.

[1419] Prompt Sentence Examples

[1420] "Please describe a system that uses a deep learning model to predict future water flow based on water flow data and environmental data. Also, explain in detail how this system is combined with an emotion engine that recognizes the user's emotions, and optimizes actions based on the prediction results depending on the user's emotional state."

[1421] When this prompt sentence is input into a generative AI model, an explanation of the system processing described above is generated.

[1422] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1423] Step 1: Collect data

[1424] server:

[1425] The server collects water flow data and environmental data from the observation points. Specifically, it obtains daily flow data, temperature, precipitation, and other data in JSON format through an API. This data is stored in a database. The input is the API response data from the observation points, and the output is the unprocessed raw data stored in the database.

[1426] Step 2: Preprocessing the data

[1427] server:

[1428] The server performs preprocessing on the collected data. Specifically, it imputes missing values ​​using data from the previous or next day and removes outliers using statistical methods. It also standardizes the data and converts it into a form suitable for analysis and model training. The input is raw data stored in a database, and the output is a preprocessed dataset.

[1429] Step 3: Building the model

[1430] server:

[1431] The server uses the preprocessed data to build a deep learning model. Specifically, it uses recurrent neural networks such as LSTM or GRU to extract features from past data. The input is the preprocessed dataset, and the output is a trained deep learning model.

[1432] Step 4: Predicting water flow

[1433] server:

[1434] The trained model is used to predict water flow. Specifically, it predicts the water flow rate for the next week or month, and the results are stored in a database or displayed in real time. The input is the trained model and newly collected preprocessed data, and the output is the predicted water flow data.

[1435] Step 5: Visualize the results

[1436] server:

[1437] The server visualizes the prediction results using graphs and dashboards. Specifically, it displays them in an easy-to-understand manner for users using line graphs, heat maps, etc. The input is the predicted water flow data, and the output is a visualized data display.

[1438] Step 6: Alert Notification

[1439] server:

[1440] If an abnormal water flow is predicted, the server sends an alert to the user or administrator via email or push notification on a smartphone. The input is the prediction result data, and the output is a notification message.

[1441] Step 7: Emotion Recognition with the Emotion Engine

[1442] server:

[1443] When the user confirms the prediction results, their reactions and feedback are captured by sensors and cameras and analyzed by the emotion engine. Specifically, the system identifies the user's emotional state from their facial expressions and tone of voice. The input is the user's reaction data, and the output is the analyzed emotional data.

[1444] Step 8: Emotion-based interaction

[1445] server:

[1446] The emotion engine recognizes the user's emotional state and provides appropriate notifications and information accordingly. Specifically, if the user appears anxious, it provides detailed explanations and additional supporting information, and if the user appears optimistic, it provides a concise report. The input is the analyzed emotion data, and the output is emotion-based notifications and information.

[1447] Step 9: Applying it to infrastructure management

[1448] Terminal (user terminal):

[1449] Based on the forecast results provided by the server, dam water release plans and irrigation channel maintenance schedules are adjusted. Specifically, dam operations and emergency power supply plans are formulated from terminals as needed. The input is predicted water flow data and emotion-based notifications and alerts, and the output is the execution of infrastructure management plans.

[1450] (Application example 2)

[1451] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1452] Conventional logistics management systems lacked sufficient collection and analysis of environmental and operational data, making efficient logistics management and forecasting difficult. Furthermore, they were unable to recognize employee emotions and interact with them based on those emotions, preventing improvements in employee satisfaction and work efficiency. This resulted in the issue of not optimizing the overall performance of the logistics center.

[1453] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting water flow data and environmental data, means for preprocessing the collected data, means for constructing and training a deep learning model using the preprocessed data, means for predicting water flow using the constructed model, means for visualizing and notifying the prediction results, means for managing infrastructure based on the prediction results, means for collecting and analyzing user emotion data, and means for adjusting interactions based on the user's emotional state. This enables efficient collection and analysis of environmental data and operation data at the logistics center, enabling optimal work instructions and interactions based on employee emotions.

[1454] "Water flow data" is information about the flow of water, including the flow rate, speed, direction, and so on.

[1455] "Environmental data" refers to information related to the physical environment of interest, including temperature, humidity, lighting levels, and the like.

[1456] "Preprocessing" refers to processes such as filling in missing values, removing outliers, and standardizing data in order to convert collected data into a form suitable for analysis and model learning.

[1457] A "deep learning model" is a machine learning model that uses neural networks to extract features from large amounts of data and perform tasks such as prediction and classification.

[1458] "Prediction results" refer to future states and values ​​calculated using deep learning models, and include predicted information about water flow, environmental conditions, etc.

[1459] "Visualization" refers to displaying data and forecast results in the form of easy-to-understand graphs and dashboards.

[1460] "Notification" refers to sending important information or alerts to users via email, smartphone notifications, etc.

[1461] "Infrastructure management" refers to the efficient operation and maintenance of equipment and systems at logistics centers and other facilities.

[1462] "Emotion data" is information about the user's emotional state, and is data obtained from facial expressions, tone of voice, behavior, and the like.

[1463] "Interaction" refers to the exchange of information and operations between a system and a user, and in particular refers to appropriate responses based on the user's emotions.

[1464] Data collection

[1465] The server collects environmental data (temperature, humidity, lighting level) and robot operation data (movement speed, location, remaining battery level) from each observation point within the logistics center. This data is obtained using an API and received in JSON format. The collected data is stored in a database.

[1466] Data Preprocessing

[1467] The server complements missing values ​​and removes outliers from the collected data, and also standardizes the data. Statistical methods are used to remove outliers, and data from the previous and following days is used to complement missing values. Libraries such as StandardScaler are used for standardization.

[1468] Model Building

[1469] The server uses the preprocessed data to build a deep learning model using a long short-term memory network (LSTM) or gated recurrent unit (GRU), which extracts features from past data and makes it possible to predict future environmental conditions and robot operating status with high accuracy.

[1470] Water flow prediction

[1471] The server uses the constructed model to make predictions, and the predicted data is stored in a database or displayed in real time on a dashboard. Based on the prediction results, optimal work plans within the logistics center can be created.

[1472] Result visualization and notification

[1473] The server visualizes the prediction results in graphs and dashboards, and if any abnormal prediction results are detected, it alerts users and administrators via email or smartphone notification functions.

[1474] Emotion Engine Operation

[1475] The server uses an emotion engine to recognize the user's emotions. It uses cameras and sensors to capture and analyze the user's reactions and feedback when checking the prediction results. It then identifies the user's emotional state from facial expressions and tone of voice.

[1476] Emotion-Based Interaction

[1477] The server selects appropriate visualization and notification methods based on the user's emotional state analyzed by the emotion engine. For example, if the user is feeling anxious, it provides detailed explanations and additional support information. On the other hand, if an optimistic emotion is recognized, it provides concise, positive feedback.

[1478] Infrastructure management applications

[1479] The server optimizes the robots' movements and work plans within the logistics center based on the prediction results and emotion data. For example, if a specific area is predicted to be hot, the robots' routes will be changed to avoid that area. Furthermore, when employees are under high stress, the robots will be used to reduce their workload.

[1480] Specific examples

[1481] Daily temperature, humidity, lighting levels, and robot operation data are collected via API from observation points installed in Area A. The server stores this data in a database, complements missing values, removes outliers, and standardizes the data. Next, an LSTM model is built using the preprocessed data to predict future environmental conditions in Area A. The prediction results are displayed in real time on a dashboard, and if high temperatures are predicted, an email notification is sent to the administrator. In addition, when the user checks the prediction results, their facial expressions are captured by a camera, and if they are feeling anxious, a detailed explanation is provided.

[1482] Example prompt sentence:

[1483] "Collect environmental data from the logistics center and robot operation data in real time, and use deep learning models to analyze and predict past data. At the same time, recognize employee emotions and provide optimal work instructions accordingly."

[1484] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1485] Step 1:

[1486] The server collects environmental data (temperature, humidity, lighting level) and robot operation data (movement speed, location, remaining battery power) from each observation point within the logistics center. This data is obtained through an API and received in JSON format. The input is various data from the observation points, and the output is a JSON file containing the raw data.

[1487] Step 2:

[1488] The server performs preprocessing on the collected data. It uses data from the previous and next days to fill in missing values ​​and statistical methods to remove outliers. It also standardizes the data using StandardScaler. The input is raw data in JSON format, and the output is preprocessed data.

[1489] Step 3:

[1490] The server uses the preprocessed data to build a deep learning model (LSTM or GRU). The model is built using the Keras or TensorFlow library. The input is the preprocessed data, and the output is a trained deep learning model.

[1491] Step 4:

[1492] The server uses the constructed model to predict future environmental conditions and the robot's operating status. The prediction results are stored in a database or displayed in real time on a dashboard. The input is the trained model and current data, and the output is predicted future data.

[1493] Step 5:

[1494] The server visualizes the prediction results and notifies users and administrators if any abnormal results are found. The server visualizes the results using graphs and dashboards, and issues alerts using email or smartphone notification functions. The input is the prediction results, and the output is the visualized data and notification messages.

[1495] Step 6:

[1496] The server uses an emotion engine to recognize the user's emotions. It uses cameras and sensors to capture the user's facial expressions and tone of voice, and then analyzes them to identify the user's emotional state. The input is the user's reaction data, and the output is analyzed emotional data.

[1497] Step 7:

[1498] The server adjusts the interaction based on the user's emotional state, for example, providing detailed explanations and additional support information if the user is anxious, or providing concise, positive feedback if the user is optimistic. The input is the analyzed emotional data, and the output is an optimized interaction.

[1499] Step 8:

[1500] The server optimizes work plans within the logistics center based on prediction results and emotional data. If high temperatures are predicted in a specific area, the robot's route is changed to avoid that area. Also, if an employee is under high stress, the robot is used to reduce their workload. The input is the prediction results and emotional data, and the output is an optimized work plan.

[1501] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1502] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1503] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1504] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1505] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1506] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1507] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1508] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1509] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1510] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1511] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1512] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1513] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1514] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1515] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1516] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1517] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1518] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1519] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1520] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1521] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1522] The following is further disclosed regarding the above embodiment.

[1523] (Claim 1)

[1524] means for collecting water flow data and environmental data;

[1525] a means for pre-processing the collected data;

[1526] A means to build and train deep learning models using preprocessed data;

[1527] A means for predicting water flow using the constructed model;

[1528] a means of visualizing and communicating the prediction results;

[1529] A means for managing infrastructure based on the prediction results;

[1530] A system including:

[1531] (Claim 2)

[1532] 2. The system of claim 1, wherein the model is constructed using a long short-term memory network (LSTM) or a gated recurrent unit (GRU).

[1533] (Claim 3)

[1534] The system of claim 1, further comprising means for performing missing value imputation and data standardization in the preprocessing stage.

[1535] "Example 1"

[1536] (Claim 1)

[1537] means for collecting water flow data and environmental data;

[1538] A means to retrieve the collected data using API and store it in a database,

[1539] A means of preprocessing the collected data to impute missing values ​​and remove outliers;

[1540] A means to build and train deep learning models using preprocessed data;

[1541] A means for predicting water flow using the constructed model;

[1542] A means to visualize forecast results in graphs and dashboards and notify of outliers,

[1543] A means for managing infrastructure based on the prediction results;

[1544] A system including:

[1545] (Claim 2)

[1546] 2. The system of claim 1, wherein the model is constructed using a long short-term memory network (LSTM) or a gated recurrent unit (GRU).

[1547] (Claim 3)

[1548] 10. The system of claim 1, further comprising means for normalizing the data in a pre-processing step.

[1549] "Application Example 1"

[1550] (Claim 1)

[1551] means for collecting water flow data and environmental data;

[1552] a means for pre-processing the collected data;

[1553] A means to build and train deep learning models using preprocessed data;

[1554] A means for predicting water flow using the constructed model;

[1555] a means of visualizing and communicating the prediction results;

[1556] A means for managing infrastructure based on the prediction results;

[1557] A means for optimizing vehicle routes based on the prediction results;

[1558] A system including:

[1559] (Claim 2)

[1560] 2. The system of claim 1, wherein the model is constructed using a long short-term memory network (LSTM) or a gated recurrent unit (GRU).

[1561] (Claim 3)

[1562] The system of claim 1, further comprising means for performing missing value imputation and data standardization in the preprocessing stage.

[1563] "Example 2: Combining Emotion Engines"

[1564] (Claim 1)

[1565] means for collecting water flow data and environmental data;

[1566] a means for pre-processing the collected data;

[1567] A means to build and train deep learning models using preprocessed data;

[1568] A means for predicting water flow using the constructed model;

[1569] a means of visualizing and communicating the prediction results;

[1570] A means for managing infrastructure based on the prediction results;

[1571] means for recognizing a user's emotion;

[1572] A means for providing appropriate notifications and information based on the recognized user's emotions;

[1573] A system including:

[1574] (Claim 2)

[1575] 2. The system of claim 1, wherein the model is constructed using a long short-term memory network (LSTM) or a gated recurrent unit (GRU).

[1576] (Claim 3)

[1577] The system of claim 1, further comprising means for performing missing value imputation and data standardization in the preprocessing stage.

[1578] "Application example 2 when combining emotion engines"

[1579] (Claim 1)

[1580] means for collecting water flow data and environmental data;

[1581] a means for pre-processing the collected data;

[1582] A means to build and train deep learning models using preprocessed data;

[1583] A means for predicting water flow using the constructed model;

[1584] a means of visualizing and communicating the prediction results;

[1585] A means for managing infrastructure based on the prediction results;

[1586] means for collecting and analyzing user emotion data;

[1587] means for adjusting interactions based on the emotional state of the user;

[1588] A system including:

[1589] (Claim 2)

[1590] 2. The system of claim 1, wherein the model is constructed using a long short-term memory network (LSTM) or a gated recurrent unit (GRU).

[1591] (Claim 3)

[1592] The system of claim 1, further comprising means for performing missing value imputation and data standardization in the preprocessing stage. [Explanation of symbols]

[1593] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for collecting water flow data and environmental data; a means for pre-processing the collected data; A means to build and train deep learning models using preprocessed data; A means for predicting water flow using the constructed model; a means of visualizing and communicating the prediction results; A means for managing infrastructure based on the prediction results; A system including:

2. 2. The system of claim 1, wherein the model is constructed using a long short-term memory network or a gated recurrent unit.

3. The system of claim 1 further comprising means for performing missing value imputation and data standardization in the preprocessing stage.

Images