system

Abstract

We provide the system. [Solution] A means of collecting remote sensor data, Means for preprocessing the collected data, A means for detecting anomalies in the natural environment by running a predictive model using preprocessed data, A means of generating and notifying warning information when an anomaly is detected, Means for visualizing the analysis results, A system that includes this.

Description

Technical Field

[0004] , ,

[0005] , , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003] [[ID=?]]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present invention is to solve the problem that in the management of the natural environment, the current monitoring technology has fragmented data collection, is difficult to detect anomalies early, and lacks prompt countermeasures after anomaly detection, resulting in the failure to achieve sustainable environmental management.

Means for Solving the Problems

[0005] Note: There seems to be an error in the original text where the tag for "Japanese Patent Application Laid - Open No. 2022 - 180282" is split into two parts in the original Japanese text. I have combined them as best as possible in translation. Also, there is an unclear tag "?". If you can provide more accurate information about those, the translation can be further refined.This invention provides a system for collecting remote sensor data, preprocessing it, and inputting it into a predictive model. This system uses machine learning algorithms to detect anomalies in the natural environment from the data in real time, and when an anomaly is detected, it quickly generates warning information and notifies relevant parties. Furthermore, by visualizing and presenting the analysis results in an easy-to-understand manner, it supports the rapid and effective formulation of natural environment management plans based on the anomaly detection results.

[0006] "Remote sensor data" refers to digital information collected by various sensors from geographically distant locations.

[0007] "Preprocessing" refers to the process of converting acquired raw data into a format or state suitable for analysis and modeling.

[0008] A "predictive model" refers to an algorithm or method used to predict future states or phenomena using past and present data.

[0009] "Anomalies in the natural environment" refers to fluctuations or changes in the natural world that deviate significantly from normal conditions or predictions.

[0010] "Warning information" refers to messages or data used to notify users when an anomaly or problem occurs.

[0011] "Visualization" refers to the technique of displaying data and information in an easy-to-understand format as diagrams and graphs.

[0012] A "machine learning algorithm" refers to a computational method used by computers to learn patterns and rules from data.

[0013] A "natural environment management plan" refers to a plan for the effective and sustainable protection and utilization of natural resources and the environment. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined. [[ID=4O]]

MODE FOR CARRYING OUT THE INVENTION

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention is a system that utilizes sensor data collected from remote locations to detect anomalies in the natural environment in real time. It is realized through the cooperation of a server, terminals, and users.

[0036] The server first periodically collects data from satellites and installed IoT sensors. This data includes environmentally relevant values ​​such as temperature, humidity, soil moisture content, and vegetation index. The server preprocesses this data into a specific format, making it ready for analysis.

[0037] Next, the server inputs the pre-processed data into a machine learning model to detect anomalies in the natural environment. This model learns from past data and has the ability to identify anomalies as new patterns. In particular, it has exceptional detection capabilities for irreversible changes such as illegal logging and forest fires.

[0038] When an anomaly is detected, the server immediately generates warning information and notifies the relevant authorities and the user's terminal. The terminal receives this information and conveys the notification to the user in an easy-to-understand manner. The visualized results are displayed as maps and graphs so that the user can quickly understand the situation.

[0039] Based on the information presented on the device, users will consider immediate countermeasures in the natural environment. They will also utilize this data in their daily natural environment management to support long-term planning. This entire process will enable the sustainable protection and management of the natural environment.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The server collects real-time environmental data from satellites and IoT sensors. This includes acquiring climate data and geographic information through data streams from remote APIs and devices.

[0043] Step 2:

[0044] The server preprocesses the collected raw data to convert it into a parseable format. This includes data cleaning, format conversion, and data imputation to ensure that the data is stored in a consistent state.

[0045] Step 3:

[0046] The server feeds pre-processed datasets into a machine learning model to perform analysis that detects anomalies in the natural environment. In this process, the model identifies anomalous patterns and phenomena and uses differential algorithms to compare them to the normal state.

[0047] Step 4:

[0048] When a machine learning model detects an anomaly, the server immediately generates an alert. This is a process that constructs detailed information including the type of anomaly, its scope of impact, and recommended immediate actions.

[0049] Step 5:

[0050] The terminal receives warning information sent from the server and visualizes it in a way that is intuitively understandable to the user. The terminal highlights anomalies on a map and displays related data in interactive graphs and other formats.

[0051] Step 6:

[0052] Users review the information provided on their devices and develop appropriate countermeasures and management plans based on that information. This assessment includes coordinating with relevant departments and rapidly allocating resources.

[0053] (Example 1)

[0054] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0055] There is a need to efficiently and accurately process natural environment data collected from remote locations and detect anomalies in real time. Furthermore, a systematic process is required to quickly share anomaly detection results and formulate and implement effective environmental conservation measures. Conventional systems have challenges in data processing efficiency and anomaly detection accuracy, making it difficult to implement immediate and long-term measures that minimize environmental impact.

[0056] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0057] In this invention, the server includes means for collecting data from a remote observation device, means for preprocessing the collected data with a data processing device, and means for detecting anomalies in the natural environment from the preprocessed data via a generative AI model. This enables rapid and accurate detection of anomalies in the natural environment, facilitating efficient information sharing and the formulation of effective environmental conservation measures.

[0058] A "remote observation device" is a device used to measure and collect data on a target environmental area in a remote location.

[0059] A "data processing device" is a device that performs preprocessing to format collected data into an analyzable format.

[0060] A "generative AI model" is a program or system that uses machine learning algorithms to learn new patterns from data and detect anomalies.

[0061] "An anomaly" refers to a state or event in the natural environment that deviates from normal conditions, and includes situations that require emergency response.

[0062] A "warning message" is notification information generated when an anomaly is detected, intended to inform relevant parties of the details.

[0063] A "data visualization device" is a device used to visually display analysis results, providing information in the form of maps, graphs, and other visual media.

[0064] "Related parties" refers to individuals or organizations involved in formulating and implementing countermeasures based on the results of anomaly detection.

[0065] This invention is a system for detecting anomalies in the natural environment in real time and taking appropriate countermeasures quickly. This system operates in cooperation with a server, terminals, and users. Specific embodiments are described below.

[0066] The server first collects natural environment data through remote monitoring equipment. This equipment includes sensors for measuring temperature, humidity, soil moisture content, vegetation index, and other parameters. Data from these sensors is transmitted to the server via the internet.

[0067] The server preprocesses the acquired data using a data processing unit. This preprocessing includes data cleaning, normalization, and missing value imputation, all of which are performed using the Python Pandas library. Data preprocessing prepares the data for machine analysis.

[0068] Next, the server inputs the pre-processed data into a generating AI model. This model is built using machine learning frameworks such as TENSORFLOW® and detects anomalies using algorithms generated based on historical data. For example, it can analyze rapidly fluctuating temperature and humidity data in a specific area to detect signs of illegal open burning.

[0069] When the server detects an anomaly, it generates a warning message and immediately notifies relevant parties and devices. Notifications are sent via email or push notifications.

[0070] The terminal receives warning messages sent from the server and displays them clearly to the user using a data visualization device. Specifically, it uses maps and graphs to show the area where the anomaly occurred and its details. This allows the user to easily understand the situation.

[0071] Based on information provided through their devices, users can quickly make real-time decisions on appropriate countermeasures in the natural environment. Furthermore, data can be used to formulate long-term natural environment conservation plans, enabling sustainable environmental management.

[0072] An example of a prompt message is, "Detect abnormal weather patterns in a specific area from the latest sensor data and predict their impact," which instructs the generated AI model to detect anomalies.

[0073] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0074] Step 1:

[0075] The server collects data on the natural environment from remote observation devices. The input is raw data transmitted from various sensors. This raw data includes information such as temperature, humidity, soil moisture content, and vegetation index. Specifically, the server periodically sends requests to retrieve data from sensors via an API. The output is the collected, unprocessed data.

[0076] Step 2:

[0077] The server preprocesses the collected raw data using a data processing device. The input is the unprocessed data collected in step 1. Data processing includes imputing missing values, removing outliers, and standardizing the data format. Specifically, the Python Pandas library is used to perform cleaning and normalization. The output is preprocessed data that is ready for analysis.

[0078] Step 3:

[0079] The server inputs preprocessed data into a generating AI model to detect anomalies. The input is the preprocessed data that is the output of step 2. Anomaly detection is performed using machine learning algorithms in the data calculation. Specifically, the TensorFlow framework is used to identify new patterns by comparing them with past data. The output is whether or not anomalies were detected and their detailed information.

[0080] Step 4:

[0081] The server generates a warning message and notifies relevant parties when an anomaly is detected. The input is the detailed information output by the anomaly detection model in step 3. Data processing organizes the information about the detected anomaly and converts it into a format for email or push notification. Specifically, the message generation system is used to create a warning that includes information about the detection location, type of anomaly, and urgency. The output is the warning message that is sent.

[0082] Step 5:

[0083] The terminal visually presents the received warning message to the user. The input is the warning message generated in step 4. Data processing extracts the message content and converts it into map and graph formats. Specifically, data visualization tools are used to highlight areas where anomalies have been detected and to display related data as a time-series graph. The output is visualized information that allows the user to immediately understand the situation.

[0084] Step 6:

[0085] The user quickly decides on countermeasures in the natural environment based on the information provided on the device. The input is the information visualized in step 5. The user assesses the situation and plans necessary field surveys and countermeasures. Specifically, they analyze the cause of the anomaly and communicate with relevant parties. The output is the countermeasures to be taken and the action plan.

[0086] (Application Example 1)

[0087] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0088] There is a need for a system that can detect abnormal events in the natural environment in real time and take appropriate countermeasures early on. Furthermore, it is necessary to support rapid decision-making by presenting the analysis results of abnormal events in a way that users can intuitively understand. In addition, there is a need to facilitate the development of environmental protection plans based on these abnormal events.

[0089] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0090] In this invention, the server includes a device for collecting remote environmental data, a device for converting the collected environmental data into an analyzable format, and a device for running a learning model using the converted environmental data to detect abnormal events. This enables real-time detection of abnormal events in the natural environment and the rapid provision of alarm information.

[0091] "Remote environmental data" refers to weather and topographic information collected using observation equipment from geographically distant locations.

[0092] "Device" refers to equipment or systems designed to achieve a specific purpose, and in this context, it refers to devices used for collecting, analyzing, and visualizing environmental data.

[0093] "Analyzable format" refers to a state in which collected data, which is not suitable for mechanical analysis in its original form, has been standardized and converted into an easily understandable format.

[0094] A "learning model" is a set of algorithms used to learn anomaly patterns based on past data and to detect anomalies from new data.

[0095] An "abnormal event" refers to a phenomenon or condition that deviates from the normal state, and in the natural environment, it particularly means unexpected weather changes or changes in topography.

[0096] "Alert information" refers to information that functions as a warning and is issued immediately when an abnormal event is detected, with the aim of promptly notifying relevant parties.

[0097] A "Geographic Information System" is a technology or system that uses digitized map data to visually analyze and display geographical information.

[0098] An "environmental protection plan" refers to a plan or strategy that includes measures and guidelines for action to minimize the impact of detected abnormal events.

[0099] The system for realizing this invention consists of a server, a terminal, and a user working together. The server first receives information from sensors that collect remote environmental data. The hardware used here includes weather sensors and image acquisition devices. This data is preprocessed within the server into a format suitable for analysis. This preprocessing involves normalizing the data and imputing missing values ​​using services such as AWS® Lambda.

[0100] The server then inputs the pre-processed data into a learning model developed via Amazon SageMaker to detect anomalies. This learning model is designed to detect new anomalies using historical data as training data. Based on the detected anomalies, the server quickly generates alarm information and sends notifications to each device using AWS SNS. This information is visualized in an intuitively understandable format on the user's device. The application installed on the device uses the Google® Maps API to display the location of the anomaly on a map and notify the user of the situation.

[0101] For example, if a rapid rise in river levels due to an approaching typhoon is detected, a warning message will appear on the user's device stating, "River levels in your area have reached dangerous levels. Please prepare to evacuate." This allows the user to take prompt and appropriate action.

[0102] Examples of prompts for the generating AI model include, "A typhoon is approaching and rising water levels are expected. What measures would be effective?" and "I would like to know about emergency response measures when there are signs of an impending earthquake."

[0103] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0104] Step 1:

[0105] The server acquires environmental data such as temperature, humidity, and river water level from remote environmental sensors. The input is raw data transmitted from the sensors, and the output is unprocessed data stored on the server.

[0106] Step 2:

[0107] The server uses AWS Lambda to preprocess the acquired raw data into a parseable format. The input is unprocessed raw data, and the output is a normalized dataset. This process includes imputation of missing values ​​and removal of outliers.

[0108] Step 3:

[0109] The server inputs a pre-processed dataset into an Amazon SageMaker trained model. The input is a formatted dataset, and the output is information about detected anomalies. The model learns from past data patterns and can detect new anomalies with high accuracy.

[0110] Step 4:

[0111] When an abnormal event is detected, the server uses AWS SNS to generate an alarm and notifies the relevant devices. The input is the abnormal event information, and the output is the alarm notification to the devices.

[0112] Step 5:

[0113] The terminal visually presents the received alarm information to the user. Using the Google Maps API, it displays the location of the anomaly and its details on a map. The input is alarm information, and the output is visual information that the user can intuitively understand.

[0114] Step 6:

[0115] Users receive information presented on their devices and select the necessary actions. For example, they might take evacuation measures or gather further information. The user's input is decision-making and action selection, while the output is the implementation of specific countermeasures.

[0116] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0117] This invention incorporates an emotion engine into a system designed for monitoring and managing the natural environment, which recognizes user emotions and optimizes responses accordingly. It operates through an integrated approach involving a server, terminal, and user.

[0118] The server continues to process environmental data collected from satellites and IoT sensors, and uses machine learning models to detect anomalies. When an anomaly is detected, the server immediately generates warning information and sends it to the terminal.

[0119] The device visualizes received warning information and notifies the user appropriately. This is where the emotion engine comes in. The device picks up emotions from the user's text input and voice and analyzes their state. For example, if the user enters a comment emphasizing urgency, the emotion engine senses that anxiety and chooses a method that emphasizes more urgent actions rather than a formal exchange.

[0120] Users consider adopting management measures for the natural environment by reviewing environmental data and anomalies displayed on their device, as well as observations presented by the emotion engine. For example, if a user is feeling anxious in a situation where concerns are heightened due to extreme weather, the system will be configured to intensify warnings and recommend immediate disaster prevention measures.

[0121] In this way, systems integrated with an emotion engine promote the sustainable management of the natural environment while enabling flexible responses that also address the emotional needs of users. This process leads to more comprehensive and human-centered environmental management.

[0122] The following describes the processing flow.

[0123] Step 1:

[0124] The server collects data about the natural environment from satellites and IoT sensors. Specifically, it retrieves climate data, soil moisture information, and other data via remote APIs.

[0125] Step 2:

[0126] The server preprocesses the collected data and arranges it into a consistent data format. This includes data cleaning, noise reduction, and format conversion.

[0127] Step 3:

[0128] The server inputs pre-processed data into a machine learning model to detect anomalies in the natural environment. Here, the model analyzes anomalous patterns in real time and makes advanced predictions.

[0129] Step 4:

[0130] If an anomaly is detected, the server generates and sends warning information to the terminal. This information includes the location and scope of the anomaly, as well as recommended countermeasures.

[0131] Step 5:

[0132] The device displays received warning information using visualization tools. Anomaly locations are highlighted on a map and details are shown in interactive graphs for intuitive user understanding.

[0133] Step 6:

[0134] The emotion engine on the device collects emotional data from user input and voice, and analyzes that emotional state. It uses natural language processing technology to analyze the emotions of comments and feedback entered by the user.

[0135] Step 7:

[0136] Based on the results of the emotion engine, the device adjusts how information is presented and displays flexible responses tailored to the user's emotions. For example, if the user is feeling stressed, it prioritizes presenting quick solutions.

[0137] Step 8:

[0138] Users review the information and recommendations presented on their devices and make environmental management decisions. A specific example is when a user decides whether or not to implement emergency measures in response to extreme weather.

[0139] (Example 2)

[0140] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0141] Systems that monitor and manage anomalies in the natural environment require flexible and adaptive responses that take user emotions into consideration. Conventional systems have focused on detecting anomalies and generating warnings, but have not adequately provided personalized responses that address users' emotional needs. This can hinder users from making appropriate decisions and taking swift action when an anomaly occurs.

[0142] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0143] In this invention, the server includes means for collecting remote information, means for preprocessing the collected information, means for executing a learning model using the preprocessed information to detect anomalies in the natural environment, means for generating and notifying warning information when an anomaly is detected, means for analyzing the user's emotions and optimizing responses, means for recommending a natural environment management plan based on the analysis results, and means for visualizing the analysis results. This enables the rapid and intuitive proposal of countermeasures for the user when an anomaly is detected, realizing flexible environmental management that responds to the user's feelings.

[0144] "Remote information" refers to digital data collected from physically distant locations, primarily obtained through sensors and communication devices.

[0145] "Preprocessing" refers to a series of steps that transform raw data into an analyzable format, including data shaping, cleansing, and filtering.

[0146] A "learning model" is an application that uses mathematical and computational methods to learn patterns and rules from data and predict events.

[0147] "Anomaly detection" is the process of identifying deviations from normal patterns or abnormal fluctuations, and is carried out through data analysis.

[0148] "Warning information" refers to notification messages generated when the system detects an anomaly, informing the user of the risks and necessary actions.

[0149] "User sentiment" is an indicator that shows the user's psychological state, and it is analyzed using natural language processing technology from text and voice input.

[0150] A "natural environment management plan" is an action plan aimed at maintaining and protecting the natural environment, and is a strategic proposal generated based on data analysis results.

[0151] "Visualization" refers to the technique of visually representing analyzed data and information to present it to users in an easy-to-understand manner, primarily in the form of graphs and charts.

[0152] This system is a technical approach to monitor the natural environment and encourage users to take appropriate management actions. It primarily relies on the collaborative functioning of three parties: the server, the terminal, and the user.

[0153] The server first collects a large amount of information about the natural environment through remote data collection devices such as satellites and IoT sensors. Raspberry Pi and Arduino are commonly used as data collection devices in this process. The server then uses Python and TensorFlow to analyze the data and run a learning model to detect anomalies. This enables the rapid detection of extreme weather events and environmental risks.

[0154] When an anomaly is detected, the server immediately generates warning information and notifies the terminal via the network. The terminal visually displays the received warning information using a GUI (Graphical User Interface) to clearly communicate it to the user. The terminal's GUI is built with JavaScript® frameworks such as React to support the user's intuitive understanding.

[0155] Furthermore, the device is equipped with an emotion engine that analyzes the user's emotions. This engine is implemented using the Hugging Face Transformers library and analyzes emotions from user input. For example, if the user inputs "I'm worried about what will happen next," the engine detects the emotion of anxiety from the text and presents appropriate countermeasures on the device.

[0156] Users can determine how to manage the natural environment based on the abnormal data and sentiment analysis results displayed on their device. The system uses a generative AI model to suggest flexible countermeasures to the user based on prompts such as, "If the system senses urgency from the comments entered by the user, please suggest appropriate follow-up."

[0157] In this way, this system enables sustainable management of the natural environment and responds quickly to the emotional needs of users, thereby achieving safer and more adaptive environmental management.

[0158] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0159] Step 1:

[0160] The server first receives environmental information from remote data collection devices. Specifically, data transmitted from sensors arrives at the server via an API. At this time, the input consists of specific environmental data such as temperature, humidity, and precipitation. This data is stored in a database, and preprocessing begins. As a result, cleansed data is output.

[0161] Step 2:

[0162] The server runs a machine learning model using preprocessed data. Specifically, it formats the data using the Pandas library and inputs it into the training model using TensorFlow. The input is a feature vector to look for potential anomalies, and the model outputs an anomaly score. Anomaly detection is performed based on this output.

[0163] Step 3:

[0164] When an anomaly is detected, the server generates warning information. This is done by packaging the anomaly information in JSON format and sending it to the terminal via a web framework such as Flask. The input is the detected anomaly data, and the output is a warning message to the user.

[0165] Step 4:

[0166] The terminal visualizes the warning information received from the server using a GUI. Here, a JavaScript framework is used to update the information in real time and display warnings to the user. The input is warning information in JSON format, and the output is a visual warning display that the user can see.

[0167] Step 5:

[0168] The device analyzes emotions from the user's text and voice input. The emotion analysis engine uses the Hugging Face library to generate emotion scores (e.g., anxiety, reassurance, etc.) from the input text. The input is the user's natural language comments, and the output is the emotion score.

[0169] Step 6:

[0170] Based on the anomaly information and sentiment analysis results displayed on the device, users can select the necessary countermeasures. The system makes suggestions such as "provide information on evacuation shelters" or "present risk mitigation measures." A prompt message such as "Please provide guidelines for taking swift action in an emergency" is generated. Based on this prompt, the AI ​​model operates and outputs recommendations appropriate to the situation.

[0171] (Application Example 2)

[0172] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0173] Monitoring and managing the natural environment requires appropriate anomaly detection and rapid response, but conventional systems do not consider the user's emotional state, which can lead to delays in necessary warnings and actions. Furthermore, when such systems are applied to the security field, they may lack useful information for the user. Therefore, there is a need for a system that can dynamically recognize user emotions and respond flexibly according to the situation and emotions.

[0174] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0175] In this invention, the server includes means for collecting remote sensor data, means for preprocessing the collected data, means for running a predictive model using the preprocessed data to detect anomalies in the natural environment, means for recognizing the user's emotional state and optimizing responses according to that state, and means for highlighting warnings based on the user's emotions regarding security situations and guiding appropriate countermeasures. This enables comprehensive and human-centered management of anomalies in the natural environment and security challenges, and the rapid provision of necessary information to the user.

[0176] "Means of collecting remote sensor data" refers to technologies that use sensor networks to monitor the natural environment and conditions within facilities and acquire necessary data.

[0177] "Preprocessing means" refers to techniques that perform the process of formatting collected raw data into a format that can be analyzed for anomaly detection.

[0178] "Means for executing predictive models and detecting anomalies in the natural environment" refers to technologies that use machine learning algorithms to perform pattern recognition on data and identify anomalous events.

[0179] "Means for recognizing a user's emotional state" refers to technologies that analyze text and voice input from a user to accurately grasp their emotions at any given moment.

[0180] "Means for optimizing responses based on emotions" refers to technologies that include a process of adjusting notification content and suggestions based on recognized emotions.

[0181] "Means of emphasizing warnings and guiding appropriate countermeasures" refers to technologies that, when an anomaly occurs, communicate information with an appropriate level of urgency that matches the user's emotions and suggest necessary actions.

[0182] The server collects data from remote sensors and preprocesses the collected raw data using Python for efficient processing. The preprocessed data is analyzed by machine learning algorithms using TensorFlow to detect anomalies in the natural environment and security conditions. Detected anomalies are monitored in real time, and different levels of warning information are generated and notified to the user's device as visual information.

[0183] The device can acquire user text input and voice data and use IBM Watson® or Google Cloud Natural Language API to recognize emotional states. This allows for the selection of appropriate responses based on the user's emotions. For example, if a user is feeling anxious, the urgency of warnings can be emphasized and suggestions can be displayed quickly.

[0184] As a concrete example, if a security manager at a facility receives a text message saying "smoke is visible" during a routine patrol, and the system determines it to be "dangerous," an optimized fire alarm and evacuation order are immediately sent to the manager. In this way, the system provides an efficient and human-centered response. An example of a prompt using a generative AI model is, "A thunderstorm is approaching, please temporarily stop the playground equipment for safety reasons." Based on this prompt, users can take appropriate action quickly and systematically.

[0185] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0186] Step 1:

[0187] The server collects data from remote sensors. The input is the raw data transmitted from each sensor, which includes measured environmental information. The output is this data stored in a temporary file. Specifically at this stage, real-time data from various sensors is received and saved to a database.

[0188] Step 2:

[0189] The raw data collected by the server is preprocessed to make it easier to process. The input is the raw data collected in step 1, and the output is the cleaned, preprocessed data. Specifically, operations such as denoising the data, correcting outliers, and scaling the data are performed.

[0190] Step 3:

[0191] The server inputs pre-processed data into a machine learning algorithm to detect anomalies. The input is pre-processed data, and the output is the anomaly detection result. The server runs an anomaly detection model using TensorFlow, analyzing patterns in the data to identify anomalous situations.

[0192] Step 4:

[0193] If an anomaly is detected, the server generates warning information about the anomaly and sends it to the terminal. The input is the anomaly detection result from step 3, and the output is the warning information notified to the user. Specifically, an appropriate message is generated based on the severity of the anomaly and sent to the user interface.

[0194] Step 5:

[0195] The system visualizes the warning information received by the device and notifies the user. The input is the warning information, and the output is a visually displayed alert for the user. The device displays a warning icon or text message on the screen so that the user can immediately check it.

[0196] Step 6:

[0197] The device recognizes the user's emotional state based on their input. The input is text or voice data entered by the user, and the output is the recognized emotional state. The device uses IBM Watson or the Google Cloud Natural Language API to analyze the user's emotions from their written or spoken text.

[0198] Step 7:

[0199] Based on the user's emotions, the device optimizes its response. Input is the user's emotional state and warning information, while output is a notification tailored to those emotions. For example, if anxiety is detected, the warning display is emphasized, and the user is given more specific instructions.

[0200] Step 8:

[0201] Based on server and terminal information, the system presents actionable steps for the user. Input consists of warning information and sentiment-based optimization results, while output is specific action suggestions. Instructions are presented as prompts using a generative AI model, enabling users to take swift action based on the suggestions received.

[0202] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0203] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search)<url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0204] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0205] [Second Embodiment]

[0206] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0207] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0208] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0209] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0210] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0211] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0212] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0213] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0214] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0215] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0216] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0217] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0218] This invention is a system that utilizes sensor data collected from remote locations to detect anomalies in the natural environment in real time. It is realized through the cooperation of a server, terminals, and users.

[0219] The server first periodically collects data from satellites and installed IoT sensors. This data includes environmentally relevant values ​​such as temperature, humidity, soil moisture content, and vegetation index. The server preprocesses this data into a specific format, making it ready for analysis.

[0220] Next, the server inputs the pre-processed data into a machine learning model to detect anomalies in the natural environment. This model learns from past data and has the ability to identify anomalies as new patterns. In particular, it has exceptional detection capabilities for irreversible changes such as illegal logging and forest fires.

[0221] When an anomaly is detected, the server immediately generates warning information and notifies the relevant authorities and the user's terminal. The terminal receives this information and conveys the notification to the user in an easy-to-understand manner. The visualized results are displayed as maps and graphs so that the user can quickly understand the situation.

[0222] Based on the information presented on the device, users will consider immediate countermeasures in the natural environment. They will also utilize this data in their daily natural environment management to support long-term planning. This entire process will enable the sustainable protection and management of the natural environment.

[0223] The following describes the processing flow.

[0224] Step 1:

[0225] The server collects real-time environmental data from satellites and IoT sensors. This includes acquiring climate data and geographic information through data streams from remote APIs and devices.

[0226] Step 2:

[0227] The server preprocesses the collected raw data to convert it into a parseable format. This includes data cleaning, format conversion, and data imputation to ensure that the data is stored in a consistent state.

[0228] Step 3:

[0229] The server feeds pre-processed datasets into a machine learning model to perform analysis that detects anomalies in the natural environment. In this process, the model identifies anomalous patterns and phenomena and uses differential algorithms to compare them to the normal state.

[0230] Step 4:

[0231] When a machine learning model detects an anomaly, the server immediately generates an alert. This is a process that constructs detailed information including the type of anomaly, its scope of impact, and recommended immediate actions.

[0232] Step 5:

[0233] The terminal receives warning information sent from the server and visualizes it in a way that is intuitively understandable to the user. The terminal highlights anomalies on a map and displays related data in interactive graphs and other formats.

[0234] Step 6:

[0235] Users review the information provided on their devices and develop appropriate countermeasures and management plans based on that information. This assessment includes coordinating with relevant departments and rapidly allocating resources.

[0236] (Example 1)

[0237] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0238] There is a need to efficiently and accurately process natural environment data collected from remote locations and detect anomalies in real time. Furthermore, a systematic process is required to quickly share anomaly detection results and formulate and implement effective environmental conservation measures. Conventional systems have challenges in data processing efficiency and anomaly detection accuracy, making it difficult to implement immediate and long-term measures that minimize environmental impact.

[0239] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0240] In this invention, the server includes means for collecting data from a remote observation device, means for preprocessing the collected data with a data processing device, and means for detecting anomalies in the natural environment from the preprocessed data via a generative AI model. This enables rapid and accurate detection of anomalies in the natural environment, facilitating efficient information sharing and the formulation of effective environmental conservation measures.

[0241] A "remote observation device" is a device used to measure and collect data on a target environmental area in a remote location.

[0242] A "data processing device" is a device that performs preprocessing to format collected data into an analyzable format.

[0243] A "generative AI model" is a program or system that uses machine learning algorithms to learn new patterns from data and detect anomalies.

[0244] "An anomaly" refers to a state or event in the natural environment that deviates from normal conditions, and includes situations that require emergency response.

[0245] A "warning message" is notification information generated when an anomaly is detected, intended to inform relevant parties of the details.

[0246] A "data visualization device" is a device used to visually display analysis results, providing information in the form of maps, graphs, and other visual media.

[0247] "Related parties" refers to individuals or organizations involved in formulating and implementing countermeasures based on the results of anomaly detection.

[0248] This invention is a system for detecting anomalies in the natural environment in real time and taking appropriate countermeasures quickly. This system operates in cooperation with a server, terminals, and users. Specific embodiments are described below.

[0249] The server first collects natural environment data through remote monitoring equipment. This equipment includes sensors for measuring temperature, humidity, soil moisture content, vegetation index, and other parameters. Data from these sensors is transmitted to the server via the internet.

[0250] The server preprocesses the acquired data using a data processing unit. This preprocessing includes data cleaning, normalization, and missing value imputation, all of which are performed using the Python Pandas library. Data preprocessing prepares the data for machine analysis.

[0251] Next, the server inputs the pre-processed data into a generating AI model. This model is built using machine learning frameworks such as TensorFlow and detects anomalies using algorithms generated based on historical data. For example, it can analyze rapidly fluctuating temperature and humidity data in a specific area to detect signs of illegal burning.

[0252] When the server detects an anomaly, it generates a warning message and immediately notifies relevant parties and devices. Notifications are sent via email or push notifications.

[0253] The terminal receives warning messages sent from the server and displays them clearly to the user using a data visualization device. Specifically, it uses maps and graphs to show the area where the anomaly occurred and its details. This allows the user to easily understand the situation.

[0254] Based on information provided through their devices, users can quickly make real-time decisions on appropriate countermeasures in the natural environment. Furthermore, data can be used to formulate long-term natural environment conservation plans, enabling sustainable environmental management.

[0255] An example of a prompt message is, "Detect abnormal weather patterns in a specific area from the latest sensor data and predict their impact," which instructs the generated AI model to detect anomalies.

[0256] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0257] Step 1:

[0258] The server collects data on the natural environment from remote observation devices. The input is raw data transmitted from various sensors. This raw data includes information such as temperature, humidity, soil moisture content, and vegetation index. Specifically, the server periodically sends requests to retrieve data from sensors via an API. The output is the collected, unprocessed data.

[0259] Step 2:

[0260] The server preprocesses the collected raw data using a data processing device. The input is the unprocessed data collected in step 1. Data processing includes imputing missing values, removing outliers, and standardizing the data format. Specifically, the Python Pandas library is used to perform cleaning and normalization. The output is preprocessed data that is ready for analysis.

[0261] Step 3:

[0262] The server inputs preprocessed data into a generating AI model to detect anomalies. The input is the preprocessed data that is the output of step 2. Anomaly detection is performed using machine learning algorithms in the data calculation. Specifically, the TensorFlow framework is used to identify new patterns by comparing them with past data. The output is whether or not anomalies were detected and their detailed information.

[0263] Step 4:

[0264] The server generates a warning message and notifies relevant parties when an anomaly is detected. The input is the detailed information output by the anomaly detection model in step 3. Data processing organizes the information about the detected anomaly and converts it into a format for email or push notification. Specifically, the message generation system is used to create a warning that includes information about the detection location, type of anomaly, and urgency. The output is the warning message that is sent.

[0265] Step 5:

[0266] The terminal visually presents the received warning message to the user. The input is the warning message generated in step 4. Data processing extracts the message content and converts it into map and graph formats. Specifically, data visualization tools are used to highlight areas where anomalies have been detected and to display related data as a time-series graph. The output is visualized information that allows the user to immediately understand the situation.

[0267] Step 6:

[0268] The user quickly decides on countermeasures in the natural environment based on the information provided on the device. The input is the information visualized in step 5. The user assesses the situation and plans necessary field surveys and countermeasures. Specifically, they analyze the cause of the anomaly and communicate with relevant parties. The output is the countermeasures to be taken and the action plan.

[0269] (Application Example 1)

[0270] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0271] There is a need for a system that can detect abnormal events in the natural environment in real time and take appropriate countermeasures early on. Furthermore, it is necessary to support rapid decision-making by presenting the analysis results of abnormal events in a way that users can intuitively understand. In addition, there is a need to facilitate the development of environmental protection plans based on these abnormal events.

[0272] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0273] In this invention, the server includes a device for collecting remote environmental data, a device for converting the collected environmental data into an analyzable format, and a device for running a learning model using the converted environmental data to detect abnormal events. This enables real-time detection of abnormal events in the natural environment and the rapid provision of alarm information.

[0274] "Remote environmental data" refers to weather and topographic information collected using observation equipment from geographically distant locations.

[0275] "Device" refers to equipment or systems designed to achieve a specific purpose, and in this context, it refers to devices used for collecting, analyzing, and visualizing environmental data.

[0276] "Analyzable format" refers to a state in which collected data, which is not suitable for mechanical analysis in its original form, has been standardized and converted into an easily understandable format.

[0277] A "learning model" is a set of algorithms used to learn anomaly patterns based on past data and to detect anomalies from new data.

[0278] An "abnormal event" refers to a phenomenon or condition that deviates from the normal state, and in the natural environment, it particularly means unexpected weather changes or changes in topography.

[0279] "Alert information" refers to information that functions as a warning and is issued immediately when an abnormal event is detected, with the aim of promptly notifying relevant parties.

[0280] A "Geographic Information System" is a technology or system that uses digitized map data to visually analyze and display geographical information.

[0281] An "environmental protection plan" refers to a plan or strategy that includes measures and guidelines for action to minimize the impact of detected abnormal events.

[0282] The system for realizing this invention consists of a server, a terminal, and a user working together. The server first receives information from sensors that collect remote environmental data. The hardware used here includes weather sensors and image acquisition devices. This data is preprocessed within the server into a format suitable for analysis. This preprocessing involves normalizing the data and imputing missing values ​​using services such as AWS Lambda.

[0283] The server then inputs the pre-processed data into a learning model developed via Amazon SageMaker to detect anomalies. This learning model is designed to detect new anomalies using historical data as training data. Based on the detected anomalies, the server quickly generates alarm information and sends notifications to each device using AWS SNS. This information is visualized in an intuitively understandable format on the user's device. The application installed on the device uses the Google Maps API to display the location of the anomaly on a map and notifies the user of the situation.

[0284] As a specific example, when a rapid rise in the river level due to the approach of a typhoon is detected, a warning "The river level in your area has reached a dangerous level. Please prepare for evacuation." is displayed on the user's terminal. This enables the user to take prompt and appropriate actions.

[0285] Examples of prompt texts for the generative AI model include "A typhoon is approaching and a rise in the water level is expected. What effective countermeasures are there?" and "I want to know the emergency response measures in case of earthquake precursors."

[0286] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0287] Step 1:

[0288] The server acquires environmental data such as temperature, humidity, and river level from remote environmental sensors. The input is the raw data transmitted from the sensors, and the output is the unprocessed data stored in the server.

[0289] Step 2:

[0290] The server uses AWS Lambda to preprocess the acquired raw data into an analyzable format. The input is the unprocessed raw data, and the output is a normalized dataset. In this process, missing value imputation and outlier exclusion are performed.

[0291] Step 3:

[0292] The server inputs the preprocessed dataset into the learning model of Amazon SageMaker. The input is the formatted dataset, and the output is information regarding the detected abnormal events. The model has learned past data patterns and can detect new anomalies with high accuracy.

[0293] Step 4:

[0294] When an abnormal event is detected, the server uses AWS SNS to generate an alarm and notifies the relevant devices. The input is the abnormal event information, and the output is the alarm notification to the devices.

[0295] Step 5:

[0296] The terminal visually presents the received alarm information to the user. Using the Google Maps API, it displays the location of the anomaly and its details on a map. The input is alarm information, and the output is visual information that the user can intuitively understand.

[0297] Step 6:

[0298] Users receive information presented on their devices and select the necessary actions. For example, they might take evacuation measures or gather further information. The user's input is decision-making and action selection, while the output is the implementation of specific countermeasures.

[0299] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0300] This invention incorporates an emotion engine into a system designed for monitoring and managing the natural environment, which recognizes user emotions and optimizes responses accordingly. It operates through an integrated approach involving a server, terminal, and user.

[0301] The server continues to process environmental data collected from satellites and IoT sensors, and uses machine learning models to detect anomalies. When an anomaly is detected, the server immediately generates warning information and sends it to the terminal.

[0302] The terminal visualizes the received warning information and appropriately notifies the user. Here, what newly joins is the emotion engine. The terminal picks up the emotion from the user's text input and voice, and analyzes that state. For example, when the user inputs a comment emphasizing urgency, the emotion engine senses the anxiety and selects a way to emphasize a more urgent countermeasure rather than a formal way of speaking.

[0303] In addition to the environmental data and anomalies displayed on the terminal, the user checks the findings presented by the emotion engine and considers adopting management of the natural environment. For example, in a situation where concerns are increasing due to abnormal weather and the user is feeling anxious, the system is set to strengthen the warning and recommend immediate disaster prevention measures.

[0304] In this way, the system integrated with the emotion engine enables flexible responses according to the user's emotional needs while promoting sustainable management of the natural environment. Through this process, more comprehensive and user-centered environmental management is realized.

[0305] The following explains the processing flow.

[0306] Step 1:

[0307] The server collects data on the natural environment from satellites and IoT sensors. Specifically, it performs operations to obtain climate data, soil humidity information, etc. through a remote API.

[0308] Step 2:

[0309] The server preprocesses the collected data and arranges it into a consistent data format. This includes data cleaning, noise removal, format conversion, etc.

[0310] Step 3:

[0311] The server inputs pre-processed data into a machine learning model to detect anomalies in the natural environment. Here, the model analyzes anomalous patterns in real time and makes advanced predictions.

[0312] Step 4:

[0313] If an anomaly is detected, the server generates and sends warning information to the terminal. This information includes the location and scope of the anomaly, as well as recommended countermeasures.

[0314] Step 5:

[0315] The device displays received warning information using visualization tools. Anomaly locations are highlighted on a map and details are shown in interactive graphs for intuitive user understanding.

[0316] Step 6:

[0317] The emotion engine on the device collects emotional data from user input and voice, and analyzes that emotional state. It uses natural language processing technology to analyze the emotions of comments and feedback entered by the user.

[0318] Step 7:

[0319] Based on the results of the emotion engine, the device adjusts how information is presented and displays flexible responses tailored to the user's emotions. For example, if the user is feeling stressed, it prioritizes presenting quick solutions.

[0320] Step 8:

[0321] Users review the information and recommendations presented on their devices and make environmental management decisions. A specific example is when a user decides whether or not to implement emergency measures in response to extreme weather.

[0322] (Example 2)

[0323] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0324] Systems that monitor and manage anomalies in the natural environment require flexible and adaptive responses that take user emotions into consideration. Conventional systems have focused on detecting anomalies and generating warnings, but have not adequately provided personalized responses that address users' emotional needs. This can hinder users from making appropriate decisions and taking swift action when an anomaly occurs.

[0325] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0326] In this invention, the server includes means for collecting remote information, means for preprocessing the collected information, means for executing a learning model using the preprocessed information to detect anomalies in the natural environment, means for generating and notifying warning information when an anomaly is detected, means for analyzing the user's emotions and optimizing responses, means for recommending a natural environment management plan based on the analysis results, and means for visualizing the analysis results. This enables the rapid and intuitive proposal of countermeasures for the user when an anomaly is detected, realizing flexible environmental management that responds to the user's feelings.

[0327] "Remote information" refers to digital data collected from physically distant locations, primarily obtained through sensors and communication devices.

[0328] "Preprocessing" refers to a series of steps that transform raw data into an analyzable format, including data shaping, cleansing, and filtering.

[0329] A "learning model" is an application that uses mathematical and computational methods to learn patterns and rules from data and predict events.

[0330] "Anomaly detection" is the process of identifying deviations from normal patterns or abnormal fluctuations, and is carried out through data analysis.

[0331] "Warning information" refers to notification messages generated when the system detects an anomaly, informing the user of the risks and necessary actions.

[0332] "User sentiment" is an indicator that shows the user's psychological state, and it is analyzed using natural language processing technology from text and voice input.

[0333] A "natural environment management plan" is an action plan aimed at maintaining and protecting the natural environment, and is a strategic proposal generated based on data analysis results.

[0334] "Visualization" refers to the technique of visually representing analyzed data and information to present it to users in an easy-to-understand manner, primarily in the form of graphs and charts.

[0335] This system is a technical approach to monitor the natural environment and encourage users to take appropriate management actions. It primarily relies on the collaborative functioning of three parties: the server, the terminal, and the user.

[0336] The server first collects a large amount of information about the natural environment through remote data collection devices such as satellites and IoT sensors. Raspberry Pi and Arduino are commonly used as data collection devices in this process. The server then uses Python and TensorFlow to analyze the data and run a learning model to detect anomalies. This enables the rapid detection of extreme weather events and environmental risks.

[0337] When an anomaly is detected, the server immediately generates warning information and notifies the terminal via the network. The terminal visually displays the received warning information using a GUI (Graphical User Interface) to clearly communicate it to the user. The terminal's GUI is built with a JavaScript framework such as React to support the user's intuitive understanding.

[0338] Furthermore, the device is equipped with an emotion engine that analyzes the user's emotions. This engine is implemented using the Hugging Face Transformers library and analyzes emotions from user input. For example, if the user inputs "I'm worried about what will happen next," the engine detects the emotion of anxiety from the text and presents appropriate countermeasures on the device.

[0339] Users can determine how to manage the natural environment based on the abnormal data and sentiment analysis results displayed on their device. The system uses a generative AI model to suggest flexible countermeasures to the user based on prompts such as, "If the system senses urgency from the comments entered by the user, please suggest appropriate follow-up."

[0340] In this way, this system enables sustainable management of the natural environment and responds quickly to the emotional needs of users, thereby achieving safer and more adaptive environmental management.

[0341] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0342] Step 1:

[0343] The server first receives environmental information from remote data collection devices. Specifically, data transmitted from sensors arrives at the server via an API. At this time, the input consists of specific environmental data such as temperature, humidity, and precipitation. This data is stored in a database, and preprocessing begins. As a result, cleansed data is output.

[0344] Step 2:

[0345] The server runs a machine learning model using preprocessed data. Specifically, it formats the data using the Pandas library and inputs it into the training model using TensorFlow. The input is a feature vector to look for potential anomalies, and the model outputs an anomaly score. Anomaly detection is performed based on this output.

[0346] Step 3:

[0347] When an anomaly is detected, the server generates warning information. This is done by packaging the anomaly information in JSON format and sending it to the terminal via a web framework such as Flask. The input is the detected anomaly data, and the output is a warning message to the user.

[0348] Step 4:

[0349] The terminal visualizes the warning information received from the server using a GUI. Here, a JavaScript framework is used to update the information in real time and display warnings to the user. The input is warning information in JSON format, and the output is a visual warning display that the user can see.

[0350] Step 5:

[0351] The device analyzes emotions from the user's text and voice input. The emotion analysis engine uses the Hugging Face library to generate emotion scores (e.g., anxiety, reassurance, etc.) from the input text. The input is the user's natural language comments, and the output is the emotion score.

[0352] Step 6:

[0353] Based on the anomaly information and sentiment analysis results displayed on the device, users can select the necessary countermeasures. The system makes suggestions such as "provide information on evacuation shelters" or "present risk mitigation measures." A prompt message such as "Please provide guidelines for taking swift action in an emergency" is generated. Based on this prompt, the AI ​​model operates and outputs recommendations appropriate to the situation.

[0354] (Application Example 2)

[0355] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0356] Monitoring and managing the natural environment requires appropriate anomaly detection and rapid response, but conventional systems do not consider the user's emotional state, which can lead to delays in necessary warnings and actions. Furthermore, when such systems are applied to the security field, they may lack useful information for the user. Therefore, there is a need for a system that can dynamically recognize user emotions and respond flexibly according to the situation and emotions.

[0357] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0358] In this invention, the server includes means for collecting remote sensor data, means for preprocessing the collected data, means for running a predictive model using the preprocessed data to detect anomalies in the natural environment, means for recognizing the user's emotional state and optimizing responses according to that state, and means for highlighting warnings based on the user's emotions regarding security situations and guiding appropriate countermeasures. This enables comprehensive and human-centered management of anomalies in the natural environment and security challenges, and the rapid provision of necessary information to the user.

[0359] "Means of collecting remote sensor data" refers to technologies that use sensor networks to monitor the natural environment and conditions within facilities and acquire necessary data.

[0360] "Preprocessing means" refers to techniques that perform the process of formatting collected raw data into a format that can be analyzed for anomaly detection.

[0361] "Means for executing predictive models and detecting anomalies in the natural environment" refers to technologies that use machine learning algorithms to perform pattern recognition on data and identify anomalous events.

[0362] "Means for recognizing a user's emotional state" refers to technologies that analyze text and voice input from a user to accurately grasp their emotions at any given moment.

[0363] "Means for optimizing responses based on emotions" refers to technologies that include a process of adjusting notification content and suggestions based on recognized emotions.

[0364] "Means of emphasizing warnings and guiding appropriate countermeasures" refers to technologies that, when an anomaly occurs, communicate information with an appropriate level of urgency that matches the user's emotions and suggest necessary actions.

[0365] The server collects data from remote sensors and preprocesses the collected raw data using Python for efficient processing. The preprocessed data is analyzed by machine learning algorithms using TensorFlow to detect anomalies in the natural environment and security conditions. Detected anomalies are monitored in real time, and different levels of warning information are generated and notified to the user's device as visual information.

[0366] The device can use IBM Watson or Google Cloud Natural Language API to acquire user text input and voice data and recognize their emotional state. This allows for the selection of an appropriate response based on the user's emotions. For example, if the user is feeling anxious, the urgency of the warning can be emphasized, and suggestions can be displayed quickly.

[0367] As a concrete example, if a security manager at a facility receives a text message saying "smoke is visible" during a routine patrol, and the system determines it to be "dangerous," an optimized fire alarm and evacuation order are immediately sent to the manager. In this way, the system provides an efficient and human-centered response. An example of a prompt using a generative AI model is, "A thunderstorm is approaching, please temporarily stop the playground equipment for safety reasons." Based on this prompt, users can take appropriate action quickly and systematically.

[0368] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0369] Step 1:

[0370] The server collects data from remote sensors. The input is the raw data transmitted from each sensor, which includes measured environmental information. The output is this data stored in a temporary file. Specifically at this stage, real-time data from various sensors is received and saved to a database.

[0371] Step 2:

[0372] The raw data collected by the server is preprocessed to make it easier to process. The input is the raw data collected in step 1, and the output is the cleaned, preprocessed data. Specifically, operations such as denoising the data, correcting outliers, and scaling the data are performed.

[0373] Step 3:

[0374] The server inputs pre-processed data into a machine learning algorithm to detect anomalies. The input is pre-processed data, and the output is the anomaly detection result. The server runs an anomaly detection model using TensorFlow, analyzing patterns in the data to identify anomalous situations.

[0375] Step 4:

[0376] If an anomaly is detected, the server generates warning information about the anomaly and sends it to the terminal. The input is the anomaly detection result from step 3, and the output is the warning information notified to the user. Specifically, an appropriate message is generated based on the severity of the anomaly and sent to the user interface.

[0377] Step 5:

[0378] The system visualizes the warning information received by the device and notifies the user. The input is the warning information, and the output is a visually displayed alert for the user. The device displays a warning icon or text message on the screen so that the user can immediately check it.

[0379] Step 6:

[0380] The device recognizes the user's emotional state based on their input. The input is text or voice data entered by the user, and the output is the recognized emotional state. The device uses IBM Watson or the Google Cloud Natural Language API to analyze the user's emotions from their written or spoken text.

[0381] Step 7:

[0382] Based on the user's emotions, the device optimizes its response. Input is the user's emotional state and warning information, while output is a notification tailored to those emotions. For example, if anxiety is detected, the warning display is emphasized, and the user is given more specific instructions.

[0383] Step 8:

[0384] Based on server and terminal information, the system presents actionable steps for the user. Input consists of warning information and sentiment-based optimization results, while output is specific action suggestions. Instructions are presented as prompts using a generative AI model, enabling users to take swift action based on the suggestions received.

[0385] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0386] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0387] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0388] [Third Embodiment]

[0389] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0390] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0391] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0392] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0393] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0394] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0395] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0396] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0397] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0398] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0399] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0400] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0401] This invention is a system that utilizes sensor data collected from remote locations to detect anomalies in the natural environment in real time. It is realized through the cooperation of a server, terminals, and users.

[0402] The server first periodically collects data from satellites and installed IoT sensors. This data includes environmentally relevant values ​​such as temperature, humidity, soil moisture content, and vegetation index. The server preprocesses this data into a specific format, making it ready for analysis.

[0403] Next, the server inputs the pre-processed data into a machine learning model to detect anomalies in the natural environment. This model learns from past data and has the ability to identify anomalies as new patterns. In particular, it has exceptional detection capabilities for irreversible changes such as illegal logging and forest fires.

[0404] When an anomaly is detected, the server immediately generates warning information and notifies the relevant authorities and the user's terminal. The terminal receives this information and conveys the notification to the user in an easy-to-understand manner. The visualized results are displayed as maps and graphs so that the user can quickly understand the situation.

[0405] Based on the information presented on the device, users will consider immediate countermeasures in the natural environment. They will also utilize this data in their daily natural environment management to support long-term planning. This entire process will enable the sustainable protection and management of the natural environment.

[0406] The following describes the processing flow.

[0407] Step 1:

[0408] The server collects real-time environmental data from satellites and IoT sensors. This includes acquiring climate data and geographic information through data streams from remote APIs and devices.

[0409] Step 2:

[0410] The server preprocesses the collected raw data to convert it into a parseable format. This includes data cleaning, format conversion, and data imputation to ensure that the data is stored in a consistent state.

[0411] Step 3:

[0412] The server feeds pre-processed datasets into a machine learning model to perform analysis that detects anomalies in the natural environment. In this process, the model identifies anomalous patterns and phenomena and uses differential algorithms to compare them to the normal state.

[0413] Step 4:

[0414] When a machine learning model detects an anomaly, the server immediately generates an alert. This is a process that constructs detailed information including the type of anomaly, its scope of impact, and recommended immediate actions.

[0415] Step 5:

[0416] The terminal receives warning information sent from the server and visualizes it in a way that is intuitively understandable to the user. The terminal highlights anomalies on a map and displays related data in interactive graphs and other formats.

[0417] Step 6:

[0418] Users review the information provided on their devices and develop appropriate countermeasures and management plans based on that information. This assessment includes coordinating with relevant departments and rapidly allocating resources.

[0419] (Example 1)

[0420] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0421] There is a need to efficiently and accurately process natural environment data collected from remote locations and detect anomalies in real time. Furthermore, a systematic process is required to quickly share anomaly detection results and formulate and implement effective environmental conservation measures. Conventional systems have challenges in data processing efficiency and anomaly detection accuracy, making it difficult to implement immediate and long-term measures that minimize environmental impact.

[0422] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0423] In this invention, the server includes means for collecting data from a remote observation device, means for preprocessing the collected data with a data processing device, and means for detecting anomalies in the natural environment from the preprocessed data via a generative AI model. This enables rapid and accurate detection of anomalies in the natural environment, facilitating efficient information sharing and the formulation of effective environmental conservation measures.

[0424] A "remote observation device" is a device used to measure and collect data on a target environmental area in a remote location.

[0425] A "data processing device" is a device that performs preprocessing to format collected data into an analyzable format.

[0426] A "generative AI model" is a program or system that uses machine learning algorithms to learn new patterns from data and detect anomalies.

[0427] "An anomaly" refers to a state or event in the natural environment that deviates from normal conditions, and includes situations that require emergency response.

[0428] A "warning message" is notification information generated when an anomaly is detected, intended to inform relevant parties of the details.

[0429] A "data visualization device" is a device used to visually display analysis results, providing information in the form of maps, graphs, and other visual media.

[0430] "Related parties" refers to individuals or organizations involved in formulating and implementing countermeasures based on the results of anomaly detection.

[0431] This invention is a system for detecting anomalies in the natural environment in real time and taking appropriate countermeasures quickly. This system operates in cooperation with a server, terminals, and users. Specific embodiments are described below.

[0432] The server first collects natural environment data through remote monitoring equipment. This equipment includes sensors for measuring temperature, humidity, soil moisture content, vegetation index, and other parameters. Data from these sensors is transmitted to the server via the internet.

[0433] The server preprocesses the acquired data using a data processing unit. This preprocessing includes data cleaning, normalization, and missing value imputation, all of which are performed using the Python Pandas library. Data preprocessing prepares the data for machine analysis.

[0434] Next, the server inputs the pre-processed data into a generating AI model. This model is built using machine learning frameworks such as TensorFlow and detects anomalies using algorithms generated based on historical data. For example, it can analyze rapidly fluctuating temperature and humidity data in a specific area to detect signs of illegal burning.

[0435] When the server detects an anomaly, it generates a warning message and immediately notifies relevant parties and devices. Notifications are sent via email or push notifications.

[0436] The terminal receives warning messages sent from the server and displays them clearly to the user using a data visualization device. Specifically, it uses maps and graphs to show the area where the anomaly occurred and its details. This allows the user to easily understand the situation.

[0437] Based on information provided through their devices, users can quickly make real-time decisions on appropriate countermeasures in the natural environment. Furthermore, data can be used to formulate long-term natural environment conservation plans, enabling sustainable environmental management.

[0438] An example of a prompt message is, "Detect abnormal weather patterns in a specific area from the latest sensor data and predict their impact," which instructs the generated AI model to detect anomalies.

[0439] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0440] Step 1:

[0441] The server collects data on the natural environment from remote observation devices. The input is raw data transmitted from various sensors. This raw data includes information such as temperature, humidity, soil moisture content, and vegetation index. Specifically, the server periodically sends requests to retrieve data from sensors via an API. The output is the collected, unprocessed data.

[0442] Step 2:

[0443] The server preprocesses the collected raw data using a data processing device. The input is the unprocessed data collected in step 1. Data processing includes imputing missing values, removing outliers, and standardizing the data format. Specifically, the Python Pandas library is used to perform cleaning and normalization. The output is preprocessed data that is ready for analysis.

[0444] Step 3:

[0445] The server inputs preprocessed data into a generating AI model to detect anomalies. The input is the preprocessed data that is the output of step 2. Anomaly detection is performed using machine learning algorithms in the data calculation. Specifically, the TensorFlow framework is used to identify new patterns by comparing them with past data. The output is whether or not anomalies were detected and their detailed information.

[0446] Step 4:

[0447] The server generates a warning message and notifies relevant parties when an anomaly is detected. The input is the detailed information output by the anomaly detection model in step 3. Data processing organizes the information about the detected anomaly and converts it into a format for email or push notification. Specifically, the message generation system is used to create a warning that includes information about the detection location, type of anomaly, and urgency. The output is the warning message that is sent.

[0448] Step 5:

[0449] The terminal visually presents the received warning message to the user. The input is the warning message generated in step 4. Data processing extracts the message content and converts it into map and graph formats. Specifically, data visualization tools are used to highlight areas where anomalies have been detected and to display related data as a time-series graph. The output is visualized information that allows the user to immediately understand the situation.

[0450] Step 6:

[0451] The user quickly decides on countermeasures in the natural environment based on the information provided on the device. The input is the information visualized in step 5. The user assesses the situation and plans necessary field surveys and countermeasures. Specifically, they analyze the cause of the anomaly and communicate with relevant parties. The output is the countermeasures to be taken and the action plan.

[0452] (Application Example 1)

[0453] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0454] There is a need for a system that can detect abnormal events in the natural environment in real time and take appropriate countermeasures early on. Furthermore, it is necessary to support rapid decision-making by presenting the analysis results of abnormal events in a way that users can intuitively understand. In addition, there is a need to facilitate the development of environmental protection plans based on these abnormal events.

[0455] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0456] In this invention, the server includes a device for collecting remote environmental data, a device for converting the collected environmental data into an analyzable format, and a device for running a learning model using the converted environmental data to detect abnormal events. This enables real-time detection of abnormal events in the natural environment and the rapid provision of alarm information.

[0457] "Remote environmental data" refers to weather and topographic information collected using observation equipment from geographically distant locations.

[0458] "Device" refers to equipment or systems designed to achieve a specific purpose, and in this context, it refers to devices used for collecting, analyzing, and visualizing environmental data.

[0459] "Analyzable format" refers to a state in which collected data, which is not suitable for mechanical analysis in its original form, has been standardized and converted into an easily understandable format.

[0460] A "learning model" is a set of algorithms used to learn anomaly patterns based on past data and to detect anomalies from new data.

[0461] An "abnormal event" refers to a phenomenon or condition that deviates from the normal state, and in the natural environment, it particularly means unexpected weather changes or changes in topography.

[0462] "Alert information" refers to information that functions as a warning and is issued immediately when an abnormal event is detected, with the aim of promptly notifying relevant parties.

[0463] A "Geographic Information System" is a technology or system that uses digitized map data to visually analyze and display geographical information.

[0464] An "environmental protection plan" refers to a plan or strategy that includes measures and guidelines for action to minimize the impact of detected abnormal events.

[0465] The system for realizing this invention consists of a server, a terminal, and a user working together. The server first receives information from sensors that collect remote environmental data. The hardware used here includes weather sensors and image acquisition devices. This data is preprocessed within the server into a format suitable for analysis. This preprocessing involves normalizing the data and imputing missing values ​​using services such as AWS Lambda.

[0466] The server then inputs the pre-processed data into a learning model developed via Amazon SageMaker to detect anomalies. This learning model is designed to detect new anomalies using historical data as training data. Based on the detected anomalies, the server quickly generates alarm information and sends notifications to each device using AWS SNS. This information is visualized in an intuitively understandable format on the user's device. The application installed on the device uses the Google Maps API to display the location of the anomaly on a map and notifies the user of the situation.

[0467] For example, if a rapid rise in river levels due to an approaching typhoon is detected, a warning message will appear on the user's device stating, "River levels in your area have reached dangerous levels. Please prepare to evacuate." This allows the user to take prompt and appropriate action.

[0468] Examples of prompts for the generating AI model include, "A typhoon is approaching and rising water levels are expected. What measures would be effective?" and "I would like to know about emergency response measures when there are signs of an impending earthquake."

[0469] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0470] Step 1:

[0471] The server acquires environmental data such as temperature, humidity, and river water level from remote environmental sensors. The input is raw data transmitted from the sensors, and the output is unprocessed data stored on the server.

[0472] Step 2:

[0473] The server uses AWS Lambda to preprocess the acquired raw data into a parseable format. The input is unprocessed raw data, and the output is a normalized dataset. This process includes imputation of missing values ​​and removal of outliers.

[0474] Step 3:

[0475] The server inputs a pre-processed dataset into an Amazon SageMaker trained model. The input is a formatted dataset, and the output is information about detected anomalies. The model learns from past data patterns and can detect new anomalies with high accuracy.

[0476] Step 4:

[0477] When an abnormal event is detected, the server uses AWS SNS to generate an alarm and notifies the relevant devices. The input is the abnormal event information, and the output is the alarm notification to the devices.

[0478] Step 5:

[0479] The terminal visually presents the received alarm information to the user. Using the Google Maps API, it displays the location of the anomaly and its details on a map. The input is alarm information, and the output is visual information that the user can intuitively understand.

[0480] Step 6:

[0481] Users receive information presented on their devices and select the necessary actions. For example, they might take evacuation measures or gather further information. The user's input is decision-making and action selection, while the output is the implementation of specific countermeasures.

[0482] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0483] This invention incorporates an emotion engine into a system designed for monitoring and managing the natural environment, which recognizes user emotions and optimizes responses accordingly. It operates through an integrated approach involving a server, terminal, and user.

[0484] The server continues to process environmental data collected from satellites and IoT sensors, and uses machine learning models to detect anomalies. When an anomaly is detected, the server immediately generates warning information and sends it to the terminal.

[0485] The device visualizes received warning information and notifies the user appropriately. This is where the emotion engine comes in. The device picks up emotions from the user's text input and voice and analyzes their state. For example, if the user enters a comment emphasizing urgency, the emotion engine senses that anxiety and chooses a method that emphasizes more urgent actions rather than a formal exchange.

[0486] Users consider adopting management measures for the natural environment by reviewing environmental data and anomalies displayed on their device, as well as observations presented by the emotion engine. For example, if a user is feeling anxious in a situation where concerns are heightened due to extreme weather, the system will be configured to intensify warnings and recommend immediate disaster prevention measures.

[0487] In this way, systems integrated with an emotion engine promote the sustainable management of the natural environment while enabling flexible responses that also address the emotional needs of users. This process leads to more comprehensive and human-centered environmental management.

[0488] The following describes the processing flow.

[0489] Step 1:

[0490] The server collects data about the natural environment from satellites and IoT sensors. Specifically, it retrieves climate data, soil moisture information, and other data via remote APIs.

[0491] Step 2:

[0492] The server preprocesses the collected data and arranges it into a consistent data format. This includes data cleaning, noise reduction, and format conversion.

[0493] Step 3:

[0494] The server inputs pre-processed data into a machine learning model to detect anomalies in the natural environment. Here, the model analyzes anomalous patterns in real time and makes advanced predictions.

[0495] Step 4:

[0496] If an anomaly is detected, the server generates and sends warning information to the terminal. This information includes the location and scope of the anomaly, as well as recommended countermeasures.

[0497] Step 5:

[0498] The device displays received warning information using visualization tools. Anomaly locations are highlighted on a map and details are shown in interactive graphs for intuitive user understanding.

[0499] Step 6:

[0500] The emotion engine on the device collects emotional data from user input and voice, and analyzes that emotional state. It uses natural language processing technology to analyze the emotions of comments and feedback entered by the user.

[0501] Step 7:

[0502] Based on the results of the emotion engine, the device adjusts how information is presented and displays flexible responses tailored to the user's emotions. For example, if the user is feeling stressed, it prioritizes presenting quick solutions.

[0503] Step 8:

[0504] Users review the information and recommendations presented on their devices and make environmental management decisions. A specific example is when a user decides whether or not to implement emergency measures in response to extreme weather.

[0505] (Example 2)

[0506] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0507] Systems that monitor and manage anomalies in the natural environment require flexible and adaptive responses that take user emotions into consideration. Conventional systems have focused on detecting anomalies and generating warnings, but have not adequately provided personalized responses that address users' emotional needs. This can hinder users from making appropriate decisions and taking swift action when an anomaly occurs.

[0508] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0509] In this invention, the server includes means for collecting remote information, means for preprocessing the collected information, means for executing a learning model using the preprocessed information to detect anomalies in the natural environment, means for generating and notifying warning information when an anomaly is detected, means for analyzing the user's emotions and optimizing responses, means for recommending a natural environment management plan based on the analysis results, and means for visualizing the analysis results. This enables the rapid and intuitive proposal of countermeasures for the user when an anomaly is detected, realizing flexible environmental management that responds to the user's feelings.

[0510] "Remote information" refers to digital data collected from physically distant locations, primarily obtained through sensors and communication devices.

[0511] "Preprocessing" refers to a series of steps that transform raw data into an analyzable format, including data shaping, cleansing, and filtering.

[0512] A "learning model" is an application that uses mathematical and computational methods to learn patterns and rules from data and predict events.

[0513] "Anomaly detection" is the process of identifying deviations from normal patterns or abnormal fluctuations, and is carried out through data analysis.

[0514] "Warning information" refers to notification messages generated when the system detects an anomaly, informing the user of the risks and necessary actions.

[0515] "User sentiment" is an indicator that shows the user's psychological state, and it is analyzed using natural language processing technology from text and voice input.

[0516] A "natural environment management plan" is an action plan aimed at maintaining and protecting the natural environment, and is a strategic proposal generated based on data analysis results.

[0517] "Visualization" refers to the technique of visually representing analyzed data and information to present it to users in an easy-to-understand manner, primarily in the form of graphs and charts.

[0518] This system is a technical approach to monitor the natural environment and encourage users to take appropriate management actions. It primarily relies on the collaborative functioning of three parties: the server, the terminal, and the user.

[0519] The server first collects a large amount of information about the natural environment through remote data collection devices such as satellites and IoT sensors. Raspberry Pi and Arduino are commonly used as data collection devices in this process. The server then uses Python and TensorFlow to analyze the data and run a learning model to detect anomalies. This enables the rapid detection of extreme weather events and environmental risks.

[0520] When an anomaly is detected, the server immediately generates warning information and notifies the terminal via the network. The terminal visually displays the received warning information using a GUI (Graphical User Interface) to clearly communicate it to the user. The terminal's GUI is built with a JavaScript framework such as React to support the user's intuitive understanding.

[0521] Furthermore, the device is equipped with an emotion engine that analyzes the user's emotions. This engine is implemented using the Hugging Face Transformers library and analyzes emotions from user input. For example, if the user inputs "I'm worried about what will happen next," the engine detects the emotion of anxiety from the text and presents appropriate countermeasures on the device.

[0522] Users can determine how to manage the natural environment based on the abnormal data and sentiment analysis results displayed on their device. The system uses a generative AI model to suggest flexible countermeasures to the user based on prompts such as, "If the system senses urgency from the comments entered by the user, please suggest appropriate follow-up."

[0523] In this way, this system enables sustainable management of the natural environment and responds quickly to the emotional needs of users, thereby achieving safer and more adaptive environmental management.

[0524] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0525] Step 1:

[0526] The server first receives environmental information from remote data collection devices. Specifically, data transmitted from sensors arrives at the server via an API. At this time, the input consists of specific environmental data such as temperature, humidity, and precipitation. This data is stored in a database, and preprocessing begins. As a result, cleansed data is output.

[0527] Step 2:

[0528] The server runs a machine learning model using preprocessed data. Specifically, it formats the data using the Pandas library and inputs it into the training model using TensorFlow. The input is a feature vector to look for potential anomalies, and the model outputs an anomaly score. Anomaly detection is performed based on this output.

[0529] Step 3:

[0530] When an anomaly is detected, the server generates warning information. This is done by packaging the anomaly information in JSON format and sending it to the terminal via a web framework such as Flask. The input is the detected anomaly data, and the output is a warning message to the user.

[0531] Step 4:

[0532] The terminal visualizes the warning information received from the server using a GUI. Here, a JavaScript framework is used to update the information in real time and display warnings to the user. The input is warning information in JSON format, and the output is a visual warning display that the user can see.

[0533] Step 5:

[0534] The device analyzes emotions from the user's text and voice input. The emotion analysis engine uses the Hugging Face library to generate emotion scores (e.g., anxiety, reassurance, etc.) from the input text. The input is the user's natural language comments, and the output is the emotion score.

[0535] Step 6:

[0536] Based on the anomaly information and sentiment analysis results displayed on the device, users can select the necessary countermeasures. The system makes suggestions such as "provide information on evacuation shelters" or "present risk mitigation measures." A prompt message such as "Please provide guidelines for taking swift action in an emergency" is generated. Based on this prompt, the AI ​​model operates and outputs recommendations appropriate to the situation.

[0537] (Application Example 2)

[0538] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0539] Monitoring and managing the natural environment requires appropriate anomaly detection and rapid response, but conventional systems do not consider the user's emotional state, which can lead to delays in necessary warnings and actions. Furthermore, when such systems are applied to the security field, they may lack useful information for the user. Therefore, there is a need for a system that can dynamically recognize user emotions and respond flexibly according to the situation and emotions.

[0540] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0541] In this invention, the server includes means for collecting remote sensor data, means for preprocessing the collected data, means for running a predictive model using the preprocessed data to detect anomalies in the natural environment, means for recognizing the user's emotional state and optimizing responses according to that state, and means for highlighting warnings based on the user's emotions regarding security situations and guiding appropriate countermeasures. This enables comprehensive and human-centered management of anomalies in the natural environment and security challenges, and the rapid provision of necessary information to the user.

[0542] "Means of collecting remote sensor data" refers to technologies that use sensor networks to monitor the natural environment and conditions within facilities and acquire necessary data.

[0543] "Preprocessing means" refers to techniques that perform the process of formatting collected raw data into a format that can be analyzed for anomaly detection.

[0544] "Means for executing predictive models and detecting anomalies in the natural environment" refers to technologies that use machine learning algorithms to perform pattern recognition on data and identify anomalous events.

[0545] "Means for recognizing a user's emotional state" refers to technologies that analyze text and voice input from a user to accurately grasp their emotions at any given moment.

[0546] "Means for optimizing responses based on emotions" refers to technologies that include a process of adjusting notification content and suggestions based on recognized emotions.

[0547] "Means of emphasizing warnings and guiding appropriate countermeasures" refers to technologies that, when an anomaly occurs, communicate information with an appropriate level of urgency that matches the user's emotions and suggest necessary actions.

[0548] The server collects data from remote sensors and preprocesses the collected raw data using Python for efficient processing. The preprocessed data is analyzed by machine learning algorithms using TensorFlow to detect anomalies in the natural environment and security conditions. Detected anomalies are monitored in real time, and different levels of warning information are generated and notified to the user's device as visual information.

[0549] The device can use IBM Watson or Google Cloud Natural Language API to acquire user text input and voice data and recognize their emotional state. This allows for the selection of an appropriate response based on the user's emotions. For example, if the user is feeling anxious, the urgency of the warning can be emphasized, and suggestions can be displayed quickly.

[0550] As a concrete example, if a security manager at a facility receives a text message saying "smoke is visible" during a routine patrol, and the system determines it to be "dangerous," an optimized fire alarm and evacuation order are immediately sent to the manager. In this way, the system provides an efficient and human-centered response. An example of a prompt using a generative AI model is, "A thunderstorm is approaching, please temporarily stop the playground equipment for safety reasons." Based on this prompt, users can take appropriate action quickly and systematically.

[0551] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0552] Step 1:

[0553] The server collects data from remote sensors. The input is the raw data transmitted from each sensor, which includes measured environmental information. The output is this data stored in a temporary file. Specifically at this stage, real-time data from various sensors is received and saved to a database.

[0554] Step 2:

[0555] The raw data collected by the server is preprocessed to make it easier to process. The input is the raw data collected in step 1, and the output is the cleaned, preprocessed data. Specifically, operations such as denoising the data, correcting outliers, and scaling the data are performed.

[0556] Step 3:

[0557] The server inputs pre-processed data into a machine learning algorithm to detect anomalies. The input is pre-processed data, and the output is the anomaly detection result. The server runs an anomaly detection model using TensorFlow, analyzing patterns in the data to identify anomalous situations.

[0558] Step 4:

[0559] If an anomaly is detected, the server generates warning information about the anomaly and sends it to the terminal. The input is the anomaly detection result from step 3, and the output is the warning information notified to the user. Specifically, an appropriate message is generated based on the severity of the anomaly and sent to the user interface.

[0560] Step 5:

[0561] The system visualizes the warning information received by the device and notifies the user. The input is the warning information, and the output is a visually displayed alert for the user. The device displays a warning icon or text message on the screen so that the user can immediately check it.

[0562] Step 6:

[0563] The device recognizes the user's emotional state based on their input. The input is text or voice data entered by the user, and the output is the recognized emotional state. The device uses IBM Watson or the Google Cloud Natural Language API to analyze the user's emotions from their written or spoken text.

[0564] Step 7:

[0565] Based on the user's emotions, the device optimizes its response. Input is the user's emotional state and warning information, while output is a notification tailored to those emotions. For example, if anxiety is detected, the warning display is emphasized, and the user is given more specific instructions.

[0566] Step 8:

[0567] Based on server and terminal information, the system presents actionable steps for the user. Input consists of warning information and sentiment-based optimization results, while output is specific action suggestions. Instructions are presented as prompts using a generative AI model, enabling users to take swift action based on the suggestions received.

[0568] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0569] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0570] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0571] [Fourth Embodiment]

[0572] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0573] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0574] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0575] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0576] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0577] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0578] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0579] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0580] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0581] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0582] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0583] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0584] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0585] This invention is a system that utilizes sensor data collected from remote locations to detect anomalies in the natural environment in real time. It is realized through the cooperation of a server, terminals, and users.

[0586] The server first periodically collects data from satellites and installed IoT sensors. This data includes environmentally relevant values ​​such as temperature, humidity, soil moisture content, and vegetation index. The server preprocesses this data into a specific format, making it ready for analysis.

[0587] Next, the server inputs the pre-processed data into a machine learning model to detect anomalies in the natural environment. This model learns from past data and has the ability to identify anomalies as new patterns. In particular, it has exceptional detection capabilities for irreversible changes such as illegal logging and forest fires.

[0588] When an anomaly is detected, the server immediately generates warning information and notifies the relevant authorities and the user's terminal. The terminal receives this information and conveys the notification to the user in an easy-to-understand manner. The visualized results are displayed as maps and graphs so that the user can quickly understand the situation.

[0589] Based on the information presented on the device, users will consider immediate countermeasures in the natural environment. They will also utilize this data in their daily natural environment management to support long-term planning. This entire process will enable the sustainable protection and management of the natural environment.

[0590] The following describes the processing flow.

[0591] Step 1:

[0592] The server collects real-time environmental data from satellites and IoT sensors. This includes acquiring climate data and geographic information through data streams from remote APIs and devices.

[0593] Step 2:

[0594] The server preprocesses the collected raw data to convert it into a parseable format. This includes data cleaning, format conversion, and data imputation to ensure that the data is stored in a consistent state.

[0595] Step 3:

[0596] The server feeds pre-processed datasets into a machine learning model to perform analysis that detects anomalies in the natural environment. In this process, the model identifies anomalous patterns and phenomena and uses differential algorithms to compare them to the normal state.

[0597] Step 4:

[0598] When a machine learning model detects an anomaly, the server immediately generates an alert. This is a process that constructs detailed information including the type of anomaly, its scope of impact, and recommended immediate actions.

[0599] Step 5:

[0600] The terminal receives warning information sent from the server and visualizes it in a way that is intuitively understandable to the user. The terminal highlights anomalies on a map and displays related data in interactive graphs and other formats.

[0601] Step 6:

[0602] Users review the information provided on their devices and develop appropriate countermeasures and management plans based on that information. This assessment includes coordinating with relevant departments and rapidly allocating resources.

[0603] (Example 1)

[0604] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0605] There is a need to efficiently and accurately process natural environment data collected from remote locations and detect anomalies in real time. Furthermore, a systematic process is required to quickly share anomaly detection results and formulate and implement effective environmental conservation measures. Conventional systems have challenges in data processing efficiency and anomaly detection accuracy, making it difficult to implement immediate and long-term measures that minimize environmental impact.

[0606] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0607] In this invention, the server includes means for collecting data from a remote observation device, means for preprocessing the collected data with a data processing device, and means for detecting anomalies in the natural environment from the preprocessed data via a generative AI model. This enables rapid and accurate detection of anomalies in the natural environment, facilitating efficient information sharing and the formulation of effective environmental conservation measures.

[0608] A "remote observation device" is a device used to measure and collect data on a target environmental area in a remote location.

[0609] A "data processing device" is a device that performs preprocessing to format collected data into an analyzable format.

[0610] A "generative AI model" is a program or system that uses machine learning algorithms to learn new patterns from data and detect anomalies.

[0611] "An anomaly" refers to a state or event in the natural environment that deviates from normal conditions, and includes situations that require emergency response.

[0612] A "warning message" is notification information generated when an anomaly is detected, intended to inform relevant parties of the details.

[0613] A "data visualization device" is a device used to visually display analysis results, providing information in the form of maps, graphs, and other visual media.

[0614] "Related parties" refers to individuals or organizations involved in formulating and implementing countermeasures based on the results of anomaly detection.

[0615] This invention is a system for detecting anomalies in the natural environment in real time and taking appropriate countermeasures quickly. This system operates in cooperation with a server, terminals, and users. Specific embodiments are described below.

[0616] The server first collects natural environment data through remote monitoring equipment. This equipment includes sensors for measuring temperature, humidity, soil moisture content, vegetation index, and other parameters. Data from these sensors is transmitted to the server via the internet.

[0617] The server preprocesses the acquired data using a data processing unit. This preprocessing includes data cleaning, normalization, and missing value imputation, all of which are performed using the Python Pandas library. Data preprocessing prepares the data for machine analysis.

[0618] Next, the server inputs the pre-processed data into a generating AI model. This model is built using machine learning frameworks such as TensorFlow and detects anomalies using algorithms generated based on historical data. For example, it can analyze rapidly fluctuating temperature and humidity data in a specific area to detect signs of illegal burning.

[0619] When the server detects an anomaly, it generates a warning message and immediately notifies relevant parties and devices. Notifications are sent via email or push notifications.

[0620] The terminal receives warning messages sent from the server and displays them clearly to the user using a data visualization device. Specifically, it uses maps and graphs to show the area where the anomaly occurred and its details. This allows the user to easily understand the situation.

[0621] Based on information provided through their devices, users can quickly make real-time decisions on appropriate countermeasures in the natural environment. Furthermore, data can be used to formulate long-term natural environment conservation plans, enabling sustainable environmental management.

[0622] An example of a prompt message is, "Detect abnormal weather patterns in a specific area from the latest sensor data and predict their impact," which instructs the generated AI model to detect anomalies.

[0623] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0624] Step 1:

[0625] The server collects data on the natural environment from remote observation devices. The input is raw data transmitted from various sensors. This raw data includes information such as temperature, humidity, soil moisture content, and vegetation index. Specifically, the server periodically sends requests to retrieve data from sensors via an API. The output is the collected, unprocessed data.

[0626] Step 2:

[0627] The server preprocesses the collected raw data using a data processing device. The input is the unprocessed data collected in step 1. Data processing includes imputing missing values, removing outliers, and standardizing the data format. Specifically, the Python Pandas library is used to perform cleaning and normalization. The output is preprocessed data that is ready for analysis.

[0628] Step 3:

[0629] The server inputs preprocessed data into a generating AI model to detect anomalies. The input is the preprocessed data that is the output of step 2. Anomaly detection is performed using machine learning algorithms in the data calculation. Specifically, the TensorFlow framework is used to identify new patterns by comparing them with past data. The output is whether or not anomalies were detected and their detailed information.

[0630] Step 4:

[0631] The server generates a warning message and notifies relevant parties when an anomaly is detected. The input is the detailed information output by the anomaly detection model in step 3. Data processing organizes the information about the detected anomaly and converts it into a format for email or push notification. Specifically, the message generation system is used to create a warning that includes information about the detection location, type of anomaly, and urgency. The output is the warning message that is sent.

[0632] Step 5:

[0633] The terminal visually presents the received warning message to the user. The input is the warning message generated in step 4. Data processing extracts the message content and converts it into map and graph formats. Specifically, data visualization tools are used to highlight areas where anomalies have been detected and to display related data as a time-series graph. The output is visualized information that allows the user to immediately understand the situation.

[0634] Step 6:

[0635] The user quickly decides on countermeasures in the natural environment based on the information provided on the device. The input is the information visualized in step 5. The user assesses the situation and plans necessary field surveys and countermeasures. Specifically, they analyze the cause of the anomaly and communicate with relevant parties. The output is the countermeasures to be taken and the action plan.

[0636] (Application Example 1)

[0637] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0638] There is a need for a system that can detect abnormal events in the natural environment in real time and take appropriate countermeasures early on. Furthermore, it is necessary to support rapid decision-making by presenting the analysis results of abnormal events in a way that users can intuitively understand. In addition, there is a need to facilitate the development of environmental protection plans based on these abnormal events.

[0639] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0640] In this invention, the server includes a device for collecting remote environmental data, a device for converting the collected environmental data into an analyzable format, and a device for running a learning model using the converted environmental data to detect abnormal events. This enables real-time detection of abnormal events in the natural environment and the rapid provision of alarm information.

[0641] "Remote environmental data" refers to weather and topographic information collected using observation equipment from geographically distant locations.

[0642] "Device" refers to equipment or systems designed to achieve a specific purpose, and in this context, it refers to devices used for collecting, analyzing, and visualizing environmental data.

[0643] "Analyzable format" refers to a state in which collected data, which is not suitable for mechanical analysis in its original form, has been standardized and converted into an easily understandable format.

[0644] A "learning model" is a set of algorithms used to learn anomaly patterns based on past data and to detect anomalies from new data.

[0645] An "abnormal event" refers to a phenomenon or condition that deviates from the normal state, and in the natural environment, it particularly means unexpected weather changes or changes in topography.

[0646] "Alert information" refers to information that functions as a warning and is issued immediately when an abnormal event is detected, with the aim of promptly notifying relevant parties.

[0647] A "Geographic Information System" is a technology or system that uses digitized map data to visually analyze and display geographical information.

[0648] An "environmental protection plan" refers to a plan or strategy that includes measures and guidelines for action to minimize the impact of detected abnormal events.

[0649] The system for realizing this invention consists of a server, a terminal, and a user working together. The server first receives information from sensors that collect remote environmental data. The hardware used here includes weather sensors and image acquisition devices. This data is preprocessed within the server into a format suitable for analysis. This preprocessing involves normalizing the data and imputing missing values ​​using services such as AWS Lambda.

[0650] The server then inputs the pre-processed data into a learning model developed via Amazon SageMaker to detect anomalies. This learning model is designed to detect new anomalies using historical data as training data. Based on the detected anomalies, the server quickly generates alarm information and sends notifications to each device using AWS SNS. This information is visualized in an intuitively understandable format on the user's device. The application installed on the device uses the Google Maps API to display the location of the anomaly on a map and notifies the user of the situation.

[0651] For example, if a rapid rise in river levels due to an approaching typhoon is detected, a warning message will appear on the user's device stating, "River levels in your area have reached dangerous levels. Please prepare to evacuate." This allows the user to take prompt and appropriate action.

[0652] Examples of prompts for the generating AI model include, "A typhoon is approaching and rising water levels are expected. What measures would be effective?" and "I would like to know about emergency response measures when there are signs of an impending earthquake."

[0653] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0654] Step 1:

[0655] The server acquires environmental data such as temperature, humidity, and river water level from remote environmental sensors. The input is raw data transmitted from the sensors, and the output is unprocessed data stored on the server.

[0656] Step 2:

[0657] The server uses AWS Lambda to preprocess the acquired raw data into a parseable format. The input is unprocessed raw data, and the output is a normalized dataset. This process includes imputation of missing values ​​and removal of outliers.

[0658] Step 3:

[0659] The server inputs a pre-processed dataset into an Amazon SageMaker trained model. The input is a formatted dataset, and the output is information about detected anomalies. The model learns from past data patterns and can detect new anomalies with high accuracy.

[0660] Step 4:

[0661] When an abnormal event is detected, the server uses AWS SNS to generate an alarm and notifies the relevant devices. The input is the abnormal event information, and the output is the alarm notification to the devices.

[0662] Step 5:

[0663] The terminal visually presents the received alarm information to the user. Using the Google Maps API, it displays the location of the anomaly and its details on a map. The input is alarm information, and the output is visual information that the user can intuitively understand.

[0664] Step 6:

[0665] Users receive information presented on their devices and select the necessary actions. For example, they might take evacuation measures or gather further information. The user's input is decision-making and action selection, while the output is the implementation of specific countermeasures.

[0666] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0667] This invention incorporates an emotion engine into a system designed for monitoring and managing the natural environment, which recognizes user emotions and optimizes responses accordingly. It operates through an integrated approach involving a server, terminal, and user.

[0668] The server continues to process environmental data collected from satellites and IoT sensors, and uses machine learning models to detect anomalies. When an anomaly is detected, the server immediately generates warning information and sends it to the terminal.

[0669] The device visualizes received warning information and notifies the user appropriately. This is where the emotion engine comes in. The device picks up emotions from the user's text input and voice and analyzes their state. For example, if the user enters a comment emphasizing urgency, the emotion engine senses that anxiety and chooses a method that emphasizes more urgent actions rather than a formal exchange.

[0670] Users consider adopting management measures for the natural environment by reviewing environmental data and anomalies displayed on their device, as well as observations presented by the emotion engine. For example, if a user is feeling anxious in a situation where concerns are heightened due to extreme weather, the system will be configured to intensify warnings and recommend immediate disaster prevention measures.

[0671] In this way, systems integrated with an emotion engine promote the sustainable management of the natural environment while enabling flexible responses that also address the emotional needs of users. This process leads to more comprehensive and human-centered environmental management.

[0672] The following describes the processing flow.

[0673] Step 1:

[0674] The server collects data about the natural environment from satellites and IoT sensors. Specifically, it retrieves climate data, soil moisture information, and other data via remote APIs.

[0675] Step 2:

[0676] The server preprocesses the collected data and arranges it into a consistent data format. This includes data cleaning, noise reduction, and format conversion.

[0677] Step 3:

[0678] The server inputs pre-processed data into a machine learning model to detect anomalies in the natural environment. Here, the model analyzes anomalous patterns in real time and makes advanced predictions.

[0679] Step 4:

[0680] If an anomaly is detected, the server generates and sends warning information to the terminal. This information includes the location and scope of the anomaly, as well as recommended countermeasures.

[0681] Step 5:

[0682] The device displays received warning information using visualization tools. Anomaly locations are highlighted on a map and details are shown in interactive graphs for intuitive user understanding.

[0683] Step 6:

[0684] The emotion engine on the device collects emotional data from user input and voice, and analyzes that emotional state. It uses natural language processing technology to analyze the emotions of comments and feedback entered by the user.

[0685] Step 7:

[0686] Based on the results of the emotion engine, the device adjusts how information is presented and displays flexible responses tailored to the user's emotions. For example, if the user is feeling stressed, it prioritizes presenting quick solutions.

[0687] Step 8:

[0688] Users review the information and recommendations presented on their devices and make environmental management decisions. A specific example is when a user decides whether or not to implement emergency measures in response to extreme weather.

[0689] (Example 2)

[0690] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0691] Systems that monitor and manage anomalies in the natural environment require flexible and adaptive responses that take user emotions into consideration. Conventional systems have focused on detecting anomalies and generating warnings, but have not adequately provided personalized responses that address users' emotional needs. This can hinder users from making appropriate decisions and taking swift action when an anomaly occurs.

[0692] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0693] In this invention, the server includes means for collecting remote information, means for preprocessing the collected information, means for executing a learning model using the preprocessed information to detect anomalies in the natural environment, means for generating and notifying warning information when an anomaly is detected, means for analyzing the user's emotions and optimizing responses, means for recommending a natural environment management plan based on the analysis results, and means for visualizing the analysis results. This enables the rapid and intuitive proposal of countermeasures for the user when an anomaly is detected, realizing flexible environmental management that responds to the user's feelings.

[0694] "Remote information" refers to digital data collected from physically distant locations, primarily obtained through sensors and communication devices.

[0695] "Preprocessing" refers to a series of steps that transform raw data into an analyzable format, including data shaping, cleansing, and filtering.

[0696] A "learning model" is an application that uses mathematical and computational methods to learn patterns and rules from data and predict events.

[0697] "Anomaly detection" is the process of identifying deviations from normal patterns or abnormal fluctuations, and is carried out through data analysis.

[0698] "Warning information" refers to notification messages generated when the system detects an anomaly, informing the user of the risks and necessary actions.

[0699] "User sentiment" is an indicator that shows the user's psychological state, and it is analyzed using natural language processing technology from text and voice input.

[0700] A "natural environment management plan" is an action plan aimed at maintaining and protecting the natural environment, and is a strategic proposal generated based on data analysis results.

[0701] "Visualization" refers to the technique of visually representing analyzed data and information to present it to users in an easy-to-understand manner, primarily in the form of graphs and charts.

[0702] This system is a technical approach to monitor the natural environment and encourage users to take appropriate management actions. It primarily relies on the collaborative functioning of three parties: the server, the terminal, and the user.

[0703] The server first collects a large amount of information about the natural environment through remote data collection devices such as satellites and IoT sensors. Raspberry Pi and Arduino are commonly used as data collection devices in this process. The server then uses Python and TensorFlow to analyze the data and run a learning model to detect anomalies. This enables the rapid detection of extreme weather events and environmental risks.

[0704] When an anomaly is detected, the server immediately generates warning information and notifies the terminal via the network. The terminal visually displays the received warning information using a GUI (Graphical User Interface) to clearly communicate it to the user. The terminal's GUI is built with a JavaScript framework such as React to support the user's intuitive understanding.

[0705] Furthermore, the device is equipped with an emotion engine that analyzes the user's emotions. This engine is implemented using the Hugging Face Transformers library and analyzes emotions from user input. For example, if the user inputs "I'm worried about what will happen next," the engine detects the emotion of anxiety from the text and presents appropriate countermeasures on the device.

[0706] Users can determine how to manage the natural environment based on the abnormal data and sentiment analysis results displayed on their device. The system uses a generative AI model to suggest flexible countermeasures to the user based on prompts such as, "If the system senses urgency from the comments entered by the user, please suggest appropriate follow-up."

[0707] In this way, this system enables sustainable management of the natural environment and responds quickly to the emotional needs of users, thereby achieving safer and more adaptive environmental management.

[0708] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0709] Step 1:

[0710] The server first receives environmental information from remote data collection devices. Specifically, data transmitted from sensors arrives at the server via an API. At this time, the input consists of specific environmental data such as temperature, humidity, and precipitation. This data is stored in a database, and preprocessing begins. As a result, cleansed data is output.

[0711] Step 2:

[0712] The server runs a machine learning model using preprocessed data. Specifically, it formats the data using the Pandas library and inputs it into the training model using TensorFlow. The input is a feature vector to look for potential anomalies, and the model outputs an anomaly score. Anomaly detection is performed based on this output.

[0713] Step 3:

[0714] When an anomaly is detected, the server generates warning information. This is done by packaging the anomaly information in JSON format and sending it to the terminal via a web framework such as Flask. The input is the detected anomaly data, and the output is a warning message to the user.

[0715] Step 4:

[0716] The terminal visualizes the warning information received from the server using a GUI. Here, a JavaScript framework is used to update the information in real time and display warnings to the user. The input is warning information in JSON format, and the output is a visual warning display that the user can see.

[0717] Step 5:

[0718] The device analyzes emotions from the user's text and voice input. The emotion analysis engine uses the Hugging Face library to generate emotion scores (e.g., anxiety, reassurance, etc.) from the input text. The input is the user's natural language comments, and the output is the emotion score.

[0719] Step 6:

[0720] Based on the anomaly information and sentiment analysis results displayed on the device, users can select the necessary countermeasures. The system makes suggestions such as "provide information on evacuation shelters" or "present risk mitigation measures." A prompt message such as "Please provide guidelines for taking swift action in an emergency" is generated. Based on this prompt, the AI ​​model operates and outputs recommendations appropriate to the situation.

[0721] (Application Example 2)

[0722] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0723] Monitoring and managing the natural environment requires appropriate anomaly detection and rapid response, but conventional systems do not consider the user's emotional state, which can lead to delays in necessary warnings and actions. Furthermore, when such systems are applied to the security field, they may lack useful information for the user. Therefore, there is a need for a system that can dynamically recognize user emotions and respond flexibly according to the situation and emotions.

[0724] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0725] In this invention, the server includes means for collecting remote sensor data, means for preprocessing the collected data, means for running a predictive model using the preprocessed data to detect anomalies in the natural environment, means for recognizing the user's emotional state and optimizing responses according to that state, and means for highlighting warnings based on the user's emotions regarding security situations and guiding appropriate countermeasures. This enables comprehensive and human-centered management of anomalies in the natural environment and security challenges, and the rapid provision of necessary information to the user.

[0726] "Means of collecting remote sensor data" refers to technologies that use sensor networks to monitor the natural environment and conditions within facilities and acquire necessary data.

[0727] "Preprocessing means" refers to techniques that perform the process of formatting collected raw data into a format that can be analyzed for anomaly detection.

[0728] "Means for executing predictive models and detecting anomalies in the natural environment" refers to technologies that use machine learning algorithms to perform pattern recognition on data and identify anomalous events.

[0729] "Means for recognizing a user's emotional state" refers to technologies that analyze text and voice input from a user to accurately grasp their emotions at any given moment.

[0730] "Means for optimizing responses based on emotions" refers to technologies that include a process of adjusting notification content and suggestions based on recognized emotions.

[0731] "Means of emphasizing warnings and guiding appropriate countermeasures" refers to technologies that, when an anomaly occurs, communicate information with an appropriate level of urgency that matches the user's emotions and suggest necessary actions.

[0732] The server collects data from remote sensors and preprocesses the collected raw data using Python for efficient processing. The preprocessed data is analyzed by machine learning algorithms using TensorFlow to detect anomalies in the natural environment and security conditions. Detected anomalies are monitored in real time, and different levels of warning information are generated and notified to the user's device as visual information.

[0733] The device can use IBM Watson or Google Cloud Natural Language API to acquire user text input and voice data and recognize their emotional state. This allows for the selection of an appropriate response based on the user's emotions. For example, if the user is feeling anxious, the urgency of the warning can be emphasized, and suggestions can be displayed quickly.

[0734] As a concrete example, if a security manager at a facility receives a text message saying "smoke is visible" during a routine patrol, and the system determines it to be "dangerous," an optimized fire alarm and evacuation order are immediately sent to the manager. In this way, the system provides an efficient and human-centered response. An example of a prompt using a generative AI model is, "A thunderstorm is approaching, please temporarily stop the playground equipment for safety reasons." Based on this prompt, users can take appropriate action quickly and systematically.

[0735] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0736] Step 1:

[0737] The server collects data from remote sensors. The input is the raw data transmitted from each sensor, which includes measured environmental information. The output is this data stored in a temporary file. Specifically at this stage, real-time data from various sensors is received and saved to a database.

[0738] Step 2:

[0739] The raw data collected by the server is preprocessed to make it easier to process. The input is the raw data collected in step 1, and the output is the cleaned, preprocessed data. Specifically, operations such as denoising the data, correcting outliers, and scaling the data are performed.

[0740] Step 3:

[0741] The server inputs pre-processed data into a machine learning algorithm to detect anomalies. The input is pre-processed data, and the output is the anomaly detection result. The server runs an anomaly detection model using TensorFlow, analyzing patterns in the data to identify anomalous situations.

[0742] Step 4:

[0743] If an anomaly is detected, the server generates warning information about the anomaly and sends it to the terminal. The input is the anomaly detection result from step 3, and the output is the warning information notified to the user. Specifically, an appropriate message is generated based on the severity of the anomaly and sent to the user interface.

[0744] Step 5:

[0745] The system visualizes the warning information received by the device and notifies the user. The input is the warning information, and the output is a visually displayed alert for the user. The device displays a warning icon or text message on the screen so that the user can immediately check it.

[0746] Step 6:

[0747] The device recognizes the user's emotional state based on their input. The input is text or voice data entered by the user, and the output is the recognized emotional state. The device uses IBM Watson or the Google Cloud Natural Language API to analyze the user's emotions from their written or spoken text.

[0748] Step 7:

[0749] Based on the user's emotions, the device optimizes its response. Input is the user's emotional state and warning information, while output is a notification tailored to those emotions. For example, if anxiety is detected, the warning display is emphasized, and the user is given more specific instructions.

[0750] Step 8:

[0751] Based on server and terminal information, the system presents actionable steps for the user. Input consists of warning information and sentiment-based optimization results, while output is specific action suggestions. Instructions are presented as prompts using a generative AI model, enabling users to take swift action based on the suggestions received.

[0752] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0753] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0754] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0755] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0756] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0757] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0758] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0759] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0760] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0761] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0762] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0763] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0764] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0765] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0766] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0767] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0768] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0769] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0770] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0771] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0772] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0773] The following is further disclosed regarding the embodiments described above.

[0774] (Claim 1)

[0775] A means of collecting remote sensor data,

[0776] Means for preprocessing the collected data,

[0777] A means for detecting anomalies in the natural environment by running a predictive model using preprocessed data,

[0778] A means of generating and notifying warning information when an anomaly is detected,

[0779] Means for visualizing the analysis results,

[0780] A system that includes this.

[0781] (Claim 2)

[0782] The system according to claim 1, wherein the predictive model uses a machine learning algorithm to detect anomalies.

[0783] (Claim 3)

[0784] The system according to claim 1, further comprising means for proposing a natural environment management plan based on the results of anomaly detection.

[0785] "Example 1"

[0786] (Claim 1)

[0787] Means for collecting data from remote observation devices,

[0788] A means for preprocessing the collected data using a data processing device,

[0789] A means for detecting anomalies in the natural environment via a generative AI model using preprocessed data,

[0790] A means of generating a warning message and sending it to relevant parties when an anomaly is detected,

[0791] A means of presenting analysis results using a data visualization device,

[0792] A system that includes this.

[0793] (Claim 2)

[0794] The system according to claim 1, wherein a generative AI model detects anomalies using a learning algorithm.

[0795] (Claim 3)

[0796] The system according to claim 1, further comprising means for presenting a natural environment conservation plan based on the results of anomaly detection.

[0797] "Application Example 1"

[0798] (Claim 1)

[0799] A device for collecting remote environmental data,

[0800] A device that converts collected environmental data into an analyzable format,

[0801] A device that runs a learning model using converted environmental data and detects abnormal events,

[0802] A device that generates and communicates alarm information when an abnormal event is detected,

[0803] A device that visualizes the analysis results using a geographic information system,

[0804] A system that includes this.

[0805] (Claim 2)

[0806] The system according to claim 1, wherein a learning model detects anomalies using data analysis techniques.

[0807] (Claim 3)

[0808] The system according to claim 1, further comprising means for constructing an environmental protection plan based on anomaly detection results.

[0809] "Example 2 of combining an emotion engine"

[0810] (Claim 1)

[0811] Means for collecting remote information,

[0812] Means for preprocessing the collected information,

[0813] A means for running a learning model using preprocessed information to detect anomalies in the natural environment,

[0814] A means of generating and notifying warning information when an anomaly is detected,

[0815] A means of analyzing user emotions and optimizing responses,

[0816] A means of recommending a natural environment management plan based on the analysis results,

[0817] Means for visualizing analysis results,

[0818] A system that includes this.

[0819] (Claim 2)

[0820] The system according to claim 1, wherein the learning model detects anomalies using an automated learning algorithm.

[0821] (Claim 3)

[0822] The system according to claim 1, further comprising means for adjusting the content of warning information based on the user's emotions.

[0823] "Application example 2 when combining with an emotional engine"

[0824] (Claim 1)

[0825] A means of collecting remote sensor data,

[0826] Means for preprocessing the collected data,

[0827] A means for detecting anomalies in the natural environment by running a predictive model using preprocessed data,

[0828] A means of generating and notifying warning information when an anomaly is detected,

[0829] Means for visualizing the analysis results,

[0830] A means of recognizing the user's emotional state and optimizing the response method according to that emotion,

[0831] A means of emphasizing warnings based on users' sentiments regarding the security situation and guiding them to appropriate countermeasures,

[0832] A system that includes this.

[0833] (Claim 2)

[0834] The system according to claim 1, wherein the predictive model uses a machine learning algorithm to detect anomalies.

[0835] (Claim 3)

[0836] The system according to claim 1, further comprising means for proposing a natural environment management plan based on anomaly detection results and user sentiment. [Explanation of symbols]

[0837] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting remote sensor data, Means for preprocessing the collected data, A means for detecting anomalies in the natural environment by running a predictive model using preprocessed data, A means of generating and notifying warning information when an anomaly is detected, Means for visualizing the analysis results, A system that includes this.

2. The system according to claim 1, wherein the predictive model uses a machine learning algorithm to detect anomalies.

3. The system according to claim 1, further comprising means for proposing a natural environment management plan based on the results of anomaly detection.

Images