system

Abstract

We provide the system. [Solution] A device for acquiring environmental data, A processing device that determines an anomaly based on environmental data acquired by the aforementioned device, A device that notifies abnormal information determined by the aforementioned processing device, The notification device is a device that transmits warning information to an information receiving device located at a distance via communication technology, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, due to the influence of abnormal weather and the like, the importance of temperature control has been increasing. However, for the elderly and those with difficulty in body temperature regulation, it is a difficult task to quickly and appropriately grasp and adjust the temperature. In the prior art, it has been difficult to quickly detect an abnormal temperature in a room and immediately transmit warning information to relevant information recipients in a remote location.

Means for Solving the Problems

[0005] The present invention solves the above problems by providing a system comprising an acquisition means for acquiring environmental information, a processing means for detecting anomalies based on the acquired environmental information, and a notification means for notifying the detected anomaly. This system can analyze acquired temperature data in real time, and when a temperature anomaly is detected, it can issue a localized warning using sound or light, and transmit warning information to remote recipients via communication. This enables a quick and appropriate response and helps reduce associated risks.

[0006] "Environmental information" refers to physical data within a given space, including physical elements such as temperature, humidity, and atmospheric pressure.

[0007] "Acquisition means" refers to a component that has the function of acquiring environmental information using sensors or data collection devices.

[0008] "Anomaly detection" is a process that identifies abnormal conditions by comparing acquired environmental information with predetermined standards and thresholds.

[0009] A "processing means" is a component that has the function of performing data analysis and calculations necessary to implement anomaly detection.

[0010] A "notification means" is a component that has the function of presenting the results of anomaly detection and conveying warnings and information to relevant parties.

[0011] "Communication means" refers to a method or device that uses network communication to transmit abnormal information to a remote location.

[0012] An "information receiving device" is a device that has the function of receiving, processing, and displaying information transmitted through a means of communication. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0019] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] The system based on this invention mainly consists of a sensor device, a server that processes data, and a terminal that handles alarms and notifications. The sensor device periodically measures room environmental information, particularly temperature, and transmits this data to the server. Upon receiving this data, the server performs processing to check whether the temperature is within a preset range. If the temperature is detected to be outside the range, the server determines this to be an anomaly and immediately sends an alarm signal to the terminal.

[0035] When the terminal receives a signal from the server, it activates an alarm using its built-in speaker and lighting to notify the user of the anomaly. Furthermore, it transmits information about the anomaly to family members and related parties living remotely via text messages and application notifications using the internet or telephone lines. In this way, both nearby users and remote parties can simultaneously understand the situation.

[0036] As a concrete example, consider a scenario where the room temperature suddenly rises to 30°C during the daytime in summer. At this time, a sensor measures the temperature and transmits the data to a server. The server compares this temperature to a pre-set appropriate temperature range (for example, 22°C to 28°C), and if it detects an abnormality, it instructs the terminal to activate an alarm. The terminal starts an audible alarm and simultaneously sends a warning message such as "The current temperature is 30°C" to registered family members living far away. The user is immediately aware of the abnormality through the alarm and can take appropriate action, and family members living far away can also check the situation and offer advice. This invention provides a mechanism that can effectively reduce risks related to temperature.

[0037] The following describes the processing flow.

[0038] Step 1:

[0039] A sensor measures the room temperature. The sensor automatically activates every 5 minutes to read the current temperature. This temperature data is converted into a digital signal and sent to a server.

[0040] Step 2:

[0041] The server receives temperature data transmitted from the sensor. The server stores the received data in a database and records it with a time stamp to allow comparison with past data.

[0042] Step 3:

[0043] The server analyzes the temperature data it receives. The server uses a data analysis module to check if the current temperature is within the set threshold range (e.g., 22°C to 28°C). If the temperature is outside the set range, it proceeds to the next step.

[0044] Step 4:

[0045] The server detects an anomaly when a threshold is exceeded. Upon detecting an anomaly, the server records the nature and time of the anomaly and initiates a process to generate an alarm signal.

[0046] Step 5:

[0047] The server sends an alarm signal to the terminal. The server provides instructions to the connected terminal to activate an audio or visual alarm. It also generates a message for remote notification.

[0048] Step 6:

[0049] The device receives an alarm signal and notifies the user. The device activates its built-in speaker and sounds an alarm. In addition, a visual warning is displayed on the screen.

[0050] Step 7:

[0051] The device sends notifications to family members and other relevant parties who live remotely. The device also sends warning messages to registered phone numbers and email addresses via the internet connection. These messages include the current temperature and indicate that an anomaly has occurred.

[0052] Step 8:

[0053] The user receives an alarm, checks it, and takes appropriate action. For example, they might adjust the air conditioner settings or open a window to regulate the room temperature. They can also receive advice from family members remotely and take additional action.

[0054] (Example 1)

[0055] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0056] Conventional environmental monitoring systems have drawbacks, such as delayed anomaly detection and difficulty in real-time notification to relevant parties in remote locations. This can hinder a rapid response to changes in the indoor environment.

[0057] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0058] In this invention, the server includes a measuring means for collecting environmental information, a data processing means for analyzing the data collected by the measuring means and detecting anomalies, and a notification means for notifying the user and relevant parties of the anomaly information detected by the data processing means. This makes it possible to quickly detect environmental anomalies and immediately notify the user and remote locations.

[0059] "Measurement means" refers to technical means for physically collecting environmental information and converting it into data.

[0060] A "data processing means" is a means that has the technical function of analyzing collected data to detect anomalies.

[0061] "Notification means" refers to technical means that use voice, light, and communication means to notify users and relevant parties of detected abnormal information.

[0062] A "physical alarm device" is a device that uses sound and light to signal an anomaly.

[0063] "Network communication means" refers to communication technologies that use the internet or telephone lines to transmit information to remote locations.

[0064] The embodiment for carrying out the invention is a system aimed at monitoring ambient temperature in real time and immediately detecting and notifying of any abnormalities. This system is composed of a sensor device as a measurement means, a server as a data processing means, and a terminal as a notification means.

[0065] The server receives temperature data acquired from the sensor device. Common environmental sensors are used; for example, a typical electronic thermometer is used as the temperature sensor. The server analyzes the data using the Python programming language and detects an anomaly if the temperature exceeds a set range.

[0066] When the terminal receives a signal from the server, it uses its built-in speaker and LEDs to notify the user of the anomaly. The terminal can be a mobile device such as a smartphone or tablet, and an application is run as needed. This application provides sound and light notifications upon receiving a notification. Furthermore, it has the functionality to send messages to remote family members and related parties via the internet or telephone lines.

[0067] As a concrete example, consider a scenario where the indoor temperature suddenly rises to 30°C during the summer. In this case, the sensor sends the data to the server. The server detects that the temperature has exceeded the set safe temperature range (e.g., 22°C to 28°C) and sends an instruction to the terminal to activate an alarm. The terminal can send a message to a registered email address or phone number along with an audio alarm, allowing the user and their associates to immediately understand the change in status and take an appropriate response.

[0068] Examples of prompts used when evaluating this system or suggesting new features using a generative AI model might be questions in the form of, "Please tell me how to optimize the temperature monitoring system." This system provides a temperature monitoring and alarm solution that combines agility and reliability.

[0069] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0070] Step 1:

[0071] The sensor device continuously measures ambient temperature data periodically. For example, it acquires the room temperature every 10 seconds. This measurement result is the input data. The data is output as the latest temperature value and sent to the server. Specifically, the sensor detects the temperature, converts that value into a digital signal, and transmits it to the server via the network.

[0072] Step 2:

[0073] The server receives temperature data transmitted from the sensor. This received data becomes the input. The server analyzes the data using a program and performs data processing to compare it with a set safe temperature range (e.g., 22°C to 28°C). Based on the comparison, it determines whether the data is outside the range, and the result of that determination is obtained as output. Specifically, a Python script is launched, and a process is executed that checks whether the temperature exceeds the range using an if statement.

[0074] Step 3:

[0075] The server generates an alarm signal if the analysis results indicate that the temperature is outside the set range. Here, it performs calculations to create an anomaly detection signal based on the judgment result, and that alarm signal is output. Specifically, the generated signal is sent to the terminal as an HTTP POST request via the REST API.

[0076] Step 4:

[0077] The terminal receives an alarm signal sent from the server. This signal becomes the input data. Based on this signal, the terminal generates an audio alarm through its built-in speaker and flashes an LED light. This is the output from the terminal. Specifically, the application is triggered by the signal, which sounds an alarm and starts providing a visual notification.

[0078] Step 5:

[0079] Users and stakeholders receive a warning message via the network, along with a notification from their device. This transmitted information serves as input data. The application uses this data to send emails and SMS messages, disseminating information to people in remote locations. This is the final output. Specifically, the application on the device sends a message to pre-configured contacts, and the process unfolds to ensure that the notification is distributed.

[0080] (Application Example 1)

[0081] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0082] In the living spaces of the elderly, there is a need for technology that can prevent health risks caused by sudden temperature changes, while also enabling family members and caregivers living far away to immediately understand the situation and take appropriate action. Current systems struggle to quickly detect abnormal temperature changes and provide appropriate notifications, resulting in insufficient real-time information sharing with relevant parties in remote locations. Therefore, improvements are needed to ensure the safety of users.

[0083] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0084] In this invention, the server includes a device for acquiring environmental data, a processing device for determining anomalies based on the environmental data acquired by the device, a device for notifying the anomaly information determined by the processing device, and a device for the notification device to transmit warning information to an information receiving device located at a distance via communication technology. This makes it possible to immediately detect temperature anomalies and notify nearby and distant stakeholders in real time.

[0085] An "environmental data acquisition device" is a sensor device that continuously measures environmental information such as indoor temperature and humidity and supplies that data to a processing device.

[0086] A "processing device for determining abnormalities" is a computer system that compares acquired environmental data with pre-set reference values ​​to identify abnormalities.

[0087] A "notification device" is a device that uses sound or visual means to communicate detected abnormal information to users or relevant parties in remote locations.

[0088] A "device that transmits warning information via communication technology" is a communication device that uses the internet or telephone lines to transmit abnormal information to a receiving device located at a distance.

[0089] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to analyze a situation based on accumulated data and proposes the optimal course of action.

[0090] This system uses temperature sensors as devices to acquire environmental data. The sensors, such as the DHT22, are temperature measuring devices that continuously monitor indoor temperature and humidity, and transmit this data to an AWS® cloud server via a small computer such as a Raspberry Pi. A Python script is executed on the server to compare the received environmental data with a pre-set temperature range to determine if an anomaly is present. If an anomaly is detected, the server generates warning information via a notification device and uses communication technology to send push notifications to nearby information receiving devices and the smartphones of relevant personnel in remote locations. In this process, audible and visual means are used for notifications.

[0091] Furthermore, the server uses a generative AI model to suggest the optimal course of action in the event of an anomaly. The AI ​​model learns from past data and analyzes the current situation to provide a concrete action plan. This result is displayed to users and stakeholders through a smartphone app.

[0092] For example, if the air conditioner malfunctions and the room temperature rises to 32°C, the server will recognize this as an anomaly and, along with a warning such as "Please check the air conditioner," will suggest a quick repair procedure for the air conditioner based on past data. In this case, the input prompt to the generated AI model would be something like, "Please tell me the appropriate course of action when the room temperature exceeds 32°C."

[0093] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0094] Step 1:

[0095] The sensor measures the indoor temperature and humidity. The data acquired by the sensor is input to the Raspberry Pi. The Raspberry Pi collects this data and transfers it to an AWS cloud server for further processing.

[0096] Step 2:

[0097] The server receives temperature and humidity data. The server analyzes this data using a Python script and compares it to a pre-set reference value (for example, a temperature range of 22°C to 28°C). This data comparison process determines whether there is an anomaly and outputs the result.

[0098] Step 3:

[0099] If an anomaly is detected, the server generates a warning message. This message includes details such as the type of anomaly and its location. The generated warning message is transmitted using communication technology to nearby information receiving devices and to the smartphones of users in remote locations.

[0100] Step 4:

[0101] The terminal receives warning information from the server. Based on this information, the terminal either sounds an alarm or displays a visual warning message on the screen. It also displays specific guidelines on the actions that should be taken.

[0102] Step 5:

[0103] The server utilizes a generative AI model to analyze anomalies. This model generates appropriate countermeasures considering past data and the current situation, and provides these to the user as prompt messages. For example, a message such as "Please tell me the appropriate countermeasures when the room temperature exceeds 32°C" might be used, providing appropriate instructions to the user.

[0104] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0105] This invention relates primarily to a system comprising a sensor device, a server for data processing, a terminal responsible for alarms and notifications, and an emotion engine for recognizing user emotions. The sensor device periodically measures environmental information, particularly temperature, and transmits this data to the server. The server receives this data and detects anomalies by comparing it with a set threshold. Furthermore, the emotion engine reads the user's emotional state, allowing the server to combine temperature data and emotion data for more precise anomaly detection.

[0106] The device optimizes alarms for the user, taking into account information from the emotion engine. Based on the emotion analysis results, the device adjusts the volume and content of the alarm, delivering information in a way that reduces user stress. Furthermore, it can send detailed warning messages, including the user's emotional state, to family members living far away, encouraging more appropriate responses.

[0107] As a concrete example, consider a scenario where the room temperature suddenly rises to 30°C, and the user simultaneously displays an anxious expression. A sensor measures the temperature at 30°C and sends it to the server. The server detects an anomaly because it exceeds the normal threshold, and at the same time, the emotion engine recognizes the user's state of anxiety. Based on this, the server quickly sends an alarm signal and emotion data to the terminal. The terminal sounds an alarm in a gentler tone than usual and displays a calm visual warning to reduce stress on the user while still drawing their attention. Additionally, it sends information to family members remotely, along with the abnormal temperature situation, stating that "the user is feeling anxious."

[0108] This system enables flexible responses that take user emotions into consideration, resulting in more appropriate temperature control and support.

[0109] The following describes the processing flow.

[0110] Step 1:

[0111] The sensor measures the room temperature. The sensor automatically activates at regular intervals, reads the current temperature, and sends it to the server.

[0112] Step 2:

[0113] The server receives temperature data transmitted from the sensor. The server records this data in a database and adds a time stamp so that it can be compared with past data.

[0114] Step 3:

[0115] The server analyzes the temperature data and checks if it falls within the set temperature range. If the temperature exceeds the set threshold, a flag is set to detect it as an anomaly.

[0116] Step 4:

[0117] The emotion engine recognizes emotions using the user's facial expressions and voice. It collects data through the user's camera and microphone and analyzes it in real time.

[0118] Step 5:

[0119] The server receives emotional data from the emotion engine. Based on this emotional data, the server determines whether the user is feeling stressed or relaxed.

[0120] Step 6:

[0121] The server integrates temperature and emotion data and, if necessary, detects anomalies. If it determines that the user's emotions are being affected by an abnormal temperature, it generates a warning message.

[0122] Step 7:

[0123] The server sends an alarm signal to the terminal, preparing to notify the user. The alarm signal includes information such as voice tone and message content that is adjusted according to the user's emotions.

[0124] Step 8:

[0125] The device sounds an alarm and displays a visual warning. If the user shows signs of anxiety, it provides a calming voice notification and uses stress-reducing colors for the visual display.

[0126] Step 9:

[0127] The device sends notifications to family members and other relevant parties who live remotely. The messages sent include not only the temperature status but also emotional information such as "The user is feeling anxious." This allows family members in remote locations to provide appropriate support.

[0128] (Example 2)

[0129] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0130] In modern living environments, there is a need to respond immediately to environmental changes and ensure a safe and comfortable life. However, conventional systems only detect environmental changes and lack consideration for the psychological state of individual users. As a result, users may experience psychological stress in addition to the abnormality of the environment. Therefore, there is a need to develop a system that can detect environmental changes and provide optimal responses while also considering the emotional state of the user.

[0131] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0132] In this invention, the server includes a collection means for collecting environmental information, an analysis means for detecting anomalies by comparing the environmental information acquired by the collection means with a threshold set based on that information, and a recognition means for analyzing the user's emotional state and integrating it with the anomaly judgment made by the analysis means. This enables not only the detection of environmental changes but also flexible warnings and notifications that respond to the user's emotional state.

[0133] "Environmental information" refers to data that indicates the physical conditions and changes in a specific space, such as temperature, humidity, and atmospheric pressure.

[0134] "Collection means" refers to a method or device for collecting environmental information using sensor devices or acquisition mechanisms.

[0135] "Analysis means" refers to a processing method or apparatus for analyzing collected environmental information and detecting anomalies based on established criteria.

[0136] A "threshold" is a numerical value or condition that serves as a standard for determining abnormalities in environmental information.

[0137] "Recognition means" refers to a technology or device for analyzing a user's emotional state and integrating the results with other information.

[0138] "Warning generation means" refers to a method or apparatus for creating warning messages or notification content based on recognized information.

[0139] "Notification means" refers to a method or device for transmitting a generated warning or notification to a specific recipient.

[0140] "Communication means" refers to a method or device for transmitting information to different locations through a network or other medium.

[0141] This invention is an integrated system that combines environmental monitoring with the visualization of the user's psychological state, enabling the selection and notification of appropriate alarms. The system consists of sensor devices, a server, terminals, and an emotion engine that analyzes the user's emotional state.

[0142] The sensor device continuously collects environmental information, primarily temperature. For example, it measures room temperature every five minutes and transmits the data digitally using protocols such as HTTP or MQTT. The measured data is transferred to the server along with an accurate timestamp.

[0143] The server is specifically a high-performance computer that compares received temperature data with programmatically set thresholds. If an anomaly is detected, the emotion engine further analyzes the user's emotional state. The emotion engine uses an AI model to determine whether the user is anxious based on image and audio data.

[0144] The device processes alarm signals sent from the server and adjusts the notification method based on the user's emotional data. It modifies the volume and message content according to the emotional analysis results, delivering information in a stress-reducing manner. For example, if it detects both rising temperature and user anxiety simultaneously, it displays a notification in a gentle tone and a calming visual message. It also sends detailed warning messages to the user's family members located remotely, prompting them to take appropriate action regarding the anomaly.

[0145] As a concrete example, consider a situation where the room temperature suddenly rises to 30°C, causing the user to become anxious. In this case, a sensor device measures the temperature at 30°C and transmits it to the server. The server detects the temperature exceeding the threshold as an anomaly, and the emotion engine notifies the user of their anxiety. Based on this, the device provides appropriate notifications through voice and visual means, and remotely communicates information such as "the user is feeling anxious" to family members.

[0146] An example of a prompt for a generating AI model is, "When the user is feeling anxious, what kind of alarm would be effective in response to a rising temperature?" In this way, the system considers both the user's emotions and the environmental situation simultaneously to provide the optimal judgment and response.

[0147] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0148] Step 1:

[0149] The sensor device acquires ambient temperature data. The input is temperature information from the surrounding environment, and the output is temperature data converted into a digital format. Based on this data, the sensor measures the current temperature every 5 minutes, generates time-stamped data, and prepares to send it to the server.

[0150] Step 2:

[0151] The server receives temperature data transmitted from the sensor device. The input is the temperature data sent from the sensor, and the output is timestamped temperature data stored in the database. The server checks the received data and prepares it for data analysis, comparing it to thresholds maintained within the program.

[0152] Step 3:

[0153] The server compares the received temperature data with a set threshold. The input is temperature data stored in the database, and the output is a flag indicating that the data is "abnormal." Specifically, when the temperature exceeds the threshold, the system flags this data as abnormal and generates a trigger to proceed to the next process.

[0154] Step 4:

[0155] The server acquires and analyzes user emotion data. Input is user facial expression and voice data, and output is the identification of the user's emotional state (e.g., anxiety, reassurance). Using an emotion engine, the acquired data is analyzed, and a generative AI model is used to determine the user's emotion.

[0156] Step 5:

[0157] The server integrates anomaly flags and emotion recognition results to generate the optimal warning signal. The inputs are anomaly flags and emotion states, and the output is the warning signal to be communicated to the user. This integration allows the system to determine the most appropriate type and tone of alarm for the user.

[0158] Step 6:

[0159] The terminal receives a warning signal from the server and prepares a notification. The input is the warning signal sent from the server, and the output is the warning information provided to the user through audio or visual means. The terminal uses gentle melodies and visual effects to convey effective attention while minimizing emotional stimuli to the user.

[0160] Step 7:

[0161] The server sends alert information to the user's family. The input is the detailed alert content generated, and the output is the message received by the remote recipient. This includes the user's current emotional state and environmental circumstances, prompting appropriate action.

[0162] (Application Example 2)

[0163] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0164] In modern care settings, there is a need to quickly and accurately understand the user's environmental conditions and emotional state. However, conventional systems only analyze environmental data and emotional state individually, making it difficult to integrate and consider both for flexible responses. Therefore, technology is needed to effectively reduce users' anxiety and stress and provide comfortable care.

[0165] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0166] In this invention, the server includes a sensor means for acquiring environmental information, a data processing means for detecting anomalies based on the acquired environmental information, and an emotion analysis means for recognizing the user's emotional state. This makes it possible to comprehensively analyze environmental data and the user's emotional state and generate optimized notifications.

[0167] "Environmental information" refers to data that quantifies the surrounding physical and meteorological conditions, such as temperature, humidity, and illuminance.

[0168] "Sensing means" refers to devices or equipment used to record environmental information and to acquire data in real time.

[0169] A "data processing means" is a device that analyzes acquired data and has the function of judging, classifying, and transferring information based on established criteria.

[0170] An "emotional analysis tool" is a mechanism that recognizes the emotional state of a user from their facial expressions and voice, and then organizes and evaluates that information.

[0171] A "notification device" is an output device that provides warnings or guidance via visual or auditory means based on detected information.

[0172] "Communication means" refers to the technologies and networks used to perform the process of transferring information to a receiving unit located in a remote location.

[0173] An "information receiving unit" refers to a device that receives, processes, and displays data transmitted via communication means.

[0174] Specific embodiments for carrying out this invention are shown below.

[0175] The server uses multiple sensors to acquire environmental information, collecting data such as temperature, humidity, and illuminance in real time. This collected data is transferred to the server via a communication network. The server uses data processing tools to analyze the received environmental information against predefined criteria and detect anomalies. AWS Lambda can be used as a data analysis platform for this process.

[0176] Furthermore, the server has emotion analysis capabilities and receives the user's facial expression data. This data is acquired by a camera installed in the smart glasses and sent to a cloud service. Here, the Azure® Face API is used to analyze the emotional state from the facial expressions. By integrating and processing the emotional data and environmental data, more accurate and flexible anomaly detection becomes possible.

[0177] The device generates optimized voice and visual warnings based on emotional data combined with detected anomaly information, serving as a notification mechanism. This ensures gentle warnings that are less burdensome for the user. Specifically, smart glasses fulfill this role. The notification method adjusts the volume and color of the light according to the user's emotional state.

[0178] Furthermore, it is possible to use communication methods to transmit detailed warning information to information receiving units in remote locations. In this case, the transmitted information includes data on environmental abnormalities and emotional states, encouraging the user's relatives or caregivers to take prompt action.

[0179] As a concrete example, consider a scenario where the room temperature rises and the user is experiencing stress. In this situation, the server considers emotional and environmental data, optimizes the notification sent to the terminal, and initiates a process of remotely transmitting information. An example of a prompt message might be, "The current room temperature is 30°C and the user is showing signs of anxiety. Please advise on appropriate actions to take."

[0180] This system can provide a more flexible warning system that incorporates the user's psychological state than conventional technologies, supporting a more comfortable life for users in care settings.

[0181] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0182] Step 1:

[0183] The server acquires environmental information such as temperature and humidity in real time using sensory means. This data is transmitted via a communication network from devices such as smart glasses. The input data is acquired as multiple pieces of environmental information and formatted by the server's data processing module. The output is then formatted in a way that allows for comparison with reference values.

[0184] Step 2:

[0185] The server uses data processing to detect anomalies in the previously acquired environmental information. Here, it performs calculations by comparing the acquired data with pre-set thresholds. If an anomaly is detected, the corresponding information is flagged as an anomaly. The output includes whether or not an anomaly was detected and detailed information about it.

[0186] Step 3:

[0187] The server uses emotion analysis tools to acquire the user's facial expression data and recognize their emotional state. This process uses images captured through the smart glasses' camera. It receives facial expression data as input and analyzes it using emotion recognition software (e.g., Azure Face API). The output is data that identifies the user's emotional state.

[0188] Step 4:

[0189] The server integrates and analyzes anomaly information and the user's emotional state to generate optimized notification content. Comprehensive data analysis is performed to determine the most appropriate notification format (a combination of sound and light) for the user. The output consists of notification parameters designed to accurately convey information in a way that does not stress the user.

[0190] Step 5:

[0191] The device delivers optimized notifications through its notification system. The smart glasses adjust the audio and visual effects, issuing warnings in a gentle tone. In doing so, the volume and color of the warning are automatically adjusted based on the user's emotional state. The output is a notification that is considerate of the user's emotional response.

[0192] Step 6:

[0193] The terminal uses communication to transmit detailed warning information to a remote information receiving unit. The transmitted information includes abnormal information and the user's emotional state. This allows family members and caregivers to immediately understand the situation and take appropriate action. The output is a display of the data by the remote information receiving unit.

[0194] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0195] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0196] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0197] [Second Embodiment]

[0198] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0199] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0200] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0201] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0202] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0203] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0204] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0205] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0206] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0207] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0208] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0209] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0210] The system based on this invention mainly consists of a sensor device, a server that processes data, and a terminal that handles alarms and notifications. The sensor device periodically measures room environmental information, particularly temperature, and transmits this data to the server. Upon receiving this data, the server performs processing to check whether the temperature is within a preset range. If the temperature is detected to be outside the range, the server determines this to be an anomaly and immediately sends an alarm signal to the terminal.

[0211] When the terminal receives a signal from the server, it activates an alarm using its built-in speaker and lighting to notify the user of the anomaly. Furthermore, it transmits information about the anomaly to family members and related parties living remotely via text messages and application notifications using the internet or telephone lines. In this way, both nearby users and remote parties can simultaneously understand the situation.

[0212] As a concrete example, consider a scenario where the room temperature suddenly rises to 30°C during the daytime in summer. At this time, a sensor measures the temperature and transmits the data to a server. The server compares this temperature to a pre-set appropriate temperature range (for example, 22°C to 28°C), and if it detects an abnormality, it instructs the terminal to activate an alarm. The terminal starts an audible alarm and simultaneously sends a warning message such as "The current temperature is 30°C" to registered family members living far away. The user is immediately aware of the abnormality through the alarm and can take appropriate action, and family members living far away can also check the situation and offer advice. This invention provides a mechanism that can effectively reduce risks related to temperature.

[0213] The following describes the processing flow.

[0214] Step 1:

[0215] A sensor measures the room temperature. The sensor automatically activates every 5 minutes to read the current temperature. This temperature data is converted into a digital signal and sent to a server.

[0216] Step 2:

[0217] The server receives temperature data transmitted from the sensor. The server stores the received data in a database and records it with a time stamp to allow comparison with past data.

[0218] Step 3:

[0219] The server analyzes the temperature data it receives. The server uses a data analysis module to check if the current temperature is within the set threshold range (e.g., 22°C to 28°C). If the temperature is outside the set range, it proceeds to the next step.

[0220] Step 4:

[0221] The server detects an anomaly when a threshold is exceeded. Upon detecting an anomaly, the server records the nature and time of the anomaly and initiates a process to generate an alarm signal.

[0222] Step 5:

[0223] The server sends an alarm signal to the terminal. The server provides instructions to the connected terminal to activate an audio or visual alarm. It also generates a message for remote notification.

[0224] Step 6:

[0225] The device receives an alarm signal and notifies the user. The device activates its built-in speaker and sounds an alarm. In addition, a visual warning is displayed on the screen.

[0226] Step 7:

[0227] The device sends notifications to family members and other relevant parties who live remotely. The device also sends warning messages to registered phone numbers and email addresses via the internet connection. These messages include the current temperature and indicate that an anomaly has occurred.

[0228] Step 8:

[0229] The user receives an alarm, checks it, and takes appropriate action. For example, they might adjust the air conditioner settings or open a window to regulate the room temperature. They can also receive advice from family members remotely and take additional action.

[0230] (Example 1)

[0231] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0232] Conventional environmental monitoring systems have drawbacks, such as delayed anomaly detection and difficulty in real-time notification to relevant parties in remote locations. This can hinder a rapid response to changes in the indoor environment.

[0233] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0234] In this invention, the server includes a measuring means for collecting environmental information, a data processing means for analyzing the data collected by the measuring means and detecting anomalies, and a notification means for notifying the user and relevant parties of the anomaly information detected by the data processing means. This makes it possible to quickly detect environmental anomalies and immediately notify the user and remote locations.

[0235] "Measurement means" refers to technical means for physically collecting environmental information and converting it into data.

[0236] A "data processing means" is a means that has the technical function of analyzing collected data to detect anomalies.

[0237] "Notification means" refers to technical means that use voice, light, and communication means to notify users and relevant parties of detected abnormal information.

[0238] A "physical alarm device" is a device that uses sound and light to signal an anomaly.

[0239] "Network communication means" refers to communication technologies that use the internet or telephone lines to transmit information to remote locations.

[0240] The embodiment for carrying out the invention is a system aimed at monitoring ambient temperature in real time and immediately detecting and notifying of any abnormalities. This system is composed of a sensor device as a measurement means, a server as a data processing means, and a terminal as a notification means.

[0241] The server receives temperature data acquired from the sensor device. Common environmental sensors are used; for example, a typical electronic thermometer is used as the temperature sensor. The server analyzes the data using the Python programming language and detects an anomaly if the temperature exceeds a set range.

[0242] When the terminal receives a signal from the server, it uses its built-in speaker and LEDs to notify the user of the anomaly. The terminal can be a mobile device such as a smartphone or tablet, and an application is run as needed. This application provides sound and light notifications upon receiving a notification. Furthermore, it has the functionality to send messages to remote family members and related parties via the internet or telephone lines.

[0243] As a concrete example, consider a scenario where the indoor temperature suddenly rises to 30°C during the summer. In this case, the sensor sends the data to the server. The server detects that the temperature has exceeded the set safe temperature range (e.g., 22°C to 28°C) and sends an instruction to the terminal to activate an alarm. The terminal can send a message to a registered email address or phone number along with an audio alarm, allowing the user and their associates to immediately understand the change in status and take an appropriate response.

[0244] Examples of prompts used when evaluating this system or suggesting new features using a generative AI model might be questions in the form of, "Please tell me how to optimize the temperature monitoring system." This system provides a temperature monitoring and alarm solution that combines agility and reliability.

[0245] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0246] Step 1:

[0247] The sensor device continuously measures ambient temperature data periodically. For example, it acquires the room temperature every 10 seconds. This measurement result is the input data. The data is output as the latest temperature value and sent to the server. Specifically, the sensor detects the temperature, converts that value into a digital signal, and transmits it to the server via the network.

[0248] Step 2:

[0249] The server receives temperature data transmitted from the sensor. This received data becomes the input. The server analyzes the data using a program and performs data processing to compare it with a set safe temperature range (e.g., 22°C to 28°C). Based on the comparison, it determines whether the data is outside the range, and the result of that determination is obtained as output. Specifically, a Python script is launched, and a process is executed that checks whether the temperature exceeds the range using an if statement.

[0250] Step 3:

[0251] The server generates an alarm signal if the analysis results indicate that the temperature is outside the set range. Here, it performs calculations to create an anomaly detection signal based on the judgment result, and that alarm signal is output. Specifically, the generated signal is sent to the terminal as an HTTP POST request via the REST API.

[0252] Step 4:

[0253] The terminal receives an alarm signal sent from the server. This signal becomes the input data. Based on this signal, the terminal generates an audio alarm through its built-in speaker and flashes an LED light. This is the output from the terminal. Specifically, the application is triggered by the signal, which sounds an alarm and starts providing a visual notification.

[0254] Step 5:

[0255] Users and stakeholders receive a warning message via the network, along with a notification from their device. This transmitted information serves as input data. The application uses this data to send emails and SMS messages, disseminating information to people in remote locations. This is the final output. Specifically, the application on the device sends a message to pre-configured contacts, and the process unfolds to ensure that the notification is distributed.

[0256] (Application Example 1)

[0257] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0258] In the living spaces of the elderly, there is a need for technology that can prevent health risks caused by sudden temperature changes, while also enabling family members and caregivers living far away to immediately understand the situation and take appropriate action. Current systems struggle to quickly detect abnormal temperature changes and provide appropriate notifications, resulting in insufficient real-time information sharing with relevant parties in remote locations. Therefore, improvements are needed to ensure the safety of users.

[0259] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0260] In this invention, the server includes a device for acquiring environmental data, a processing device for determining anomalies based on the environmental data acquired by the device, a device for notifying the anomaly information determined by the processing device, and a device for the notification device to transmit warning information to an information receiving device located at a distance via communication technology. This makes it possible to immediately detect temperature anomalies and notify nearby and distant stakeholders in real time.

[0261] An "environmental data acquisition device" is a sensor device that continuously measures environmental information such as indoor temperature and humidity and supplies that data to a processing device.

[0262] A "processing device for determining abnormalities" is a computer system that compares acquired environmental data with pre-set reference values ​​to identify abnormalities.

[0263] A "notification device" is a device that uses sound or visual means to communicate detected abnormal information to users or relevant parties in remote locations.

[0264] A "device that transmits warning information via communication technology" is a communication device that uses the internet or telephone lines to transmit abnormal information to a receiving device located at a distance.

[0265] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to analyze a situation based on accumulated data and proposes the optimal course of action.

[0266] This system uses temperature sensors to acquire environmental data. The sensors, such as the DHT22, are temperature measuring devices that continuously monitor indoor temperature and humidity, transmitting the data to an AWS cloud server via a small computer like a Raspberry Pi. A Python script runs on the server, comparing the received environmental data to a pre-defined temperature range to determine if an anomaly is present. If an anomaly is detected, the server generates warning information via a notification device and uses communication technology to send push notifications to nearby information receiving devices and the smartphones of relevant personnel in remote locations. This notification process utilizes both audible and visual means.

[0267] Furthermore, the server uses a generative AI model to suggest the optimal course of action in the event of an anomaly. The AI ​​model learns from past data and analyzes the current situation to provide a concrete action plan. This result is displayed to users and stakeholders through a smartphone app.

[0268] For example, if the air conditioner malfunctions and the room temperature rises to 32°C, the server will recognize this as an anomaly and, along with a warning such as "Please check the air conditioner," will suggest a quick repair procedure for the air conditioner based on past data. In this case, the input prompt to the generated AI model would be something like, "Please tell me the appropriate course of action when the room temperature exceeds 32°C."

[0269] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0270] Step 1:

[0271] The sensor measures the indoor temperature and humidity. The data acquired by the sensor is input to the Raspberry Pi. The Raspberry Pi collects this data and transfers it to an AWS cloud server for further processing.

[0272] Step 2:

[0273] The server receives temperature and humidity data. The server analyzes this data using a Python script and compares it to a pre-set reference value (for example, a temperature range of 22°C to 28°C). This data comparison process determines whether there is an anomaly and outputs the result.

[0274] Step 3:

[0275] If an anomaly is detected, the server generates a warning message. This message includes details such as the type of anomaly and its location. The generated warning message is transmitted using communication technology to nearby information receiving devices and to the smartphones of users in remote locations.

[0276] Step 4:

[0277] The terminal receives warning information from the server. Based on this information, the terminal either sounds an alarm or displays a visual warning message on the screen. It also displays specific guidelines on the actions that should be taken.

[0278] Step 5:

[0279] The server utilizes a generative AI model to analyze anomalies. This model generates appropriate countermeasures considering past data and the current situation, and provides these to the user as prompt messages. For example, a message such as "Please tell me the appropriate countermeasures when the room temperature exceeds 32°C" might be used, providing appropriate instructions to the user.

[0280] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0281] This invention relates primarily to a system comprising a sensor device, a server for data processing, a terminal responsible for alarms and notifications, and an emotion engine for recognizing user emotions. The sensor device periodically measures environmental information, particularly temperature, and transmits this data to the server. The server receives this data and detects anomalies by comparing it with a set threshold. Furthermore, the emotion engine reads the user's emotional state, allowing the server to combine temperature data and emotion data for more precise anomaly detection.

[0282] The device optimizes alarms for the user, taking into account information from the emotion engine. Based on the emotion analysis results, the device adjusts the volume and content of the alarm, delivering information in a way that reduces user stress. Furthermore, it can send detailed warning messages, including the user's emotional state, to family members living far away, encouraging more appropriate responses.

[0283] As a specific example, consider a case where the temperature in a room suddenly rises to 30°C, and at the same time, the user has a worried expression. The sensor measures the temperature of 30°C and transmits it to the server. Since the server has exceeded the normal threshold, it detects an abnormality. At the same time, the emotion engine recognizes the user's anxious state. Based on this, the server promptly transmits an alarm signal and emotion data to the terminal. The terminal sounds the alarm in a gentler tone than usual and displays a visually mild warning to prompt the user while reducing stress on the user. In addition, for remote family members, it transmits additional information such as "The user is feeling anxious" along with the abnormal temperature situation.

[0284] This system enables flexible responses considering the user's emotions and can achieve more appropriate temperature management and support.

[0285] The following describes the processing flow.

[0286] Step 1:

[0287] The sensor measures the room temperature. The sensor automatically activates at regular time intervals, reads the current temperature, and transmits it to the server.

[0288] Step 2:

[0289] The server receives the temperature data transmitted from the sensor. The server records this data in the database and adds a time stamp so that it can be compared with past data.

[0290] Step 3:

[0291] The server analyzes the temperature data and checks if it is within the set temperature range. If the temperature exceeds the set threshold, a flag for detecting an abnormality is set.

[0292] Step 4:

[0293] The emotion engine recognizes emotions using the user's facial expressions and voice. It collects data through the user's camera and microphone and analyzes it in real time.

[0294] Step 5:

[0295] The server receives emotional data from the emotion engine. Based on this emotional data, the server determines whether the user is feeling stressed or relaxed.

[0296] Step 6:

[0297] The server integrates temperature and emotion data and, if necessary, detects anomalies. If it determines that the user's emotions are being affected by an abnormal temperature, it generates a warning message.

[0298] Step 7:

[0299] The server sends an alarm signal to the terminal, preparing to notify the user. The alarm signal includes information such as voice tone and message content that is adjusted according to the user's emotions.

[0300] Step 8:

[0301] The device sounds an alarm and displays a visual warning. If the user shows signs of anxiety, it provides a calming voice notification and uses stress-reducing colors for the visual display.

[0302] Step 9:

[0303] The device sends notifications to family members and other relevant parties who live remotely. The messages sent include not only the temperature status but also emotional information such as "The user is feeling anxious." This allows family members in remote locations to provide appropriate support.

[0304] (Example 2)

[0305] Next, Example 2 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart glasses 214 are referred to as a "terminal".

[0306] In a modern living environment, it is required to immediately respond to environmental changes and ensure a safe and comfortable life. However, conventional systems only sense environmental changes and lack responses that take into account the psychological states of individual users. For this reason, users may feel psychological stress in addition to environmental abnormalities. Therefore, it is necessary to develop a system that senses environmental changes and further makes optimal responses considering the emotional states of users.

[0307] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following respective means.

[0308] In this invention, the server includes a collection means for collecting environmental information, an analysis means for detecting an abnormality by comparing with a threshold value set based on the environmental information acquired by the collection means, and a recognition means for analyzing the emotional state of the user and integrating it with the abnormality determination by the analysis means. Thereby, not only can environmental changes be detected, but also flexible warnings and notifications according to the emotional state of the user become possible.

[0309] "Environmental information" is data indicating the physical state and changes in a specific space, such as temperature, humidity, and atmospheric pressure.

[0310] "Collection means" is a method or device for collecting environmental information using a sensor device or an acquisition mechanism.

[0311] "Analysis means" is a processing method or device for analyzing the collected environmental information and detecting an abnormality based on a set criterion.

[0312] "Threshold value" is a numerical value or condition serving as a criterion for determining an abnormality in environmental information.

[0313] "Recognition means" refers to a technology or device for analyzing a user's emotional state and integrating the results with other information.

[0314] "Warning generation means" refers to a method or apparatus for creating warning messages or notification content based on recognized information.

[0315] "Notification means" refers to a method or device for transmitting a generated warning or notification to a specific recipient.

[0316] "Communication means" refers to a method or device for transmitting information to different locations through a network or other medium.

[0317] This invention is an integrated system that combines environmental monitoring with the visualization of the user's psychological state, enabling the selection and notification of appropriate alarms. The system consists of sensor devices, a server, terminals, and an emotion engine that analyzes the user's emotional state.

[0318] The sensor device continuously collects environmental information, primarily temperature. For example, it measures room temperature every five minutes and transmits the data digitally using protocols such as HTTP or MQTT. The measured data is transferred to the server along with an accurate timestamp.

[0319] The server is specifically a high-performance computer that compares received temperature data with programmatically set thresholds. If an anomaly is detected, the emotion engine further analyzes the user's emotional state. The emotion engine uses an AI model to determine whether the user is anxious based on image and audio data.

[0320] The device processes alarm signals sent from the server and adjusts the notification method based on the user's emotional data. It modifies the volume and message content according to the emotional analysis results, delivering information in a stress-reducing manner. For example, if it detects both rising temperature and user anxiety simultaneously, it displays a notification in a gentle tone and a calming visual message. It also sends detailed warning messages to the user's family members located remotely, prompting them to take appropriate action regarding the anomaly.

[0321] As a concrete example, consider a situation where the room temperature suddenly rises to 30°C, causing the user to become anxious. In this case, a sensor device measures the temperature at 30°C and transmits it to the server. The server detects the temperature exceeding the threshold as an anomaly, and the emotion engine notifies the user of their anxiety. Based on this, the device provides appropriate notifications through voice and visual means, and remotely communicates information such as "the user is feeling anxious" to family members.

[0322] An example of a prompt for a generating AI model is, "When the user is feeling anxious, what kind of alarm would be effective in response to a rising temperature?" In this way, the system considers both the user's emotions and the environmental situation simultaneously to provide the optimal judgment and response.

[0323] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0324] Step 1:

[0325] The sensor device acquires ambient temperature data. The input is temperature information from the surrounding environment, and the output is temperature data converted into a digital format. Based on this data, the sensor measures the current temperature every 5 minutes, generates time-stamped data, and prepares to send it to the server.

[0326] Step 2:

[0327] The server receives temperature data transmitted from the sensor device. The input is the temperature data sent from the sensor, and the output is timestamped temperature data stored in the database. The server checks the received data and prepares it for data analysis, comparing it to thresholds maintained within the program.

[0328] Step 3:

[0329] The server compares the received temperature data with a set threshold. The input is temperature data stored in the database, and the output is a flag indicating that the data is "abnormal." Specifically, when the temperature exceeds the threshold, the system flags this data as abnormal and generates a trigger to proceed to the next process.

[0330] Step 4:

[0331] The server acquires and analyzes user emotion data. Input is user facial expression and voice data, and output is the identification of the user's emotional state (e.g., anxiety, reassurance). Using an emotion engine, the acquired data is analyzed, and a generative AI model is used to determine the user's emotion.

[0332] Step 5:

[0333] The server integrates anomaly flags and emotion recognition results to generate the optimal warning signal. The inputs are anomaly flags and emotion states, and the output is the warning signal to be communicated to the user. This integration allows the system to determine the most appropriate type and tone of alarm for the user.

[0334] Step 6:

[0335] The terminal receives a warning signal from the server and prepares a notification. The input is the warning signal sent from the server, and the output is the warning information provided to the user through audio or visual means. The terminal uses gentle melodies and visual effects to convey effective attention while minimizing emotional stimuli to the user.

[0336] Step 7:

[0337] The server sends alert information to the user's family. The input is the detailed alert content generated, and the output is the message received by the remote recipient. This includes the user's current emotional state and environmental circumstances, prompting appropriate action.

[0338] (Application Example 2)

[0339] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0340] In modern care settings, there is a need to quickly and accurately understand the user's environmental conditions and emotional state. However, conventional systems only analyze environmental data and emotional state individually, making it difficult to integrate and consider both for flexible responses. Therefore, technology is needed to effectively reduce users' anxiety and stress and provide comfortable care.

[0341] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0342] In this invention, the server includes a sensor means for acquiring environmental information, a data processing means for detecting anomalies based on the acquired environmental information, and an emotion analysis means for recognizing the user's emotional state. This makes it possible to comprehensively analyze environmental data and the user's emotional state and generate optimized notifications.

[0343] "Environmental information" refers to data that quantifies the surrounding physical and meteorological conditions, such as temperature, humidity, and illuminance.

[0344] "Sensing means" refers to devices or equipment used to record environmental information and to acquire data in real time.

[0345] A "data processing means" is a device that analyzes acquired data and has the function of judging, classifying, and transferring information based on established criteria.

[0346] An "emotional analysis tool" is a mechanism that recognizes the emotional state of a user from their facial expressions and voice, and then organizes and evaluates that information.

[0347] A "notification device" is an output device that provides warnings or guidance via visual or auditory means based on detected information.

[0348] "Communication means" refers to the technologies and networks used to perform the process of transferring information to a receiving unit located in a remote location.

[0349] An "information receiving unit" refers to a device that receives, processes, and displays data transmitted via communication means.

[0350] Specific embodiments for carrying out this invention are shown below.

[0351] The server uses multiple sensors to acquire environmental information, collecting data such as temperature, humidity, and illuminance in real time. This collected data is transferred to the server via a communication network. The server uses data processing tools to analyze the received environmental information against predefined criteria and detect anomalies. AWS Lambda can be used as a data analysis platform for this process.

[0352] Furthermore, the server has emotion analysis capabilities and receives the user's facial expression data. This data is acquired by a camera installed in the smart glasses and sent to a cloud service. Here, the Azure Face API is used to analyze the emotional state from the facial expressions. By integrating and processing the emotional data and environmental data, more accurate and flexible anomaly detection becomes possible.

[0353] The device generates optimized voice and visual warnings based on emotional data combined with detected anomaly information, serving as a notification mechanism. This ensures gentle warnings that are less burdensome for the user. Specifically, smart glasses fulfill this role. The notification method adjusts the volume and color of the light according to the user's emotional state.

[0354] Furthermore, it is possible to use communication methods to transmit detailed warning information to information receiving units in remote locations. In this case, the transmitted information includes data on environmental abnormalities and emotional states, encouraging the user's relatives or caregivers to take prompt action.

[0355] As a concrete example, consider a scenario where the room temperature rises and the user is experiencing stress. In this situation, the server considers emotional and environmental data, optimizes the notification sent to the terminal, and initiates a process of remotely transmitting information. An example of a prompt message might be, "The current room temperature is 30°C and the user is showing signs of anxiety. Please advise on appropriate actions to take."

[0356] This system can provide a more flexible warning system that incorporates the user's psychological state than conventional technologies, supporting a more comfortable life for users in care settings.

[0357] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0358] Step 1:

[0359] The server acquires environmental information such as temperature and humidity in real time using sensory means. This data is transmitted via a communication network from devices such as smart glasses. The input data is acquired as multiple pieces of environmental information and formatted by the server's data processing module. The output is then formatted in a way that allows for comparison with reference values.

[0360] Step 2:

[0361] The server uses data processing to detect anomalies in the previously acquired environmental information. Here, it performs calculations by comparing the acquired data with pre-set thresholds. If an anomaly is detected, the corresponding information is flagged as an anomaly. The output includes whether or not an anomaly was detected and detailed information about it.

[0362] Step 3:

[0363] The server uses emotion analysis tools to acquire the user's facial expression data and recognize their emotional state. This process uses images captured through the smart glasses' camera. It receives facial expression data as input and analyzes it using emotion recognition software (e.g., Azure Face API). The output is data that identifies the user's emotional state.

[0364] Step 4:

[0365] The server integrates and analyzes anomaly information and the user's emotional state to generate optimized notification content. Comprehensive data analysis is performed to determine the most appropriate notification format (a combination of sound and light) for the user. The output consists of notification parameters designed to accurately convey information in a way that does not stress the user.

[0366] Step 5:

[0367] The device delivers optimized notifications through its notification system. The smart glasses adjust the audio and visual effects, issuing warnings in a gentle tone. In doing so, the volume and color of the warning are automatically adjusted based on the user's emotional state. The output is a notification that is considerate of the user's emotional response.

[0368] Step 6:

[0369] The terminal uses communication to transmit detailed warning information to a remote information receiving unit. The transmitted information includes abnormal information and the user's emotional state. This allows family members and caregivers to immediately understand the situation and take appropriate action. The output is a display of the data by the remote information receiving unit.

[0370] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0371] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0372] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0373] [Third Embodiment]

[0374] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0375] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0376] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0377] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0378] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0379] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0380] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0381] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0382] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0383] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0384] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0385] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0386] The system based on this invention mainly consists of a sensor device, a server that processes data, and a terminal that handles alarms and notifications. The sensor device periodically measures room environmental information, particularly temperature, and transmits this data to the server. Upon receiving this data, the server performs processing to check whether the temperature is within a preset range. If the temperature is detected to be outside the range, the server determines this to be an anomaly and immediately sends an alarm signal to the terminal.

[0387] When the terminal receives a signal from the server, it activates an alarm using its built-in speaker and lighting to notify the user of the anomaly. Furthermore, it transmits information about the anomaly to family members and related parties living remotely via text messages and application notifications using the internet or telephone lines. In this way, both nearby users and remote parties can simultaneously understand the situation.

[0388] As a concrete example, consider a scenario where the room temperature suddenly rises to 30°C during the daytime in summer. At this time, a sensor measures the temperature and transmits the data to a server. The server compares this temperature to a pre-set appropriate temperature range (for example, 22°C to 28°C), and if it detects an abnormality, it instructs the terminal to activate an alarm. The terminal starts an audible alarm and simultaneously sends a warning message such as "The current temperature is 30°C" to registered family members living far away. The user is immediately aware of the abnormality through the alarm and can take appropriate action, and family members living far away can also check the situation and offer advice. This invention provides a mechanism that can effectively reduce risks related to temperature.

[0389] The following describes the processing flow.

[0390] Step 1:

[0391] A sensor measures the room temperature. The sensor automatically activates every 5 minutes to read the current temperature. This temperature data is converted into a digital signal and sent to a server.

[0392] Step 2:

[0393] The server receives temperature data transmitted from the sensor. The server stores the received data in a database and records it with a time stamp to allow comparison with past data.

[0394] Step 3:

[0395] The server analyzes the temperature data it receives. The server uses a data analysis module to check if the current temperature is within the set threshold range (e.g., 22°C to 28°C). If the temperature is outside the set range, it proceeds to the next step.

[0396] Step 4:

[0397] The server detects an anomaly when a threshold is exceeded. Upon detecting an anomaly, the server records the nature and time of the anomaly and initiates a process to generate an alarm signal.

[0398] Step 5:

[0399] The server sends an alarm signal to the terminal. The server provides instructions to the connected terminal to activate an audio or visual alarm. It also generates a message for remote notification.

[0400] Step 6:

[0401] The device receives an alarm signal and notifies the user. The device activates its built-in speaker and sounds an alarm. In addition, a visual warning is displayed on the screen.

[0402] Step 7:

[0403] The device sends notifications to family members and other relevant parties who live remotely. The device also sends warning messages to registered phone numbers and email addresses via the internet connection. These messages include the current temperature and indicate that an anomaly has occurred.

[0404] Step 8:

[0405] The user receives an alarm, checks it, and takes appropriate action. For example, they might adjust the air conditioner settings or open a window to regulate the room temperature. They can also receive advice from family members remotely and take additional action.

[0406] (Example 1)

[0407] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0408] Conventional environmental monitoring systems have drawbacks, such as delayed anomaly detection and difficulty in real-time notification to relevant parties in remote locations. This can hinder a rapid response to changes in the indoor environment.

[0409] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0410] In this invention, the server includes a measuring means for collecting environmental information, a data processing means for analyzing the data collected by the measuring means and detecting anomalies, and a notification means for notifying the user and relevant parties of the anomaly information detected by the data processing means. This makes it possible to quickly detect environmental anomalies and immediately notify the user and remote locations.

[0411] "Measurement means" refers to technical means for physically collecting environmental information and converting it into data.

[0412] A "data processing means" is a means that has the technical function of analyzing collected data to detect anomalies.

[0413] "Notification means" refers to technical means that use voice, light, and communication means to notify users and relevant parties of detected abnormal information.

[0414] A "physical alarm device" is a device that uses sound and light to signal an anomaly.

[0415] "Network communication means" refers to communication technologies that use the internet or telephone lines to transmit information to remote locations.

[0416] The embodiment for carrying out the invention is a system aimed at monitoring ambient temperature in real time and immediately detecting and notifying of any abnormalities. This system is composed of a sensor device as a measurement means, a server as a data processing means, and a terminal as a notification means.

[0417] The server receives temperature data acquired from the sensor device. Common environmental sensors are used; for example, a typical electronic thermometer is used as the temperature sensor. The server analyzes the data using the Python programming language and detects an anomaly if the temperature exceeds a set range.

[0418] When the terminal receives a signal from the server, it uses its built-in speaker and LEDs to notify the user of the anomaly. The terminal can be a mobile device such as a smartphone or tablet, and an application is run as needed. This application provides sound and light notifications upon receiving a notification. Furthermore, it has the functionality to send messages to remote family members and related parties via the internet or telephone lines.

[0419] As a concrete example, consider a scenario where the indoor temperature suddenly rises to 30°C during the summer. In this case, the sensor sends the data to the server. The server detects that the temperature has exceeded the set safe temperature range (e.g., 22°C to 28°C) and sends an instruction to the terminal to activate an alarm. The terminal can send a message to a registered email address or phone number along with an audio alarm, allowing the user and their associates to immediately understand the change in status and take an appropriate response.

[0420] Examples of prompts used when evaluating this system or suggesting new features using a generative AI model might be questions in the form of, "Please tell me how to optimize the temperature monitoring system." This system provides a temperature monitoring and alarm solution that combines agility and reliability.

[0421] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0422] Step 1:

[0423] The sensor device continuously measures ambient temperature data periodically. For example, it acquires the room temperature every 10 seconds. This measurement result is the input data. The data is output as the latest temperature value and sent to the server. Specifically, the sensor detects the temperature, converts that value into a digital signal, and transmits it to the server via the network.

[0424] Step 2:

[0425] The server receives temperature data transmitted from the sensor. This received data becomes the input. The server analyzes the data using a program and performs data processing to compare it with a set safe temperature range (e.g., 22°C to 28°C). Based on the comparison, it determines whether the data is outside the range, and the result of that determination is obtained as output. Specifically, a Python script is launched, and a process is executed that checks whether the temperature exceeds the range using an if statement.

[0426] Step 3:

[0427] The server generates an alarm signal if the analysis results indicate that the temperature is outside the set range. Here, it performs calculations to create an anomaly detection signal based on the judgment result, and that alarm signal is output. Specifically, the generated signal is sent to the terminal as an HTTP POST request via the REST API.

[0428] Step 4:

[0429] The terminal receives an alarm signal sent from the server. This signal becomes the input data. Based on this signal, the terminal generates an audio alarm through its built-in speaker and flashes an LED light. This is the output from the terminal. Specifically, the application is triggered by the signal, which sounds an alarm and starts providing a visual notification.

[0430] Step 5:

[0431] Users and stakeholders receive a warning message via the network, along with a notification from their device. This transmitted information serves as input data. The application uses this data to send emails and SMS messages, disseminating information to people in remote locations. This is the final output. Specifically, the application on the device sends a message to pre-configured contacts, and the process unfolds to ensure that the notification is distributed.

[0432] (Application Example 1)

[0433] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0434] In the living spaces of the elderly, there is a need for technology that can prevent health risks caused by sudden temperature changes, while also enabling family members and caregivers living far away to immediately understand the situation and take appropriate action. Current systems struggle to quickly detect abnormal temperature changes and provide appropriate notifications, resulting in insufficient real-time information sharing with relevant parties in remote locations. Therefore, improvements are needed to ensure the safety of users.

[0435] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0436] In this invention, the server includes a device for acquiring environmental data, a processing device for determining anomalies based on the environmental data acquired by the device, a device for notifying the anomaly information determined by the processing device, and a device for the notification device to transmit warning information to an information receiving device located at a distance via communication technology. This makes it possible to immediately detect temperature anomalies and notify nearby and distant stakeholders in real time.

[0437] An "environmental data acquisition device" is a sensor device that continuously measures environmental information such as indoor temperature and humidity and supplies that data to a processing device.

[0438] A "processing device for determining abnormalities" is a computer system that compares acquired environmental data with pre-set reference values ​​to identify abnormalities.

[0439] A "notification device" is a device that uses sound or visual means to communicate detected abnormal information to users or relevant parties in remote locations.

[0440] A "device that transmits warning information via communication technology" is a communication device that uses the internet or telephone lines to transmit abnormal information to a receiving device located at a distance.

[0441] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to analyze a situation based on accumulated data and proposes the optimal course of action.

[0442] This system uses temperature sensors to acquire environmental data. The sensors, such as the DHT22, are temperature measuring devices that continuously monitor indoor temperature and humidity, transmitting the data to an AWS cloud server via a small computer like a Raspberry Pi. A Python script runs on the server, comparing the received environmental data to a pre-defined temperature range to determine if an anomaly is present. If an anomaly is detected, the server generates warning information via a notification device and uses communication technology to send push notifications to nearby information receiving devices and the smartphones of relevant personnel in remote locations. This notification process utilizes both audible and visual means.

[0443] Furthermore, the server uses a generative AI model to suggest the optimal course of action in the event of an anomaly. The AI ​​model learns from past data and analyzes the current situation to provide a concrete action plan. This result is displayed to users and stakeholders through a smartphone app.

[0444] For example, if the air conditioner malfunctions and the room temperature rises to 32°C, the server will recognize this as an anomaly and, along with a warning such as "Please check the air conditioner," will suggest a quick repair procedure for the air conditioner based on past data. In this case, the input prompt to the generated AI model would be something like, "Please tell me the appropriate course of action when the room temperature exceeds 32°C."

[0445] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0446] Step 1:

[0447] The sensor measures the indoor temperature and humidity. The data acquired by the sensor is input to the Raspberry Pi. The Raspberry Pi collects this data and transfers it to an AWS cloud server for further processing.

[0448] Step 2:

[0449] The server receives temperature and humidity data. The server analyzes this data using a Python script and compares it to a pre-set reference value (for example, a temperature range of 22°C to 28°C). This data comparison process determines whether there is an anomaly and outputs the result.

[0450] Step 3:

[0451] If an anomaly is detected, the server generates a warning message. This message includes details such as the type of anomaly and its location. The generated warning message is transmitted using communication technology to nearby information receiving devices and to the smartphones of users in remote locations.

[0452] Step 4:

[0453] The terminal receives warning information from the server. Based on this information, the terminal either sounds an alarm or displays a visual warning message on the screen. It also displays specific guidelines on the actions that should be taken.

[0454] Step 5:

[0455] The server utilizes a generative AI model to analyze anomalies. This model generates appropriate countermeasures considering past data and the current situation, and provides these to the user as prompt messages. For example, a message such as "Please tell me the appropriate countermeasures when the room temperature exceeds 32°C" might be used, providing appropriate instructions to the user.

[0456] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0457] This invention relates primarily to a system comprising a sensor device, a server for data processing, a terminal responsible for alarms and notifications, and an emotion engine for recognizing user emotions. The sensor device periodically measures environmental information, particularly temperature, and transmits this data to the server. The server receives this data and detects anomalies by comparing it with a set threshold. Furthermore, the emotion engine reads the user's emotional state, allowing the server to combine temperature data and emotion data for more precise anomaly detection.

[0458] The device optimizes alarms for the user, taking into account information from the emotion engine. Based on the emotion analysis results, the device adjusts the volume and content of the alarm, delivering information in a way that reduces user stress. Furthermore, it can send detailed warning messages, including the user's emotional state, to family members living far away, encouraging more appropriate responses.

[0459] As a concrete example, consider a scenario where the room temperature suddenly rises to 30°C, and the user simultaneously displays an anxious expression. A sensor measures the temperature at 30°C and sends it to the server. The server detects an anomaly because it exceeds the normal threshold, and at the same time, the emotion engine recognizes the user's state of anxiety. Based on this, the server quickly sends an alarm signal and emotion data to the terminal. The terminal sounds an alarm in a gentler tone than usual and displays a calm visual warning to reduce stress on the user while still drawing their attention. Additionally, it sends information to family members remotely, along with the abnormal temperature situation, stating that "the user is feeling anxious."

[0460] This system enables flexible responses that take user emotions into consideration, resulting in more appropriate temperature control and support.

[0461] The following describes the processing flow.

[0462] Step 1:

[0463] The sensor measures the room temperature. The sensor automatically activates at regular intervals, reads the current temperature, and sends it to the server.

[0464] Step 2:

[0465] The server receives temperature data transmitted from the sensor. The server records this data in a database and adds a time stamp so that it can be compared with past data.

[0466] Step 3:

[0467] The server analyzes the temperature data and checks if it falls within the set temperature range. If the temperature exceeds the set threshold, a flag is set to detect it as an anomaly.

[0468] Step 4:

[0469] The emotion engine recognizes emotions using the user's facial expressions and voice. It collects data through the user's camera and microphone and analyzes it in real time.

[0470] Step 5:

[0471] The server receives emotional data from the emotion engine. Based on this emotional data, the server determines whether the user is feeling stressed or relaxed.

[0472] Step 6:

[0473] The server integrates temperature and emotion data and, if necessary, detects anomalies. If it determines that the user's emotions are being affected by an abnormal temperature, it generates a warning message.

[0474] Step 7:

[0475] The server sends an alarm signal to the terminal, preparing to notify the user. The alarm signal includes information such as voice tone and message content that is adjusted according to the user's emotions.

[0476] Step 8:

[0477] The device sounds an alarm and displays a visual warning. If the user shows signs of anxiety, it provides a calming voice notification and uses stress-reducing colors for the visual display.

[0478] Step 9:

[0479] The device sends notifications to family members and other relevant parties who live remotely. The messages sent include not only the temperature status but also emotional information such as "The user is feeling anxious." This allows family members in remote locations to provide appropriate support.

[0480] (Example 2)

[0481] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0482] In modern living environments, there is a need to respond immediately to environmental changes and ensure a safe and comfortable life. However, conventional systems only detect environmental changes and lack consideration for the psychological state of individual users. As a result, users may experience psychological stress in addition to the abnormality of the environment. Therefore, there is a need to develop a system that can detect environmental changes and provide optimal responses while also considering the emotional state of the user.

[0483] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0484] In this invention, the server includes a collection means for collecting environmental information, an analysis means for detecting anomalies by comparing the environmental information acquired by the collection means with a threshold set based on that information, and a recognition means for analyzing the user's emotional state and integrating it with the anomaly judgment made by the analysis means. This enables not only the detection of environmental changes but also flexible warnings and notifications that respond to the user's emotional state.

[0485] "Environmental information" refers to data that indicates the physical conditions and changes in a specific space, such as temperature, humidity, and atmospheric pressure.

[0486] "Collection means" refers to a method or device for collecting environmental information using sensor devices or acquisition mechanisms.

[0487] "Analysis means" refers to a processing method or apparatus for analyzing collected environmental information and detecting anomalies based on established criteria.

[0488] A "threshold" is a numerical value or condition that serves as a standard for determining abnormalities in environmental information.

[0489] "Recognition means" refers to a technology or device for analyzing a user's emotional state and integrating the results with other information.

[0490] "Warning generation means" refers to a method or apparatus for creating warning messages or notification content based on recognized information.

[0491] "Notification means" refers to a method or device for transmitting a generated warning or notification to a specific recipient.

[0492] "Communication means" refers to a method or device for transmitting information to different locations through a network or other medium.

[0493] This invention is an integrated system that combines environmental monitoring with the visualization of the user's psychological state, enabling the selection and notification of appropriate alarms. The system consists of sensor devices, a server, terminals, and an emotion engine that analyzes the user's emotional state.

[0494] The sensor device continuously collects environmental information, primarily temperature. For example, it measures room temperature every five minutes and transmits the data digitally using protocols such as HTTP or MQTT. The measured data is transferred to the server along with an accurate timestamp.

[0495] The server is specifically a high-performance computer that compares received temperature data with programmatically set thresholds. If an anomaly is detected, the emotion engine further analyzes the user's emotional state. The emotion engine uses an AI model to determine whether the user is anxious based on image and audio data.

[0496] The device processes alarm signals sent from the server and adjusts the notification method based on the user's emotional data. It modifies the volume and message content according to the emotional analysis results, delivering information in a stress-reducing manner. For example, if it detects both rising temperature and user anxiety simultaneously, it displays a notification in a gentle tone and a calming visual message. It also sends detailed warning messages to the user's family members located remotely, prompting them to take appropriate action regarding the anomaly.

[0497] As a concrete example, consider a situation where the room temperature suddenly rises to 30°C, causing the user to become anxious. In this case, a sensor device measures the temperature at 30°C and transmits it to the server. The server detects the temperature exceeding the threshold as an anomaly, and the emotion engine notifies the user of their anxiety. Based on this, the device provides appropriate notifications through voice and visual means, and remotely communicates information such as "the user is feeling anxious" to family members.

[0498] An example of a prompt for a generating AI model is, "When the user is feeling anxious, what kind of alarm would be effective in response to a rising temperature?" In this way, the system considers both the user's emotions and the environmental situation simultaneously to provide the optimal judgment and response.

[0499] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0500] Step 1:

[0501] The sensor device acquires ambient temperature data. The input is temperature information from the surrounding environment, and the output is temperature data converted into a digital format. Based on this data, the sensor measures the current temperature every 5 minutes, generates time-stamped data, and prepares to send it to the server.

[0502] Step 2:

[0503] The server receives temperature data transmitted from the sensor device. The input is the temperature data sent from the sensor, and the output is timestamped temperature data stored in the database. The server checks the received data and prepares it for data analysis, comparing it to thresholds maintained within the program.

[0504] Step 3:

[0505] The server compares the received temperature data with a set threshold. The input is temperature data stored in the database, and the output is a flag indicating that the data is "abnormal." Specifically, when the temperature exceeds the threshold, the system flags this data as abnormal and generates a trigger to proceed to the next process.

[0506] Step 4:

[0507] The server acquires and analyzes user emotion data. Input is user facial expression and voice data, and output is the identification of the user's emotional state (e.g., anxiety, reassurance). Using an emotion engine, the acquired data is analyzed, and a generative AI model is used to determine the user's emotion.

[0508] Step 5:

[0509] The server integrates anomaly flags and emotion recognition results to generate the optimal warning signal. The inputs are anomaly flags and emotion states, and the output is the warning signal to be communicated to the user. This integration allows the system to determine the most appropriate type and tone of alarm for the user.

[0510] Step 6:

[0511] The terminal receives a warning signal from the server and prepares a notification. The input is the warning signal sent from the server, and the output is the warning information provided to the user through audio or visual means. The terminal uses gentle melodies and visual effects to convey effective attention while minimizing emotional stimuli to the user.

[0512] Step 7:

[0513] The server sends alert information to the user's family. The input is the detailed alert content generated, and the output is the message received by the remote recipient. This includes the user's current emotional state and environmental circumstances, prompting appropriate action.

[0514] (Application Example 2)

[0515] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0516] In modern care settings, there is a need to quickly and accurately understand the user's environmental conditions and emotional state. However, conventional systems only analyze environmental data and emotional state individually, making it difficult to integrate and consider both for flexible responses. Therefore, technology is needed to effectively reduce users' anxiety and stress and provide comfortable care.

[0517] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0518] In this invention, the server includes a sensor means for acquiring environmental information, a data processing means for detecting anomalies based on the acquired environmental information, and an emotion analysis means for recognizing the user's emotional state. This makes it possible to comprehensively analyze environmental data and the user's emotional state and generate optimized notifications.

[0519] "Environmental information" refers to data that quantifies the surrounding physical and meteorological conditions, such as temperature, humidity, and illuminance.

[0520] "Sensing means" refers to devices or equipment used to record environmental information and to acquire data in real time.

[0521] A "data processing means" is a device that analyzes acquired data and has the function of judging, classifying, and transferring information based on established criteria.

[0522] An "emotional analysis tool" is a mechanism that recognizes the emotional state of a user from their facial expressions and voice, and then organizes and evaluates that information.

[0523] A "notification device" is an output device that provides warnings or guidance via visual or auditory means based on detected information.

[0524] "Communication means" refers to the technologies and networks used to perform the process of transferring information to a receiving unit located in a remote location.

[0525] An "information receiving unit" refers to a device that receives, processes, and displays data transmitted via communication means.

[0526] Specific embodiments for carrying out this invention are shown below.

[0527] The server uses multiple sensors to acquire environmental information, collecting data such as temperature, humidity, and illuminance in real time. This collected data is transferred to the server via a communication network. The server uses data processing tools to analyze the received environmental information against predefined criteria and detect anomalies. AWS Lambda can be used as a data analysis platform for this process.

[0528] Furthermore, the server has emotion analysis capabilities and receives the user's facial expression data. This data is acquired by a camera installed in the smart glasses and sent to a cloud service. Here, the Azure Face API is used to analyze the emotional state from the facial expressions. By integrating and processing the emotional data and environmental data, more accurate and flexible anomaly detection becomes possible.

[0529] The device generates optimized voice and visual warnings based on emotional data combined with detected anomaly information, serving as a notification mechanism. This ensures gentle warnings that are less burdensome for the user. Specifically, smart glasses fulfill this role. The notification method adjusts the volume and color of the light according to the user's emotional state.

[0530] Furthermore, it is possible to use communication methods to transmit detailed warning information to information receiving units in remote locations. In this case, the transmitted information includes data on environmental abnormalities and emotional states, encouraging the user's relatives or caregivers to take prompt action.

[0531] As a concrete example, consider a scenario where the room temperature rises and the user is experiencing stress. In this situation, the server considers emotional and environmental data, optimizes the notification sent to the terminal, and initiates a process of remotely transmitting information. An example of a prompt message might be, "The current room temperature is 30°C and the user is showing signs of anxiety. Please advise on appropriate actions to take."

[0532] This system can provide a more flexible warning system that incorporates the user's psychological state than conventional technologies, supporting a more comfortable life for users in care settings.

[0533] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0534] Step 1:

[0535] The server acquires environmental information such as temperature and humidity in real time using sensory means. This data is transmitted via a communication network from devices such as smart glasses. The input data is acquired as multiple pieces of environmental information and formatted by the server's data processing module. The output is then formatted in a way that allows for comparison with reference values.

[0536] Step 2:

[0537] The server uses data processing to detect anomalies in the previously acquired environmental information. Here, it performs calculations by comparing the acquired data with pre-set thresholds. If an anomaly is detected, the corresponding information is flagged as an anomaly. The output includes whether or not an anomaly was detected and detailed information about it.

[0538] Step 3:

[0539] The server uses emotion analysis tools to acquire the user's facial expression data and recognize their emotional state. This process uses images captured through the smart glasses' camera. It receives facial expression data as input and analyzes it using emotion recognition software (e.g., Azure Face API). The output is data that identifies the user's emotional state.

[0540] Step 4:

[0541] The server integrates and analyzes anomaly information and the user's emotional state to generate optimized notification content. Comprehensive data analysis is performed to determine the most appropriate notification format (a combination of sound and light) for the user. The output consists of notification parameters designed to accurately convey information in a way that does not stress the user.

[0542] Step 5:

[0543] The device delivers optimized notifications through its notification system. The smart glasses adjust the audio and visual effects, issuing warnings in a gentle tone. In doing so, the volume and color of the warning are automatically adjusted based on the user's emotional state. The output is a notification that is considerate of the user's emotional response.

[0544] Step 6:

[0545] The terminal uses communication to transmit detailed warning information to a remote information receiving unit. The transmitted information includes abnormal information and the user's emotional state. This allows family members and caregivers to immediately understand the situation and take appropriate action. The output is a display of the data by the remote information receiving unit.

[0546] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0547] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0548] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0549] [Fourth Embodiment]

[0550] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0551] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0552] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0553] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0554] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0555] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0556] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0557] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0558] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0559] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0560] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0561] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0562] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0563] The system based on this invention mainly consists of a sensor device, a server that processes data, and a terminal that handles alarms and notifications. The sensor device periodically measures room environmental information, particularly temperature, and transmits this data to the server. Upon receiving this data, the server performs processing to check whether the temperature is within a preset range. If the temperature is detected to be outside the range, the server determines this to be an anomaly and immediately sends an alarm signal to the terminal.

[0564] When the terminal receives a signal from the server, it activates an alarm using its built-in speaker and lighting to notify the user of the anomaly. Furthermore, it transmits information about the anomaly to family members and related parties living remotely via text messages and application notifications using the internet or telephone lines. In this way, both nearby users and remote parties can simultaneously understand the situation.

[0565] As a concrete example, consider a scenario where the room temperature suddenly rises to 30°C during the daytime in summer. At this time, a sensor measures the temperature and transmits the data to a server. The server compares this temperature to a pre-set appropriate temperature range (for example, 22°C to 28°C), and if it detects an abnormality, it instructs the terminal to activate an alarm. The terminal starts an audible alarm and simultaneously sends a warning message such as "The current temperature is 30°C" to registered family members living far away. The user is immediately aware of the abnormality through the alarm and can take appropriate action, and family members living far away can also check the situation and offer advice. This invention provides a mechanism that can effectively reduce risks related to temperature.

[0566] The following describes the processing flow.

[0567] Step 1:

[0568] A sensor measures the room temperature. The sensor automatically activates every 5 minutes to read the current temperature. This temperature data is converted into a digital signal and sent to a server.

[0569] Step 2:

[0570] The server receives temperature data transmitted from the sensor. The server stores the received data in a database and records it with a time stamp to allow comparison with past data.

[0571] Step 3:

[0572] The server analyzes the temperature data it receives. The server uses a data analysis module to check if the current temperature is within the set threshold range (e.g., 22°C to 28°C). If the temperature is outside the set range, it proceeds to the next step.

[0573] Step 4:

[0574] The server detects an anomaly when a threshold is exceeded. Upon detecting an anomaly, the server records the nature and time of the anomaly and initiates a process to generate an alarm signal.

[0575] Step 5:

[0576] The server sends an alarm signal to the terminal. The server provides instructions to the connected terminal to activate an audio or visual alarm. It also generates a message for remote notification.

[0577] Step 6:

[0578] The device receives an alarm signal and notifies the user. The device activates its built-in speaker and sounds an alarm. In addition, a visual warning is displayed on the screen.

[0579] Step 7:

[0580] The device sends notifications to family members and other relevant parties who live remotely. The device also sends warning messages to registered phone numbers and email addresses via the internet connection. These messages include the current temperature and indicate that an anomaly has occurred.

[0581] Step 8:

[0582] The user receives an alarm, checks it, and takes appropriate action. For example, they might adjust the air conditioner settings or open a window to regulate the room temperature. They can also receive advice from family members remotely and take additional action.

[0583] (Example 1)

[0584] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0585] Conventional environmental monitoring systems have drawbacks, such as delayed anomaly detection and difficulty in real-time notification to relevant parties in remote locations. This can hinder a rapid response to changes in the indoor environment.

[0586] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0587] In this invention, the server includes a measuring means for collecting environmental information, a data processing means for analyzing the data collected by the measuring means and detecting anomalies, and a notification means for notifying the user and relevant parties of the anomaly information detected by the data processing means. This makes it possible to quickly detect environmental anomalies and immediately notify the user and remote locations.

[0588] "Measurement means" refers to technical means for physically collecting environmental information and converting it into data.

[0589] A "data processing means" is a means that has the technical function of analyzing collected data to detect anomalies.

[0590] "Notification means" refers to technical means that use voice, light, and communication means to notify users and relevant parties of detected abnormal information.

[0591] A "physical alarm device" is a device that uses sound and light to signal an anomaly.

[0592] "Network communication means" refers to communication technologies that use the internet or telephone lines to transmit information to remote locations.

[0593] The embodiment for carrying out the invention is a system aimed at monitoring ambient temperature in real time and immediately detecting and notifying of any abnormalities. This system is composed of a sensor device as a measurement means, a server as a data processing means, and a terminal as a notification means.

[0594] The server receives temperature data acquired from the sensor device. Common environmental sensors are used; for example, a typical electronic thermometer is used as the temperature sensor. The server analyzes the data using the Python programming language and detects an anomaly if the temperature exceeds a set range.

[0595] When the terminal receives a signal from the server, it uses its built-in speaker and LEDs to notify the user of the anomaly. The terminal can be a mobile device such as a smartphone or tablet, and an application is run as needed. This application provides sound and light notifications upon receiving a notification. Furthermore, it has the functionality to send messages to remote family members and related parties via the internet or telephone lines.

[0596] As a concrete example, consider a scenario where the indoor temperature suddenly rises to 30°C during the summer. In this case, the sensor sends the data to the server. The server detects that the temperature has exceeded the set safe temperature range (e.g., 22°C to 28°C) and sends an instruction to the terminal to activate an alarm. The terminal can send a message to a registered email address or phone number along with an audio alarm, allowing the user and their associates to immediately understand the change in status and take an appropriate response.

[0597] Examples of prompts used when evaluating this system or suggesting new features using a generative AI model might be questions in the form of, "Please tell me how to optimize the temperature monitoring system." This system provides a temperature monitoring and alarm solution that combines agility and reliability.

[0598] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0599] Step 1:

[0600] The sensor device continuously measures ambient temperature data periodically. For example, it acquires the room temperature every 10 seconds. This measurement result is the input data. The data is output as the latest temperature value and sent to the server. Specifically, the sensor detects the temperature, converts that value into a digital signal, and transmits it to the server via the network.

[0601] Step 2:

[0602] The server receives temperature data transmitted from the sensor. This received data becomes the input. The server analyzes the data using a program and performs data processing to compare it with a set safe temperature range (e.g., 22°C to 28°C). Based on the comparison, it determines whether the data is outside the range, and the result of that determination is obtained as output. Specifically, a Python script is launched, and a process is executed that checks whether the temperature exceeds the range using an if statement.

[0603] Step 3:

[0604] The server generates an alarm signal if the analysis results indicate that the temperature is outside the set range. Here, it performs calculations to create an anomaly detection signal based on the judgment result, and that alarm signal is output. Specifically, the generated signal is sent to the terminal as an HTTP POST request via the REST API.

[0605] Step 4:

[0606] The terminal receives an alarm signal sent from the server. This signal becomes the input data. Based on this signal, the terminal generates an audio alarm through its built-in speaker and flashes an LED light. This is the output from the terminal. Specifically, the application is triggered by the signal, which sounds an alarm and starts providing a visual notification.

[0607] Step 5:

[0608] Users and stakeholders receive a warning message via the network, along with a notification from their device. This transmitted information serves as input data. The application uses this data to send emails and SMS messages, disseminating information to people in remote locations. This is the final output. Specifically, the application on the device sends a message to pre-configured contacts, and the process unfolds to ensure that the notification is distributed.

[0609] (Application Example 1)

[0610] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0611] In the living spaces of the elderly, there is a need for technology that can prevent health risks caused by sudden temperature changes, while also enabling family members and caregivers living far away to immediately understand the situation and take appropriate action. Current systems struggle to quickly detect abnormal temperature changes and provide appropriate notifications, resulting in insufficient real-time information sharing with relevant parties in remote locations. Therefore, improvements are needed to ensure the safety of users.

[0612] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0613] In this invention, the server includes a device for acquiring environmental data, a processing device for determining anomalies based on the environmental data acquired by the device, a device for notifying the anomaly information determined by the processing device, and a device for the notification device to transmit warning information to an information receiving device located at a distance via communication technology. This makes it possible to immediately detect temperature anomalies and notify nearby and distant stakeholders in real time.

[0614] An "environmental data acquisition device" is a sensor device that continuously measures environmental information such as indoor temperature and humidity and supplies that data to a processing device.

[0615] A "processing device for determining abnormalities" is a computer system that compares acquired environmental data with pre-set reference values ​​to identify abnormalities.

[0616] A "notification device" is a device that uses sound or visual means to communicate detected abnormal information to users or relevant parties in remote locations.

[0617] A "device that transmits warning information via communication technology" is a communication device that uses the internet or telephone lines to transmit abnormal information to a receiving device located at a distance.

[0618] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to analyze a situation based on accumulated data and proposes the optimal course of action.

[0619] This system uses temperature sensors to acquire environmental data. The sensors, such as the DHT22, are temperature measuring devices that continuously monitor indoor temperature and humidity, transmitting the data to an AWS cloud server via a small computer like a Raspberry Pi. A Python script runs on the server, comparing the received environmental data to a pre-defined temperature range to determine if an anomaly is present. If an anomaly is detected, the server generates warning information via a notification device and uses communication technology to send push notifications to nearby information receiving devices and the smartphones of relevant personnel in remote locations. This notification process utilizes both audible and visual means.

[0620] Furthermore, the server uses a generative AI model to suggest the optimal course of action in the event of an anomaly. The AI ​​model learns from past data and analyzes the current situation to provide a concrete action plan. This result is displayed to users and stakeholders through a smartphone app.

[0621] For example, if the air conditioner malfunctions and the room temperature rises to 32°C, the server will recognize this as an anomaly and, along with a warning such as "Please check the air conditioner," will suggest a quick repair procedure for the air conditioner based on past data. In this case, the input prompt to the generated AI model would be something like, "Please tell me the appropriate course of action when the room temperature exceeds 32°C."

[0622] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0623] Step 1:

[0624] The sensor measures the indoor temperature and humidity. The data acquired by the sensor is input to the Raspberry Pi. The Raspberry Pi collects this data and transfers it to an AWS cloud server for further processing.

[0625] Step 2:

[0626] The server receives temperature and humidity data. The server analyzes this data using a Python script and compares it to a pre-set reference value (for example, a temperature range of 22°C to 28°C). This data comparison process determines whether there is an anomaly and outputs the result.

[0627] Step 3:

[0628] If an anomaly is detected, the server generates a warning message. This message includes details such as the type of anomaly and its location. The generated warning message is transmitted using communication technology to nearby information receiving devices and to the smartphones of users in remote locations.

[0629] Step 4:

[0630] The terminal receives warning information from the server. Based on this information, the terminal either sounds an alarm or displays a visual warning message on the screen. It also displays specific guidelines on the actions that should be taken.

[0631] Step 5:

[0632] The server utilizes a generative AI model to analyze anomalies. This model generates appropriate countermeasures considering past data and the current situation, and provides these to the user as prompt messages. For example, a message such as "Please tell me the appropriate countermeasures when the room temperature exceeds 32°C" might be used, providing appropriate instructions to the user.

[0633] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0634] This invention relates primarily to a system comprising a sensor device, a server for data processing, a terminal responsible for alarms and notifications, and an emotion engine for recognizing user emotions. The sensor device periodically measures environmental information, particularly temperature, and transmits this data to the server. The server receives this data and detects anomalies by comparing it with a set threshold. Furthermore, the emotion engine reads the user's emotional state, allowing the server to combine temperature data and emotion data for more precise anomaly detection.

[0635] The device optimizes alarms for the user, taking into account information from the emotion engine. Based on the emotion analysis results, the device adjusts the volume and content of the alarm, delivering information in a way that reduces user stress. Furthermore, it can send detailed warning messages, including the user's emotional state, to family members living far away, encouraging more appropriate responses.

[0636] As a concrete example, consider a scenario where the room temperature suddenly rises to 30°C, and the user simultaneously displays an anxious expression. A sensor measures the temperature at 30°C and sends it to the server. The server detects an anomaly because it exceeds the normal threshold, and at the same time, the emotion engine recognizes the user's state of anxiety. Based on this, the server quickly sends an alarm signal and emotion data to the terminal. The terminal sounds an alarm in a gentler tone than usual and displays a calm visual warning to reduce stress on the user while still drawing their attention. Additionally, it sends information to family members remotely, along with the abnormal temperature situation, stating that "the user is feeling anxious."

[0637] This system enables flexible responses that take user emotions into consideration, resulting in more appropriate temperature control and support.

[0638] The following describes the processing flow.

[0639] Step 1:

[0640] The sensor measures the room temperature. The sensor automatically activates at regular intervals, reads the current temperature, and sends it to the server.

[0641] Step 2:

[0642] The server receives temperature data transmitted from the sensor. The server records this data in a database and adds a time stamp so that it can be compared with past data.

[0643] Step 3:

[0644] The server analyzes the temperature data and checks if it falls within the set temperature range. If the temperature exceeds the set threshold, a flag is set to detect it as an anomaly.

[0645] Step 4:

[0646] The emotion engine recognizes emotions using the user's facial expressions and voice. It collects data through the user's camera and microphone and analyzes it in real time.

[0647] Step 5:

[0648] The server receives emotional data from the emotion engine. Based on this emotional data, the server determines whether the user is feeling stressed or relaxed.

[0649] Step 6:

[0650] The server integrates temperature and emotion data and, if necessary, detects anomalies. If it determines that the user's emotions are being affected by an abnormal temperature, it generates a warning message.

[0651] Step 7:

[0652] The server sends an alarm signal to the terminal, preparing to notify the user. The alarm signal includes information such as voice tone and message content that is adjusted according to the user's emotions.

[0653] Step 8:

[0654] The device sounds an alarm and displays a visual warning. If the user shows signs of anxiety, it provides a calming voice notification and uses stress-reducing colors for the visual display.

[0655] Step 9:

[0656] The device sends notifications to family members and other relevant parties who live remotely. The messages sent include not only the temperature status but also emotional information such as "The user is feeling anxious." This allows family members in remote locations to provide appropriate support.

[0657] (Example 2)

[0658] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0659] In modern living environments, there is a need to respond immediately to environmental changes and ensure a safe and comfortable life. However, conventional systems only detect environmental changes and lack consideration for the psychological state of individual users. As a result, users may experience psychological stress in addition to the abnormality of the environment. Therefore, there is a need to develop a system that can detect environmental changes and provide optimal responses while also considering the emotional state of the user.

[0660] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0661] In this invention, the server includes a collection means for collecting environmental information, an analysis means for detecting anomalies by comparing the environmental information acquired by the collection means with a threshold set based on that information, and a recognition means for analyzing the user's emotional state and integrating it with the anomaly judgment made by the analysis means. This enables not only the detection of environmental changes but also flexible warnings and notifications that respond to the user's emotional state.

[0662] "Environmental information" refers to data that indicates the physical conditions and changes in a specific space, such as temperature, humidity, and atmospheric pressure.

[0663] "Collection means" refers to a method or device for collecting environmental information using sensor devices or acquisition mechanisms.

[0664] "Analysis means" refers to a processing method or apparatus for analyzing collected environmental information and detecting anomalies based on established criteria.

[0665] A "threshold" is a numerical value or condition that serves as a standard for determining abnormalities in environmental information.

[0666] "Recognition means" refers to a technology or device for analyzing a user's emotional state and integrating the results with other information.

[0667] "Warning generation means" refers to a method or apparatus for creating warning messages or notification content based on recognized information.

[0668] "Notification means" refers to a method or device for transmitting a generated warning or notification to a specific recipient.

[0669] "Communication means" refers to a method or device for transmitting information to different locations through a network or other medium.

[0670] This invention is an integrated system that combines environmental monitoring with the visualization of the user's psychological state, enabling the selection and notification of appropriate alarms. The system consists of sensor devices, a server, terminals, and an emotion engine that analyzes the user's emotional state.

[0671] The sensor device continuously collects environmental information, primarily temperature. For example, it measures room temperature every five minutes and transmits the data digitally using protocols such as HTTP or MQTT. The measured data is transferred to the server along with an accurate timestamp.

[0672] The server is specifically a high-performance computer that compares received temperature data with programmatically set thresholds. If an anomaly is detected, the emotion engine further analyzes the user's emotional state. The emotion engine uses an AI model to determine whether the user is anxious based on image and audio data.

[0673] The device processes alarm signals sent from the server and adjusts the notification method based on the user's emotional data. It modifies the volume and message content according to the emotional analysis results, delivering information in a stress-reducing manner. For example, if it detects both rising temperature and user anxiety simultaneously, it displays a notification in a gentle tone and a calming visual message. It also sends detailed warning messages to the user's family members located remotely, prompting them to take appropriate action regarding the anomaly.

[0674] As a concrete example, consider a situation where the room temperature suddenly rises to 30°C, causing the user to become anxious. In this case, a sensor device measures the temperature at 30°C and transmits it to the server. The server detects the temperature exceeding the threshold as an anomaly, and the emotion engine notifies the user of their anxiety. Based on this, the device provides appropriate notifications through voice and visual means, and remotely communicates information such as "the user is feeling anxious" to family members.

[0675] An example of a prompt for a generating AI model is, "When the user is feeling anxious, what kind of alarm would be effective in response to a rising temperature?" In this way, the system considers both the user's emotions and the environmental situation simultaneously to provide the optimal judgment and response.

[0676] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0677] Step 1:

[0678] The sensor device acquires ambient temperature data. The input is temperature information from the surrounding environment, and the output is temperature data converted into a digital format. Based on this data, the sensor measures the current temperature every 5 minutes, generates time-stamped data, and prepares to send it to the server.

[0679] Step 2:

[0680] The server receives temperature data transmitted from the sensor device. The input is the temperature data sent from the sensor, and the output is timestamped temperature data stored in the database. The server checks the received data and prepares it for data analysis, comparing it to thresholds maintained within the program.

[0681] Step 3:

[0682] The server compares the received temperature data with a set threshold. The input is temperature data stored in the database, and the output is a flag indicating that the data is "abnormal." Specifically, when the temperature exceeds the threshold, the system flags this data as abnormal and generates a trigger to proceed to the next process.

[0683] Step 4:

[0684] The server acquires and analyzes user emotion data. Input is user facial expression and voice data, and output is the identification of the user's emotional state (e.g., anxiety, reassurance). Using an emotion engine, the acquired data is analyzed, and a generative AI model is used to determine the user's emotion.

[0685] Step 5:

[0686] The server integrates anomaly flags and emotion recognition results to generate the optimal warning signal. The inputs are anomaly flags and emotion states, and the output is the warning signal to be communicated to the user. This integration allows the system to determine the most appropriate type and tone of alarm for the user.

[0687] Step 6:

[0688] The terminal receives a warning signal from the server and prepares a notification. The input is the warning signal sent from the server, and the output is the warning information provided to the user through audio or visual means. The terminal uses gentle melodies and visual effects to convey effective attention while minimizing emotional stimuli to the user.

[0689] Step 7:

[0690] The server sends alert information to the user's family. The input is the detailed alert content generated, and the output is the message received by the remote recipient. This includes the user's current emotional state and environmental circumstances, prompting appropriate action.

[0691] (Application Example 2)

[0692] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0693] In modern care settings, there is a need to quickly and accurately understand the user's environmental conditions and emotional state. However, conventional systems only analyze environmental data and emotional state individually, making it difficult to integrate and consider both for flexible responses. Therefore, technology is needed to effectively reduce users' anxiety and stress and provide comfortable care.

[0694] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0695] In this invention, the server includes a sensor means for acquiring environmental information, a data processing means for detecting anomalies based on the acquired environmental information, and an emotion analysis means for recognizing the user's emotional state. This makes it possible to comprehensively analyze environmental data and the user's emotional state and generate optimized notifications.

[0696] "Environmental information" refers to data that quantifies the surrounding physical and meteorological conditions, such as temperature, humidity, and illuminance.

[0697] "Sensing means" refers to devices or equipment used to record environmental information and to acquire data in real time.

[0698] A "data processing means" is a device that analyzes acquired data and has the function of judging, classifying, and transferring information based on established criteria.

[0699] An "emotional analysis tool" is a mechanism that recognizes the emotional state of a user from their facial expressions and voice, and then organizes and evaluates that information.

[0700] A "notification device" is an output device that provides warnings or guidance via visual or auditory means based on detected information.

[0701] "Communication means" refers to the technologies and networks used to perform the process of transferring information to a receiving unit located in a remote location.

[0702] An "information receiving unit" refers to a device that receives, processes, and displays data transmitted via communication means.

[0703] Specific embodiments for carrying out this invention are shown below.

[0704] The server uses multiple sensors to acquire environmental information, collecting data such as temperature, humidity, and illuminance in real time. This collected data is transferred to the server via a communication network. The server uses data processing tools to analyze the received environmental information against predefined criteria and detect anomalies. AWS Lambda can be used as a data analysis platform for this process.

[0705] Furthermore, the server has emotion analysis capabilities and receives the user's facial expression data. This data is acquired by a camera installed in the smart glasses and sent to a cloud service. Here, the Azure Face API is used to analyze the emotional state from the facial expressions. By integrating and processing the emotional data and environmental data, more accurate and flexible anomaly detection becomes possible.

[0706] The device generates optimized voice and visual warnings based on emotional data combined with detected anomaly information, serving as a notification mechanism. This ensures gentle warnings that are less burdensome for the user. Specifically, smart glasses fulfill this role. The notification method adjusts the volume and color of the light according to the user's emotional state.

[0707] Furthermore, it is possible to use communication methods to transmit detailed warning information to information receiving units in remote locations. In this case, the transmitted information includes data on environmental abnormalities and emotional states, encouraging the user's relatives or caregivers to take prompt action.

[0708] As a concrete example, consider a scenario where the room temperature rises and the user is experiencing stress. In this situation, the server considers emotional and environmental data, optimizes the notification sent to the terminal, and initiates a process of remotely transmitting information. An example of a prompt message might be, "The current room temperature is 30°C and the user is showing signs of anxiety. Please advise on appropriate actions to take."

[0709] This system can provide a more flexible warning system that incorporates the user's psychological state than conventional technologies, supporting a more comfortable life for users in care settings.

[0710] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0711] Step 1:

[0712] The server acquires environmental information such as temperature and humidity in real time using sensory means. This data is transmitted via a communication network from devices such as smart glasses. The input data is acquired as multiple pieces of environmental information and formatted by the server's data processing module. The output is then formatted in a way that allows for comparison with reference values.

[0713] Step 2:

[0714] The server uses data processing to detect anomalies in the previously acquired environmental information. Here, it performs calculations by comparing the acquired data with pre-set thresholds. If an anomaly is detected, the corresponding information is flagged as an anomaly. The output includes whether or not an anomaly was detected and detailed information about it.

[0715] Step 3:

[0716] The server uses emotion analysis tools to acquire the user's facial expression data and recognize their emotional state. This process uses images captured through the smart glasses' camera. It receives facial expression data as input and analyzes it using emotion recognition software (e.g., Azure Face API). The output is data that identifies the user's emotional state.

[0717] Step 4:

[0718] The server integrates and analyzes anomaly information and the user's emotional state to generate optimized notification content. Comprehensive data analysis is performed to determine the most appropriate notification format (a combination of sound and light) for the user. The output consists of notification parameters designed to accurately convey information in a way that does not stress the user.

[0719] Step 5:

[0720] The device delivers optimized notifications through its notification system. The smart glasses adjust the audio and visual effects, issuing warnings in a gentle tone. In doing so, the volume and color of the warning are automatically adjusted based on the user's emotional state. The output is a notification that is considerate of the user's emotional response.

[0721] Step 6:

[0722] The terminal uses communication to transmit detailed warning information to a remote information receiving unit. The transmitted information includes abnormal information and the user's emotional state. This allows family members and caregivers to immediately understand the situation and take appropriate action. The output is a display of the data by the remote information receiving unit.

[0723] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0724] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0725] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0726] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0727] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0728] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0729] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0730] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0731] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0732] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0733] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0734] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0735] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0736] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0737] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0738] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0739] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0740] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0741] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0742] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0743] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0744] The following is further disclosed regarding the embodiments described above.

[0745] (Claim 1)

[0746] Means for acquiring environmental information,

[0747] A processing means for detecting anomalies based on environmental information acquired by the acquisition means,

[0748] A notification means for notifying abnormal information detected by the processing means,

[0749] A system that includes this.

[0750] (Claim 2)

[0751] The system according to claim 1, characterized in that the notification means generates warning information at a local location and provides notification using sound or light.

[0752] (Claim 3)

[0753] The system according to claim 1, characterized in that the notification means transmits warning information to an information receiving device located in a remote location via a communication means.

[0754] "Example 1"

[0755] (Claim 1)

[0756] Measurement means for collecting environmental information,

[0757] A data processing means analyzes the data collected by the aforementioned measurement means and detects anomalies,

[0758] A notification means for notifying users and relevant parties of abnormal information detected by the data processing means,

[0759] A system that includes this.

[0760] (Claim 2)

[0761] The system according to claim 1, characterized in that the notification means uses a physical alarm device to signal an abnormality with sound and light.

[0762] (Claim 3)

[0763] The system according to claim 1, characterized in that the notification means transmits alarm information to a remote receiving device via network communication means.

[0764] "Application Example 1"

[0765] (Claim 1)

[0766] A device for acquiring environmental data,

[0767] A processing device that determines an anomaly based on environmental data acquired by the aforementioned device,

[0768] A device that notifies abnormal information determined by the aforementioned processing device,

[0769] The notification device is a device that transmits warning information to an information receiving device located at a distance via communication technology,

[0770] A system that includes this.

[0771] (Claim 2)

[0772] The system according to claim 1, characterized in that the notification device generates warning information at a nearby location and provides notification using sound or light.

[0773] (Claim 3)

[0774] The system according to claim 1, characterized in that the processing device proposes countermeasures using a generated AI model based on the generated abnormal information.

[0775] "Example 2 of combining an emotion engine"

[0776] (Claim 1)

[0777] means of collecting environmental information,

[0778] An analysis means for detecting anomalies by comparing the environmental information acquired by the aforementioned collection means with a threshold set based on that information,

[0779] A recognition means that analyzes the user's emotional state and integrates it with the abnormality judgment made by the analysis means,

[0780] Based on the results of the recognition means, a warning generation means for constructing an optimal warning,

[0781] A notification means for locally transmitting the warning generated by the warning generation means,

[0782] A communication means that transmits a warning to a remote location such as the user's family using the aforementioned notification means,

[0783] A system that includes this.

[0784] (Claim 2)

[0785] The system according to claim 1, wherein the notification means is characterized by adjusting the warning information using voice or visual means according to the user's emotional state.

[0786] (Claim 3)

[0787] The system according to claim 1, wherein the communication means transmits detailed warning information, including emotional states, to a remote information receiving device.

[0788] "Application example 2 when combining with an emotional engine"

[0789] (Claim 1)

[0790] Sensor means for acquiring environmental information,

[0791] A data processing means that performs anomaly detection based on acquired environmental information,

[0792] A means of sentiment analysis to recognize the emotional state of the user,

[0793] A notification means that optimizes notifications based on abnormal information and emotional state,

[0794] A communication means for sending notifications to an information receiving unit located in a remote location,

[0795] A system that includes this.

[0796] (Claim 2)

[0797] The system according to claim 1, characterized in that the notification means generates warning information adjusted to take into account emotional state at a local location and provides notification using sound or light.

[0798] (Claim 3)

[0799] The system according to claim 1, characterized in that the communication means transmits abnormal information and detailed warning information, including the emotional state of the user, to an information receiving unit located in a remote location. [Explanation of symbols]

[0800] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A device for acquiring environmental data, A processing device that determines an anomaly based on environmental data acquired by the aforementioned device, A notification device that notifies abnormal information determined by the processing device, The notification device is a device that transmits warning information to an information receiving device located at a distance via communication technology, A system that includes this.

2. The system according to claim 1, characterized in that the notification device generates warning information at a nearby location and provides notification using sound or light.

3. The system according to claim 1, characterized in that the processing device proposes countermeasures using a generated AI model based on the generated abnormal information.

Images